Anyone who has operated a keyboard in virtual reality knows that this is cumbersome and slow. Facebook researchers have developed a clever way of entering text called “pinchtype” that takes advantage of hand tracking, but takes some getting used to.
Users bring the thumb together with another finger of the same hand to choose from a group of letters that correspond to the QWERTY keyboard layout .
An example: If you bring your index finger together with your thumb, you select the group of letters that you tap on the 10-finger system with your index finger.
This should make it easier to get started with the new text input. The fingertips and groups of letters are also color-coded . This helps users who have not internalized the ten-finger system.
If you have selected several letter groups in succession, an AI language model determines a series of words that could correspond to this combination of letters and sorts them in a list according to their probability. Alternatively, users can choose the right word from this list themselves.
This is accomplished through a combination of pinch gestures from the left and right hands and up and down movements of the right hand. A pinch gesture combination is also used to press the spacebar. Both thumbs are brought together with the index fingers.
Pinchtype needs to be learned
Facebook tested the suitability of XR text input in a preliminary study with 14 participants. The criteria were speed and accuracy when first used .
The study participants had to type in 40 sentences. The result: The participants achieved an average speed of 12 words per minute. If they did not correct their mistakes, they would manage 20 words per minute. They also reported that pinchtype is more convenient to use than a floating keyboard.
It was probably tested with an Oculus Quest . The self-sufficient VR glasses support optical hand and finger tracking (see info) . Text entry with Oculus Quest is currently still quite cumbersome: You have to select a letter with the cursor point and bring your thumb and forefinger together to confirm.
Pinchtype should have been introduced this month at the CHI 2020 conference in Honolulu, as well as the new version of the haptic bracelet Tasbi . The event was canceled due to the corona crisis.
Author Profile
- Amram is a technical analyst and partner at DFI Club Research, a high-tech research and advisory firm .He has over 10 years of technical and business experience with leading high-tech companies including Huawei,Nokia,Ericsson on ICT, Semiconductor, Microelectronics Systems and embedded systems.Amram focuses on the business critical points where new technologies drive innovations.
