This Glove Translates Sign Language to Text—And Could Eventually Give Virtual Reality the Sense of Touch

It may not be speedy, but it works. A gloved hand forms letters in sign language, and like magic, the motions are translated to text. "U-C-S-D," the hand slowly spells out in text, referring to the University of California San Diego, where researchers developed the glove.

The achievement is detailed in a video posted this week by UCSD. As it stands now, the glove, which the team built for less than $100 using flexible electronics that are available commercially, can translate the entire American Sign Language alphabet into text. The leather athletic glove (which looks quite similar to what a batter might wear in baseball) uses flexible sensors to detect the movement of the signer's knuckles, which creates a nine-digit binary key (0s and 1s) that corresponds to a letter. The glove also has an accelerometer and a pressure sensor to help distinguish the movements of letters that look similar to one another.

While translating sign language is a practical use for the glove, the engineers at UCSD have more in mind going forward.

"Gesture recognition is just one demonstration of this glove's capabilities," Timothy O'Connor, a nanoengineering Ph.D. student at UC San Diego and the first author of a study on the researchers' work, said in a statement. "Our ultimate goal is to make this a smart glove that in the future will allow people to use their hands in virtual reality, which is much more intuitive than using a joystick and other existing controllers. This could be better for games and entertainment, but more importantly for virtual training procedures in medicine, for example, where it would be advantageous to actually simulate the use of one's hands."

Tracking hand motion could allow users to have better control over objects in virtual reality. The researchers are working on a new version of the glove that will come with a sense of touch: A user will control a robotic or virtual hand that then feeds tactile sensations back to the user's hand.