Technology Catches Up with Sign-to-Spoken-Word Translation

Like many young people in Singapore and elsewhere, 25-year-old Quek Kai Yu sometimes finds it difficult to communicate with his parents. But in his case, the generation gap isn't always to blame. Quek has normal hearing, but both his parents are deaf and mute—and they speak two different sign languages, to boot. "My mom uses the Singapore style, but my dad uses an ancient Chinese one," says Quek. "The same gesture could mean something different in both languages."

For Quek and his family, the SLinto dictionary may one day come in handy. SLinto, which means "sign language into dictionary," is the world's first online, crowdsourced database for signs. "If you want to show dog in American Sign Language, you type it into Google, and a video shows up," says SLinto's founder, Japanese entrepreneur Junto Ohki. But it's much harder to do the reverse, that is, to describe the hand gestures and figure out what they mean.

Herein lies SLinto's ingenuity. "If you know the sign language but not the meaning, you can use our dictionary to search for it," says Ohki.

To obtain sign-to-spoken-word translations, users search SLinto's database—which currently comprises roughly 3,200 words from Japanese sign language—via its special online keyboard. Instead of the alphabet, keys show pictorial representations of the hands or parts of the body. Users choose from the 18 Hand and nine Body keys available to indicate, respectively, which fingers create the unknown sign and where they're placed against the body (for example, the chest, forehead or beside the ear). Four keys in total to represent both hands.

The search results throw up a number of corresponding words for a sign. By watching the short video clip that accompanies each word, users can compare the orientation, hand shape and movement of the hands with the sign they're trying to translate, and thereby determine its correct meaning.

It may not work 100 percent of the time. "It cannot be assumed that there is always sign-word correspondence in meaning," says Robert Adam, who leads an expert group on sign language and deaf studies at the World Federation of the Deaf.

SLinto relies on users to contribute words and videos, as well as to vote on the accuracy of translations using a Fit/Unfit button.

Later this year, SLinto's database will expand to include American Sign Language, and Ohki hopes it will eventually encompass as many as possible of the world's 126 sign languages, used by roughly 70 million people worldwide: "I hope SLinto will be the platform for all sign languages to share words with each other."