Published by

Signs of Change: Video Chatting Software to Help the Hearing-Impaired

Email a Friend

Sure, cell phone companies would love you to believe that video chatting from your mobile phone will change your life—but what if it really did? This week, engineers at the University of Washington will conclude testing on software that makes it possible for hearing-impaired mobile users to do just that: communicate through video.

In June, the iPhone4's video chat function blew consumer's minds, but the tool wasn't sophisticated enough to allow hearing-impaired users to communicate through American Sign Language (ASL). Enter Eve Riskin, an electrical engineering professor at U-Dub, whose 2005 study on video compression led to the creation of MobileASL. The program allows hearing-impaired users to communicate fluently via real-time video chatting on cell phones.

Thanks to a National Science Foundation grant, Riskin's team, along with Cornell University researchers, began working on ways to compress video on computers. In 2008, as cell phone technology made video more and more possible, Riskin's team seized the opportunity to work with cell phones - and put them in the hands of student testers. "We never expected to use cell phones, so it became an interesting problem to solve," Riskin says.

Fluency in ASL hinges on clarity of motion, and also on facial expressions.  Proper nouns that don't have specific signs are spelled with fingers, and facial expressions are critical to giving meaning to conversation, according to Matt Huenerfauth, head of CUNY's Linguistic and Assistive Technologies Laboratory.  These two elements are easily lost over choppy cell phone video connections.

"Probably a good analogy would be trying to overhear a conversation in a noisy environment," Huenerfauth says.

Riskin adds that people often wonder why hearing-impaired students don't just stick to text-messaging. "Texting can easily get confusing especially if you're a native ASL speaker texting in your second language," she says. "Many people who have ASL have limited English. Imagine texting in French!"

Though video chatting uses significantly more cell phone bandwidth than a phone call, the study's biggest hurdle wasn't compressing the video to be transmitted. It was the limited functionality of the cell phones themselves. Riskin's team also had to sort out problems with cell phone battery life, since video chats use more power, though the latest generation of video-chat-enabled phones will likely last longer.

As all kinds of smartphone apps continue to emerge—from apps that allow you to "tickle" someone or let you pretend-drink a beer—Huenerfauth says he's not yet sure what the timetable is for widespread usage of MobileASL. "I would say two to five years," Huenerfauth says. "Probably closer to five."