A project proposal submitted by Mathew Ponon
This project is called “Bridge: an AI powered sign language interpreter”. This is a piece of software that is able to translate American Sign Language into written or spoken English. This software is geared towards people who are hearing or speech impaired who often have problems communicating with people who communicate verbally. Such problems lead them to have communication gaps which often lessen their opportunities. My study aims to create this software to lessen such gaps and facilitate efficient communication. Using emerging technological methods and other research to guide my process, my goal is to create a prototype that translates sign language in an accurate and user-friendly way.
By the end of this course, I will have created a working prototype that can translate simple American sign language phrases into phrases in English. This aim implies several goals:
Aside from this, I aim to develop an effective user interface that best minimizes the communication gap between people communicating. This implies the following goal(s):
This project mainly aims to aid those who use sign language on a day-to-day basis. Despite having 72 million people around the globe using sign language, language barriers are still present as people would only learn sign language if they or a person they know has some form of disability. These language barriers often lead to a lack of access to opportunity or proper education. An established solution to this is often through hiring translators or interpreters to facilitate a conversation, but this is often costly or difficult to attain given the small amount of people that have mastered sign language.
With the emergence of new technologies such as machine learning and artificial intelligence, new methods for aiding the hearing or speech impaired are becoming more feasible. Currently, technological advances in computer vision and machine learning are paving the way to create systems that understand human movement in the virtual space. Some projects have even started to explore different ways to translate sign language into written speech. This is where my study comes in. I aim to add onto this continuous quest to finding efficient ways to bridge gaps in communication for sign language users.
Currently there are no commercialized products in doing this. Mainly because of the scale and complexity of language. Several solutions which have the same goal try to use different methods to interface body movements into language digitally. The technology behind this is often complicated, making access to it difficult. This is why I mainly want to find ways to create an open source and easily to access system for interpreting while being accurate and user friendly.
There are several other projects that aim to give technological solutions to enhance communications amongst sign language users. These range from usage of speech-to-text technologies to articulate use of sensors and wearable technology. The following are relevant to this study: