Main Goal and Objectives

This project is called “Bridge: an AI powered sign language interpreter”. The main goal of this project is to create a working prototype that can translate American Sign Language into English.

It is important to create something that is not limited by the confines of computer science and experimentation but also something that is tangible and could be utilized by worldwide sign language users. Although still in its stages of infancy, BRIDGE aims to enhance communication using emerging technologies and making them accessible to those who needs them.

To achieve this goal, the following objectives must be met:

• Have at least 15 simple phrases which it can translate.
  • Discover several possible methods of training a machine learning model that can scale (as languages have thousands of words)
• Discover an acceptable accuracy for proper use.
  • Define a valid accuracy measurement system.
  • Have an accuracy rate of at least 80%.
• Minimize delays in translation using various optimizers.
  • Prototype must not have translations that have significant delays (>5s)
• Find an effective compromise between natural communication and technological hurdles.
  • Have a user-friendliness rating of at least 8/10.

Critical Tasks

The following are the tasks necessary to do the main goals and objectives:

1. Initial Research
   1. Find meaningful sources of inspiration.
   2. Find systems to better create the software.
      1. Find ways to optimize model training and make it accurate.
   3. Research in developing assistive technologies.
      1. Consult Mentors for guidance.
      2. Talk to people that use sign language.
      3. Consult other professionals (psychologist, linguist, etc.)
2. Develop Software
   1. Train model
      1. Develop movement recording system.
      2. Record all possible words with corresponding action.
      3. Collect data and upload to Kaggle.
      4. Develop proper deep learning architecture.
      5. Start Training model with data.
      6. Optimize model using different optimizers (experiment)
   2. Create UI/UX
      1. Refer to research for possible UI/UX ideas.
      2. Cross-reference to conversations with target users to make schematics.
      3. Design UI/UX and implement them onto software.
   3. Optimize Software
      1. Iteratively improve the user friendliness and accuracy of software
3. Test Software
   1. Prepare software for user-testing.
      1. Export as a testable app (.exe file)
   2. User testing Experiment
      1. Find people that will try out software (focus group)
      2. Reserve a time and place for user-testing.
      3. Collect feedback and measure results.
      4. Analyze Data
4. Present Software
   1. Prepare presentation materials.
   2. Rehearse the presentation.
   3. Deliver the presentation to relevant stakeholders.

Materials, Tools, and Resources

To complete these tasks, the following resources should be made available:

For the development of the Software: