This project is called “Bridge: an AI powered sign language interpreter”. This project intends to create a prototype ****that can translate International Sign Language into English. Considering that this project is an intersection of different things such as computation and accessibility, I have reached out and talked to important individuals that can help me shape my research.

Preparation

For the past three weeks, I have had Rodolpho Cossovich (Rudi) as my mentor to help me design my research. He is a Professor in NYU Shanghai and specializes in creating assistive technologies. He is currently teaching an Assistive technology course in the school. He is a great pick for a mentor as he has experience in this field and knows some people in communities which I can reach out to.

Another Mentor that I have approached is a current Junior Student in NYUSH named Arghya Sakar. Despite his status as a student, he is an expert in machine learning and have a lot of experience in projects and different possibilities in the field. He is a research assistant for a professor in Purdue University and is well versed in the process of research in computing. I believe he is a great pick as he has a more practical approach in doing research and giving advice.

Details of the meeting

Talking with Rudi the past weeks, I have prepared several things, for example, I needed to show him products or researches similar to mine. This was easy as I already had done this already. Another thing is that I was prepared to show some collaborations that I had from people back home, which gave some context to my relationship with my study.

For Arghya, I had prepared an introduction of what I am doing and the plan that I had to pursue, such as which model I will be using, with what device, logistics of the experiment. I also prepared some questions such as the format of the data I will use and some speech recognition systems I need to find.

Notes

Notes with meeting with Rudi:

• It is difficult to make decisions by yourself for any considerations in designing the product.
• Deaf culture is often politicized and you should include as much voices from the actual community as possible.
• Get to know the actual need for the product, and whether the compromise they will have in using this technology is worth it compared to not having it at all.
• Talk to people from lili’s cafe and the guest adam to verify the project and specifications

Notes with Arghya:

• You can refer to Hugging face for free text2speech models and even large language models, they are opensource and may be beneficial.
• Segmentation of words are referred to as tokens, researching these will be helpful in making accurate predictions