My project is titled Bridge, it is an AI powered sign language interpreter and as the week progresses, I have been considering a lot of the technical directions that I will take moving into the project. Along with this, I have started to communicate with a mentor that might help me improve the considerations of my study.

Learnings from Lili’s Cafe and Rudi’s Sign language Workshop

As Rudi had invited me to come to the workshop and tag along with the cafe, I went to learn more about the things I had to consider, along with validating some ideas that I had, with some new insights about the project.

For the sign language workshop, it was definitely helpful to learn a practical side of sign language. Knowing a little bit of ASL, I realized how similar yet, so different CSL is. This gave me some confusion as to which language to actually choose. Surprisingly, CSL also uses the English alphabet to fingerspell, however it is still different to some extent. I can definitely see some similarities and think this sort of reflects the spatial nature of the language and how it connects to the human innate gesture-based expression.

Another great thing that I learned are the technologies that the deaf practically use. They currently have a real-time translation device that takes any possible audio and writes it down for them, it can either be in English or in Chinese.

In the cafe, I have had the opportunity to talk to the owners and bounce back some of the ideas that I had. I realized how hard it was to explain the technology transfer necessary with the technical limitations of the project. However, with my conversation I have discovered 2 things. One, is that aggregating a sentence first and giving the possible sentences after while making the user choose which one they meant is an acceptable idea. Two, using International Sign language for my model is recommended as it becomes easier to branch into other more specific sign languages. This removes the complications of regionalized dialects.

I have also considered the technical aspects of the project, such as the use of computer vision algorithms to recognize and track hand gestures, as well as natural language processing techniques to interpret and translate sign language into spoken or written language. Additionally, I have started to explore potential partnerships with organizations that specialize in assistive technologies for the deaf community, in order to ensure that the Bridge project aligns with their needs and priorities.

Research and where to place my study.

Over the past week, I have been trying to find similar studies that relate to my own.

As there are a lot of really related topics, I find it difficult to make a unique characteristic of my project as opposed to the other ones I read. Mostly because it feels like that, I am trying to mix different aspects together. And though I think that it is valid enough to make my study unique, I still need something that would make my study standout.

After a lot of reflection, a big focus that I was considering the segmentation system. This was not present amongst the studies that I have read. I think maybe because their methods of testing included only individual words out of the context of an actual sentence. The logistics of the segmentation is really difficult as each movement is not the same duration. At the same time the computer only takes about 30 frames per capture. It is hard to standardize the whole gesture into a specific frame count.