A 20-year-old Indian engineering student, Priyanjali Gupta, has used AI to develop software that translates sign language into words in real time.
Priyanjali Gupta, a third-year student at the Vellore Institute of Technology in the state of Tamil Nadu (India), has developed software that uses a webcam and AI-based technology to identify the gesture of sign language and translate it instantly into words. For the moment, this innovative software only translates American Sign Language (each language signing differently). Available as open source, it is very likely that the scientific community will take up this innovation to duplicate it, adapt it to other sign languages and improve it.
Solving the world’s problems
Priyanjali shared the creation of her software on the LinkedIn platform, reaching over 66 000 likes. She says: “I am self-taught with a keen interest in development and research. I look for opportunities to apply my technical knowledge to build things and solve current problems of the world.” Inclusive technology is where the inventive future engineer started to develop this very promising project.
Communicating in real time
Her system successfully converts sign language gestures into text by analyzing the movements of several body parts, such as the arms and fingers. The image recognition technology that the student used is based on the digitization of the sign language of several people. On GitHub, the world’s largest and most advanced development platform, where the software code was posted, the post went viral. This AI software will provide a dynamic way to communicate with people who are deaf or hard of hearing, in real time. To begin with, it can convert the movements of about six gestures – Yes, No, Please, Thank you, I love you and Hello. A lot more data is needed to create a more complete model, but as a proof of concept, the system works. Priyanjali Gupta is currently investigating the use of long-term memory networks, an artificial recurrent neural network, to integrate multiple images into the software.
A successful first step in bridging the communication gap between deaf and hard of hearing people and their interlocutors and highlighting the progress that still needs to be made in terms of inclusiveness, to promote technological accessibility for all.
#India, #languageofsigns, #inclusiveness, #technology, #IA,
