Automated Speech to Sign language Conversion using Google API and NLP

Abstract

Sign Language is a visual language that requires manual modality to express what a person wants to convey. Sign Language has its own grammar and lexicon. It is used by dumb and deaf people to por- tray what they want to say. It acts as a basic mode of communication for deaf and dumb people, without which communication between them and normal people might be difficult. Thus, this paper is focused to develop an automated software system to convert speech to sign language so that the words of a normal person can be conveyed to a deaf/dumb person. This automated system first recognizes the speech, the second converts it to text, third matches tokenized text with the visual sign word library (videos of sign language), fourth concatenates all the matched videos ac- cording to the text recognized and finally display the merged video to the deaf/dumb person. System accuracy is compared with the state-of- the-art approaches and found to be the best.

Keywords: Sign language, Natural Language Processing, Google Speech- to-text API