Sign Language is the visual means of communication among the people with hearing and speaking impairments. In our project, Nepali Sign Language Recognition and Translation in Text and Speech using Open CV and CNN, we use Laptop’s webcam to capture and stream hand gestures and provide it as an input to the machine learning model, which uses Convolutional Neural Network to recognize Nepali Sign Language gestures. A Convolutional Neural Network (CNN) using Tensorflow was created. CNN is a type of Artificial Neural Network (ANN), which is widely used for image or object recognition and classification. For each Nepali Sign Language gestures, 2100 images were captured and stored in grayscale format and further preprocessing of the dataset was done. Then the machine learning model in Tensorflow was trained with the collected dataset. After training, the model was tested with the unseen data to get an accurate measure of how the model performs. Then parameter tuning of the model was done to see if its accuracy and to see if it can be improved in any way. Finally, Nepali Sign Language was predicted and the required output was extracted to the UI that displayed the text and also converted it into speech using text to speech conversion library.
Keywords: Convolutional Neural Network (CNN), Deep Learning, Machine Learning, Open CV, Tensorflow, Text-to-speech etc