Graph data science-driven framework to aid auditory and speech impaired individuals by accelerating sign image analysis and knowledge relegation through deep learning technique

In India, the prevalence of speech and hearing impairments is a major public health concern due to the negative consequences it has on a large number of people. So far, researchers have ignored the linguistic features of sign language and have treated spoken and sign language as directly identical. Thus, to facilitate real-time communication between sign language users and non-sign language users, a state-of-the-art end-to-end system for sign language recognition and translation is needed. The system combines sign language translation (SLT) and sign language recognition (SLR) approaches to ensure precise real-time recognition of sign gestures. A Convolution Neural Network is used in the proposed study as a deep learning model to train a huge dataset of hand gestures and perform image analysis utilizing the MediaPipe library for landmark estimation and identification. The experiment is conducted by a thorough analysis of a large dataset made up of 52,000 sign images. Using graph data science and NetworkX, the SLT module in the proposed model creates a directed weighted graph of Indian Sign Language gestures and their corresponding English translations, which it uses to convert gesture sequences into meaningful English phrases. By using knowledge relegation, the study compares the suggested model with existing models, which show that they are dependable as tools for communication and enable people with speech and hearing impairments to have natural conversations with non-sign language users. It achieves a remarkable accuracy of 98.64%, highlighting the significance and potential of combining SLR and SLT techniques.