International Journal of Science, Engineering and TechnologyAuthors: Dr. Harsha R. Vyawahare, Sukhada Shripad Tare, Ashwini Nitin Shingane, Shreya Sunil Shinde, Bhavika Suraj Jain
Abstract: This paper presents a lightweight and practical bidirectional communication system designed to translate between speech and Indian Sign Language (ISL) using machine learning and computer vision techniques. The system operates in two modes: (i) speech-to-ISL translation, where spoken input is converted into text and further mapped into a sequence of ISL alphabet images, and (ii) ISL-to-text and speech translation, where hand gestures captured through a webcam are recognized using a Convolutional Neural Network (CNN) model and converted into readable text and audio output. Unlike existing approaches that rely on complex natural language processing techniques or computationally expensive 3D avatar rendering, the proposed system focuses on simplicity, real-time performance, and ease of implementation. By utilizing a TensorFlow/Keras-based CNN model for gesture recognition and a predefined ISL image dataset for visual representation, the system achieves efficient and accurate translation with low computational requirements. The system is implemented using Python with libraries such as OpenCV, Streamlit, speech_recognition, and pyttsx3, enabling an interactive and user-friendly interface. The proposed solution provides a cost-effective and accessible tool to bridge the communication gap between hearing individuals and the Deaf and Hard-of-Hearing (DHH) community, making it suitable for real-world applications.
DOI: https://doi.org/10.5281/zenodo.19484295
