SignSpeak: Filipino Sign Language to Speech Translator

Abstract

SignSpeak: Filipino Sign Language to Speech Translator is an assistive Android mobile application developed to facilitate communication between deaf students and hearing individuals with limited knowledge of Filipino Sign Language (FSL), particularly in an educational setting. The project combined an Artificial Intelligence Modeling Cycle with a Software/System Engineering Lifecycle. A Multi-Layer Perceptron (MLP) was used for static alphabet and number gestures, while a Long Short-Term Memory (LSTM) network was used for dynamic word gestures. MediaPipe supported landmark extraction; static data were augmented through horizontal flipping, while dynamic sequences were normalized relative to the shoulder landmark. The datasets consisted of 62,400 alphabet images, 24,000 number images, and 9,000 word-gesture videos. Two domain experts, including a certified FSL interpreter and a Deaf validator, evaluated a validation subset, producing a Cohen's Kappa of 0.4949, interpreted as moderate agreement. The best-performing 80/10/10 split achieved test accuracies of 97.22% for dynamic words, 99.89% for alphabets, and 99.96% for numbers. The system was evaluated by 15 respondents composed of 12 students and 3 teachers/instructors. The Software Usability Measurement Inventory-based assessment produced an overall weighted mean of 4.51, interpreted as acceptable and usable. Economic, technical, and operational assessments further supported the feasibility of the developed application.