DEEP LEARNING-BASED SIGN LANGUAGE RECOGNITION FOR INCLUSIVE COMMUNICATION ACCESS

Abstract

The three leading deep learning models used were Vision Transformer, EfficientNet-B0, and ResNet-50. To find out each model's performance in reliably detecting sign gestures, its accuracy, precision, recall, and F1 score were determined. The ResNet-50 expressed good feature recognitions with continuous learning between test and training data sets, and it had gained its top accuracy level of 98.9%. With 97.5% F1 score, its 97% accuracy and 98% recall expressed its reliable prediction accuracy of sign movements. The EfficientNet-B0 followed with 97.4% accuracy level as well. As expressed by the rising validation loss graph, it expressed overfiting signs despite its training speed being very fast. With 95% accuracy and 96% recall, it had 95.5% F1 score. With 92.5% F1 score and appropriate precision and recall of 92% and 93%, respectively, the prototype of Vision Transformer developed a 93% accuracy level. In comparison with EfficientNet-B0, the prototype of Vision Transformer expressed more stability throughout training with minimum overloading, albeit with somewhat lower accuracy. These results are valuable in illustrating model usability in real-world applications and international sign recognitions, wherein reliable performances are mandatory. This study shows that deep learning might provide reliable, scalable alternatives to real-time sign gesture processes when optimisation steps and infrastructures are carefully chosen.