CNN Algorithm with SIFT to Enhance the Arabic Sign Language Recognition
Article Sidebar
Main Article Content
Abstract
Sign language is used as a primary means of communication by millions of people who suffer from hearing problems. The unhearing people used visual language to interact with each other, Represented in sign language. There are features that the hearing impaired use to understand each other, which are difficult for normal people to understand. Therefore, deaf people will struggle to interact with society. This research aims to introduce a system for recognizing hand gestures in Arabic Sign Language (ArSL) through training the Convolutional Neural Network (CNN) on the images of ArSL gestures launched by the University of Prince Mohammad Bin Fahd, Saudi Arabia. A Scale Invariant Feature Transform (SIFT) algorithm is used for creating the feature vectors that contain shape, finger position, size, center points of palm, and hand margin by extracting the Important features for images of ArSL and transforming them to points of the vector. The accuracy of the proposed system is 97% using the SIFT with CNN, and equal to 94.8% nearly without SIFT. Finally, the proposed system was tried and tested on a group of persons and its effectiveness was proven after considering their observations.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
How to Cite
References
S. Arooj, S. Altaf, S. Ahmad, H. Mahmoud, and A. S. N. Mohamed, "Enhancing sign language recognition using CNN and SIFT: A case study on Pakistan sign language," Journal of King Saud University-Computer and Information Sciences, vol. 36, no. 2, p. 101934, 2024. https://doi.org/10.1016/j.jksuci.2024.101934
R. A. Kadhim and M. Khamees, "A Real-Time American Sign Language Recognition System using Convolutional Neural Network for Real Datasets," Tem Journal, vol. 9, no. 3, 2020.
https://doi.org/10.18421/TEM93-14
N. S. Sagheer and S. A. Yousif, "Canopy with k-means clustering algorithm for big data analytics," in AIP Conference Proceedings, 2021, vol. 2334, no. 1: AIP Publishing. https://doi.org/10.1063/5.0042398
N. S. Sagheer and S. A. Yousif, "A parallel clustering analysis based on Hadoop multi-node and Apache mahout," Iraqi Journal of Science, pp. 2431-2444, 2021. https://doi.org/10.24996/ijs.2021.62.7.32
C. Janiesch, P. Zschech, and K. Heinrich, "Machine learning and deep learning," Electronic Markets, vol. 31, no. 3, pp. 685-695, 2021. https://doi.org/10.1007/s12525-021-00475-2
Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, "A survey of convolutional neural networks: analysis, applications, and prospects," IEEE Transactions on neural networks and learning systems, vol. 33, no. 12, pp. 6999-7019, 2021. https://doi.org /10.1109/TNNLS.2021.3084827
K.-Z. Liu, P.-J. Lee, G.-C. Xu, and B.-H. Chang, "SIFT-enhanced CNN based objects recognition for satellite image," in 2020 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-Taiwan), 2020, pp. 1-2: IEEE. DOI: 10.1109/ICCE-Taiwan49838.2020.9258037
A. Alnahhas, B. Alkhatib, N. Al-Boukaee, N. Alhakim, O. Alzabibi, and N. Ajalyakeen, "Enhancing the recognition of Arabic sign language by using deep learning and leap motion controller," Int. J. Sci. Technol. Res, vol. 9, no. 4, pp. 1865-1870, 2020.
T. H. Noor et al., "Real-Time Arabic Sign Language Recognition Using a Hybrid Deep Learning Model," Sensors, vol. 24, no. 11, p. 3683, 2024. https://drepo.sdl.edu.sa/handle/20.500.14154/2832
R. Marzouk, F. Alrowais, F. N. Al-Wesabi, and A. M. Hilal, "Atom search optimization with deep learning enabled Arabic sign language recognition for speaking and hearing disability persons," in Healthcare, 2022, vol. 10, no. 9, p. 1606: MDPI. https://doi.org/10.3390/healthcare10091606
F. H. A. Noor S. Sagheer, Manar Hamza Bashaa, "Enhancing Arabic Sign Language Recognition using Deep Learning," International Journal of Innovative Technology and Exploring Engineering (IJITEE), vol. 13, no. 5, p. 6, 30l4l2024 2024. https://doi.org/10.35940/ijitee.E9844.13050424
R. Abdul Ameer, M. Ahmed, Z. Al-Qaysi, M. Salih, and M. L. Shuwandy, "Empowering Communication: A Deep Learning Framework for Arabic Sign Language Recognition with an Attention Mechanism," Computers, vol. 13, no. 6, p. 153, 2024. https://doi.org/10.3390/computers13060153
G. Latif, N. Mohammad, J. Alghazo, R. AlKhalaf, and R. AlKhalaf, "ArASL: Arabic alphabets sign language dataset," Data in brief, vol. 23, p. 103777, 2019. https://doi.org/10.1016/j.dib.2019.103777
M. Haroon, S. Altaf, S. Ahmad, M. Zaindin, S. Huda, and S. Iqbal, "Hand gesture recognition with a symmetric pattern under diverse illuminated conditions using artificial neural network," Symmetry, vol. 14, no. 10, p. 2045, 2022. https://doi.org/10.3390/sym14102045
M. S. Amin, S. T. H. Rizvi, and M. M. Hossain, "A comparative review on applications of different sensors for sign language recognition," Journal of Imaging, vol. 8, no. 4, p. 98, 2022. https://doi.org/10.3390/jimaging8040098
M. J. Cheok, Z. Omar, and M. H. Jaward, "A review of hand gesture and sign language recognition techniques," International Journal of Machine Learning and Cybernetics, vol. 10, pp. 131-153, 2019. https://doi.org/10.1007/s13042-017-0705-5
R. Chauhan, K. K. Ghanshala, and R. Joshi, "Convolutional neural network (CNN) for image detection and recognition," in 2018 First International Conference on secure cyber computing and communication (ICSCCC), 2018, pp. 278-282: IEEE. DOI: 10.1109/ICSCCC.2018.8703316
K. Shridhar, F. Laumann, and M. Liwicki, "A comprehensive guide to Bayesian convolutional neural network with variational inference," arXiv preprint arXiv:1901.02731, 2019. https://doi.org/10.48550/arXiv.1901.02731
M. K. Singh, V. Kekatos, and G. B. Giannakis, "Learning to solve the AC-OPF using sensitivity-informed deep neural networks," IEEE Transactions on Power Systems, vol. 37, no. 4, pp. 2833-2846, 2021. DOI: 10.1109/TPWRS.2021.3127189
R. Schmitz et al., "Multi-scale fully convolutional neural networks for histopathology image segmentation: from nuclear aberrations to the global tissue architecture," Medical image analysis, vol. 70, p. 101996, 2021. https://doi.org/10.1016/j.media.2021.101996
J. Fontenrose, Python: A Study of Delphic Myth and Its Origins. Univ of California Press, 2023.
W. Vallejo, C. Díaz-Uribe, and C. Fajardo, "Google collab and virtual simulations: practical e-learning tools to support the teaching of thermodynamics and to introduce coding to students," ACS omega, vol. 7, no. 8, pp. 7421-7429, 2022. https://doi.org/10.1021/acsomega.2c00362
Suharjito, Gunawan, H., & Thiracitta, N. (2020). Optimization of Modified Hidden Markov Model for Vision-Based Indonesian Sign Languages Recognition. In International Journal of Recent Technology and Engineering (IJRTE) (Vol. 8, Issue 5, pp. 615–621). https://doi.org/10.35940/ijrte.e5666.018520
Das, S., S, S., M, A., & Jayaram, S. (2021). Deep Learning Convolutional Neural Network for Defect Identification and Classification in Woven Fabric. In Indian Journal of Artificial Intelligence and Neural Networking (Vol. 1, Issue 2, pp. 9–13). https://doi.org/10.54105/ijainn.b1011.041221
Kumar, P., & Rawat, S. (2019). Implementing Convolutional Neural Networks for Simple Image Classification. In International Journal of Engineering and Advanced Technology (Vol. 9, Issue 2, pp. 3616–3619). https://doi.org/10.35940/ijeat.b3279.129219
Raj, H., Duggal., A., M., A. K. S., Uppara, S., & S., S. M. (2020). Hand Motion Analysis using CNN. In International Journal of Soft Computing and Engineering (Vol. 9, Issue 6, pp. 26–30). https://doi.org/10.35940/ijsce.f3409.059620
Magapu, H., Krishna Sai, M. R., & Goteti, B. (2024). Human Deep Neural Networks with Artificial Intelligence and Mathematical Formulas. In International Journal of Emerging Science and Engineering (Vol. 12, Issue 4, pp. 1–2). https://doi.org/10.35940/ijese.c9803.12040324