Deep Learning for Real-time Affective Hand Gesture Recognition in EMASPEL

Article Sidebar

PDF
Published: Mar 30, 2024
DOI: https://doi.org/10.35940/ijrte.F8012.12060324
Keywords:
Real-time Affective Hand Gesture Recognition, Deep Learning, EMASPEL

Main Article Content

Dr. Mohamed Ben Ammar
Department of Information Systems, Faculty of Computing and Information Technology, Northern Border University, Rafha, Saudi Arabia.
https://orcid.org/0000-0001-8990-3924
Jihane Ben Slimane
Department of Computer Sciences Faculty of Computing and Information Technology, Northern Border University, Rafha, Saudi Arabia, National Engineering School of Tunis, LR11ES20 Analysis Design and Control of Systems Laboratory, University of Tunis El Manar, Tunis, Tunisia.
Taoufik Saidani
Department of Computer Sciences Faculty of Computing and Information Technology, Northern Border University, Rafha, Saudi Arabia.
Refka Ghodhbani
Department of Computer Sciences Faculty of Computing and Information Technology, Northern Border University, Rafha, Saudi Arabia.

Abstract

This research marks a transformative leap in personalized learning through real-time affective hand gesture recognition in EMASPEL (Emotional Multi-Agents System for Peer-to-peer E-Learning), an educational platform. Our deep learning model, a meticulously crafted ensemble of convolutional and recurrent neural networks, deciphers the unspoken language of emotions embedded within student gestures, accurately capturing both spatial and temporal patterns. This detailed emotional map empowers EMASPEL to tailor its interactions with exquisite precision, addressing frustration, nurturing curiosity, and maximizing student engagement. The impact is profound: students flourish in personalized learning environments, experiencing enhanced outcomes and a newfound connection to their educational journey. Teachers, equipped with real-time emotional insights, provide targeted support and cultivate a more inclusive, responsive classroom. Beyond gestures, we envision a future enriched by multimodal data integration, encompassing facial expressions, voice analysis, and potentially physiological sensors, to paint even richer portraits of student emotions and cognitive states. Continuous refinement through rigorous longitudinal studies will pave the way for deeper understanding and ensure responsible implementation. Ultimately, this research reimagines education as a dynamic ensemble of personalized learning, where technology serves as a bridge between teacher and student, unlocking not just academic success but a lifelong love of knowledge.

Downloads

Download data is not yet available.

Article Details

How to Cite
[1]
Dr. Mohamed Ben Ammar, Jihane Ben Slimane, Taoufik Saidani, and Refka Ghodhbani , Trans., “Deep Learning for Real-time Affective Hand Gesture Recognition in EMASPEL”, IJRTE, vol. 12, no. 6, pp. 25–34, Mar. 2024, doi: 10.35940/ijrte.F8012.12060324.
Issue
Vol. 12 No. 6 (2024): Volume-12 Issue-6, March 2024
Section
Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

CC-BY-NC-ND 4.0

How to Cite

[1]
Dr. Mohamed Ben Ammar, Jihane Ben Slimane, Taoufik Saidani, and Refka Ghodhbani , Trans., “Deep Learning for Real-time Affective Hand Gesture Recognition in EMASPEL”, IJRTE, vol. 12, no. 6, pp. 25–34, Mar. 2024, doi: 10.35940/ijrte.F8012.12060324.
Share |
Crossref
Scopus
Google Scholar
Europe PMC

References

Alpaydin, E. (2022). Introduction to machine learning (4th ed.). MIT Press.

Ben Ammar, M., Neji, M., ALIMI, M.A and Gouardères, G. (2010) ‘‘The Affective Tutoring System’’, Expert Systems with Applications 37 3013-3023. Elsevier. https://doi.org/10.1016/j.eswa.2009.09.031

Cao, C., Xu, D., Yang, Z., & Wang, R. (2020). Real-time hand gesture recognition for human-computer interaction with a multi-scale convolutional neural network. Sensors, 20(13), 3705. doi:10.3390/s20133705 https://doi.org/10.3390/s20133705

Ekman, P. (2009). Unmasking the face: A guide to recognizing emotions from facial expressions (Rev. and updated ed.). Malor Books.

Fernández-Rodríguez, A., Martinez-Gonzalez, A., & Gonzalez-Mancera, A. (2023). A review of affective computing applications in education. Sensors, 23(10), 5957. doi:10.3390/s23105957

Galati, S., Martinez, A. M., & Nobile, A. (2020). Real-time affective hand gesture recognition using a hybrid architecture. Pattern Recognition Letters, 139, 182-192. doi:10.1016/j.patrec.2020.07.022 https://doi.org/10.1016/j.patrec.2020.07.022

Hegde, R. S., & Kavita, T. (2022). Affective gesture recognition for educational systems: A review. International Journal of Computer Applications, 15, 1-6. doi:10.24093/ijca.1186

Kshirsagar, S., & Kulkarni, U. G. (2023). Real-time hand gesture recognition using computer vision for human-computer interaction: A survey. International Journal of Advanced Research in Computer Science and Software Engineering, 13(3), 876-884.

Li, H., Liu, Y., Wu, X., & Li, R. (2022). Real-time hand gesture recognition using transfer learning with convolutional neural networks. Electronics, 11(12), 2005. doi:10.3390/electronics11122005

Lopez-Martinez, A., Galati, S., Martinez, A. M., & Nobile, A. (2022). Affective human-computer interaction through real-time hand gesture recognition with dynamic neural networks. Neural Processing Letters, 48(2), 703-723. doi:10.1007/s11974-021-02805-0

Lu, M., Zhang, W., & Liu, Z. (2023). A survey of recent advances in hand gesture recognition. Pattern Recognition Letters, 173, 192-209. doi:10.1016/j.patrec.2023.08.004 https://doi.org/10.1016/j.patrec.2023.08.004

Meyer, K., Lüke, B., & Kunzendorf, E. (2022). Cultural differences in nonverbal communication and gestures: Implications for the design of gesture-based human-computer interaction systems. Human-Computer Interaction, 37(4), 1098-1142. doi:10.1080/07357206.2021.1930574

Pantic, M., & Rothkrantz, L. J. M. (2007). Affective computing for measuring user experience and emotions. User Modeling and User-Adapted Interaction, 16(3-4), 345-391. doi:10.1007/s11257-007-9074-4

Pfau-Gray, C., Muntele, I., & Brössert, C. (2023). Affective gesture recognition in educational software: Exploring the potential of embodied learning with children. Frontiers in Computer Science, 7, 815791. doi:10.3389/fcomp.2023.815791

Queirolo, D., De Salvo, G., Greco, A., & Pisani, C. (2023). Deep learning for robust hand gesture recognition: A survey. Robotics and Autonomous Systems, 170, 103600. doi:10.1016/j.robot.2023.103600

Ruiz-Garcia, P., Sanchez-Casado, P., Fernandez-Moral, J. L., & Madrid Morales, V. (2023). Affective learning with intelligent tutoring systems: A review of the state of the art. Applied Sciences, 13(23), 12202. doi:10.3390/app132312202

Jaafar, J., Yusof, H. M., Hassan, S., Adtrudin, K. F., & Ahmad, R. (2019). Nexus between Emotional Intelligence (EQ-I) and Entrepreneurial Culture. In International Journal of Engineering and Advanced Technology (Vol. 8, Issue 6s3, pp. 986–992). https://doi.org/10.35940/ijeat.f1093.0986s319

Srinivasa, M. S., & Vijayashree, D. L. (2021). A Study on Impact of Sensitive Intelligence and Perceived Stress. In International Journal of Management and Humanities (Vol. 5, Issue 7, pp. 27–29). https://doi.org/10.35940/ijmh.g1257.035721

Srividya, M. S., & R, Dr. A. M. (2020). Research trends in Hand Gesture Recognition techniques. In International Journal of Recent Technology and Engineering (IJRTE) (Vol. 8, Issue 6, pp. 1059–1064). https://doi.org/10.35940/ijrte.f7519.038620

Kaur, S., & Bhatla, Er. N. (2019). An Efficient Gesture Recognition with ABC-ANN Classification and Key-Point Features Extraction for Hand Images. In International Journal of Innovative Technology and Exploring Engineering (Vol. 8, Issue 10, pp. 3193–3199). https://doi.org/10.35940/ijitee.j1153.0881019

M R, Dr. P. (2022). Sign Language Recognition System. In Indian Journal of Software Engineering and Project Management (Vol. 2, Issue 1, pp. 1–3). https://doi.org/10.54105/ijsepm.c9011.011322