Deep Learning for Real-time Affective Hand Gesture Recognition in EMASPEL
Article Sidebar
Main Article Content
Abstract
This research marks a transformative leap in personalized learning through real-time affective hand gesture recognition in EMASPEL (Emotional Multi-Agents System for Peer-to-peer E-Learning), an educational platform. Our deep learning model, a meticulously crafted ensemble of convolutional and recurrent neural networks, deciphers the unspoken language of emotions embedded within student gestures, accurately capturing both spatial and temporal patterns. This detailed emotional map empowers EMASPEL to tailor its interactions with exquisite precision, addressing frustration, nurturing curiosity, and maximizing student engagement. The impact is profound: students flourish in personalized learning environments, experiencing enhanced outcomes and a newfound connection to their educational journey. Teachers, equipped with real-time emotional insights, provide targeted support and cultivate a more inclusive, responsive classroom. Beyond gestures, we envision a future enriched by multimodal data integration, encompassing facial expressions, voice analysis, and potentially physiological sensors, to paint even richer portraits of student emotions and cognitive states. Continuous refinement through rigorous longitudinal studies will pave the way for deeper understanding and ensure responsible implementation. Ultimately, this research reimagines education as a dynamic ensemble of personalized learning, where technology serves as a bridge between teacher and student, unlocking not just academic success but a lifelong love of knowledge.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
How to Cite
References
Alpaydin, E. (2022). Introduction to machine learning (4th ed.). MIT Press.
Ben Ammar, M., Neji, M., ALIMI, M.A and Gouardères, G. (2010) ‘‘The Affective Tutoring System’’, Expert Systems with Applications 37 3013-3023. Elsevier. https://doi.org/10.1016/j.eswa.2009.09.031
Cao, C., Xu, D., Yang, Z., & Wang, R. (2020). Real-time hand gesture recognition for human-computer interaction with a multi-scale convolutional neural network. Sensors, 20(13), 3705. doi:10.3390/s20133705 https://doi.org/10.3390/s20133705
Ekman, P. (2009). Unmasking the face: A guide to recognizing emotions from facial expressions (Rev. and updated ed.). Malor Books.
Fernández-Rodríguez, A., Martinez-Gonzalez, A., & Gonzalez-Mancera, A. (2023). A review of affective computing applications in education. Sensors, 23(10), 5957. doi:10.3390/s23105957
Galati, S., Martinez, A. M., & Nobile, A. (2020). Real-time affective hand gesture recognition using a hybrid architecture. Pattern Recognition Letters, 139, 182-192. doi:10.1016/j.patrec.2020.07.022 https://doi.org/10.1016/j.patrec.2020.07.022
Hegde, R. S., & Kavita, T. (2022). Affective gesture recognition for educational systems: A review. International Journal of Computer Applications, 15, 1-6. doi:10.24093/ijca.1186
Kshirsagar, S., & Kulkarni, U. G. (2023). Real-time hand gesture recognition using computer vision for human-computer interaction: A survey. International Journal of Advanced Research in Computer Science and Software Engineering, 13(3), 876-884.
Li, H., Liu, Y., Wu, X., & Li, R. (2022). Real-time hand gesture recognition using transfer learning with convolutional neural networks. Electronics, 11(12), 2005. doi:10.3390/electronics11122005
Lopez-Martinez, A., Galati, S., Martinez, A. M., & Nobile, A. (2022). Affective human-computer interaction through real-time hand gesture recognition with dynamic neural networks. Neural Processing Letters, 48(2), 703-723. doi:10.1007/s11974-021-02805-0
Lu, M., Zhang, W., & Liu, Z. (2023). A survey of recent advances in hand gesture recognition. Pattern Recognition Letters, 173, 192-209. doi:10.1016/j.patrec.2023.08.004 https://doi.org/10.1016/j.patrec.2023.08.004
Meyer, K., Lüke, B., & Kunzendorf, E. (2022). Cultural differences in nonverbal communication and gestures: Implications for the design of gesture-based human-computer interaction systems. Human-Computer Interaction, 37(4), 1098-1142. doi:10.1080/07357206.2021.1930574
Pantic, M., & Rothkrantz, L. J. M. (2007). Affective computing for measuring user experience and emotions. User Modeling and User-Adapted Interaction, 16(3-4), 345-391. doi:10.1007/s11257-007-9074-4
Pfau-Gray, C., Muntele, I., & Brössert, C. (2023). Affective gesture recognition in educational software: Exploring the potential of embodied learning with children. Frontiers in Computer Science, 7, 815791. doi:10.3389/fcomp.2023.815791
Queirolo, D., De Salvo, G., Greco, A., & Pisani, C. (2023). Deep learning for robust hand gesture recognition: A survey. Robotics and Autonomous Systems, 170, 103600. doi:10.1016/j.robot.2023.103600
Ruiz-Garcia, P., Sanchez-Casado, P., Fernandez-Moral, J. L., & Madrid Morales, V. (2023). Affective learning with intelligent tutoring systems: A review of the state of the art. Applied Sciences, 13(23), 12202. doi:10.3390/app132312202
Jaafar, J., Yusof, H. M., Hassan, S., Adtrudin, K. F., & Ahmad, R. (2019). Nexus between Emotional Intelligence (EQ-I) and Entrepreneurial Culture. In International Journal of Engineering and Advanced Technology (Vol. 8, Issue 6s3, pp. 986–992). https://doi.org/10.35940/ijeat.f1093.0986s319
Srinivasa, M. S., & Vijayashree, D. L. (2021). A Study on Impact of Sensitive Intelligence and Perceived Stress. In International Journal of Management and Humanities (Vol. 5, Issue 7, pp. 27–29). https://doi.org/10.35940/ijmh.g1257.035721
Srividya, M. S., & R, Dr. A. M. (2020). Research trends in Hand Gesture Recognition techniques. In International Journal of Recent Technology and Engineering (IJRTE) (Vol. 8, Issue 6, pp. 1059–1064). https://doi.org/10.35940/ijrte.f7519.038620
Kaur, S., & Bhatla, Er. N. (2019). An Efficient Gesture Recognition with ABC-ANN Classification and Key-Point Features Extraction for Hand Images. In International Journal of Innovative Technology and Exploring Engineering (Vol. 8, Issue 10, pp. 3193–3199). https://doi.org/10.35940/ijitee.j1153.0881019
M R, Dr. P. (2022). Sign Language Recognition System. In Indian Journal of Software Engineering and Project Management (Vol. 2, Issue 1, pp. 1–3). https://doi.org/10.54105/ijsepm.c9011.011322