| ISSN: 2583-4371 | ISSN: 2583-4371JOURNAL ARTICLE
Abstract: Lecturers in large classes cannot watch every student, so declining attention is usually noticed only after the lesson. This paper presents SEMS (Student Engagement Monitoring System), a computer vision system that classifies each student as Engaged, Partially Engaged or Disengaged from five visual cues (eye gaze, eye openness, yawning, head pose and body posture), alerts the lecturer in real time and suggests a teaching action that suits the lesson. The cues are measured with MediaPipe face and pose landmarks, converted into indicator levels and classified by explicit rules and by trained models. A dataset of 6,780 student images was built from the public SCB classroom datasets and split by classroom photographs, so that test classrooms were never seen in training. A rule baseline, a Decision Tree, a multilayer perception, an EfficientNetV2-B0 image model and a weighted fusion of cue and image predictions were compared. The fusion reached 79.8% accuracy and a macro F1 of 0.796 on 1,328 test students, doubling the 39.95% of the first model but remaining below the 90% target; sleeping students, whose faces are hidden, were the main source of error. In a simulated class of 24 students the live system flagged disengaged students 2.4 s after drowsiness began, raised no alerts while the class was attentive, named the affected rows and delivered alerts to a phone dashboard in about 3 ms.
Keywords: student engagement, computer vision, facial landmarks, learning analytics, real-time alerts, teacher dashboard.
Article Info: Received: 10 Oct 2026, Received in revised form: 05 Nov 2026, Accepted: 09 Nov 2026, Available online: 13 Nov 2026
[1] J. A. Fredricks, P. C. Blumenfeld, and A. H. Paris, “School engagement: Potential of the concept, state of the evidence,” Rev. Educ. Res., vol. 74, no. 1, pp. 59–109, 2004, doi: 10.3102/00346543074001059.
[2] M. Mat Halif et al., “Moderating effects of student motivation on the relationship between learning styles and student engagement,” Asian J. Univ. Educ., vol. 16, no. 2, pp. 94–103, 2020, doi: 10.24191/ajue.v16i2.10301.
[3] K. S. Gurman, Y. F. Chan, M. W. Khor, A. Samsul Kamil, A. A. Norazah, and P. Krishnan, “Learner engagement in 21st century technology integrated classrooms: A Malaysian perspective,” Asian J. Univ. Educ., vol. 21, no. 3, 2025, doi: 10.24191/ajue.v21i3.76.
[4] J. Whitehill, Z. Serpell, Y.-C. Lin, A. Foster, and J. R. Movellan, “The faces of engagement: Automatic recognition of student engagement from facial expressions,” IEEE Trans. Affect. Comput., vol. 5, no. 1, pp. 86–98, 2014, doi: 10.1109/TAFFC.2014.2316163.
[5] A. Gupta, A. D’Cunha, K. Awasthi, and V. Balasubramanian, “DAiSEE: Towards user engagement recognition in the wild,” arXiv:1609.01885, 2016.
[6] M. Stidi and M. Nasser, “Multi-modal student engagement detection using eye-gaze and posture analysis under obstructed conditions,” Int. J. Teach. Learn. Educ., vol. 4, no. 6, pp. 1–7, 2025.
[7] Z. Wang, J. Yao, C. Zeng, W. Wu, H. Xu, and Y. Yang, “YOLOv5 enhanced learning behavior recognition and analysis in smart classroom with multiple students,” in Proc. Int. Conf. Intell. Educ. Intell. Res. (IEIR), 2022, pp. 23–29, doi: 10.1109/IEIR56323.2022.10069015.
[8] B. Jiang, W. Xu, C. Guo, W. Liu, and W. Cheng, “A classroom concentration model based on computer vision,” in Proc. ACM Turing Celebration Conf.–China, 2019, pp. 1–6, doi: 10.1145/3321408.3322856.
[9] B. N. Anh et al., “A computer-vision based application for student behavior monitoring in classroom,” Appl. Sci., vol. 9, no. 22, Art. no. 4729, 2019, doi: 10.3390/app9224729.
[10] A. Sabuncuoğlu and T. M. Sezgin, “Developing a multimodal classroom engagement analysis dashboard for higher-education students,” Proc. ACM Hum.-Comput. Interact., vol. 7, no. EICS, Art. no. 188, 2023, doi: 10.1145/3593240.
[11] Z. Dan, Z. Yali, and Z. Tong, “A multimodal AI framework for real-time student engagement detection and adaptive feedback in higher education,” J. King Saud Univ.–Comput. Inf. Sci., vol. 38, Art. no. 243, 2026, doi: 10.1016/j.jksuci.2026.243.
[12] M. Firat, “Predicting student disengagement: Harnessing visual cues for intelligent tutoring systems,” London J. Soc. Sci., vol. 6, p. 107, 2023, doi: 10.31039/ljss.2023.6.107.
[13] A. S. Pillai, “Student engagement detection in classrooms through computer vision and deep learning: A novel approach using YOLOv4,” SSRN preprint, 2021, doi: 10.2139/ssrn.5187374.
[14] Q. Liu, X. Jiang, and R. Jiang, “Classroom behavior recognition using computer vision: A systematic review,” Sensors, vol. 25, no. 2, Art. no. 373, 2025, doi: 10.3390/s25020373.
[15] T. Lowe and Y. El Hakim, Eds., A Handbook for Student Engagement in Higher Education: Theory into Practice. London, U.K.: Routledge, 2020, doi: 10.4324/9780429023033.
[16] P. Chapman et al., CRISP-DM 1.0: Step-by-Step Data Mining Guide. SPSS Inc., 2000.
[17] F. Yang, “SCB-dataset: A dataset for detecting student classroom behavior,” arXiv:2304.02488, 2023.
[18] C. Lugaresi et al., “MediaPipe: A framework for building perception pipelines,” arXiv:1906.08172, 2019.
[19] Y. Kartynnik, A. Ablavatski, I. Grishchenko, and M. Grundmann, “Real-time facial surface geometry from monocular video on mobile GPUs,” arXiv:1907.06724, 2019.
[20] V. Bazarevsky, I. Grishchenko, K. Raveendran, T. Zhu, F. Zhang, and M. Grundmann, “BlazePose: On-device real-time body pose tracking,” arXiv:2006.10204, 2020.
[21] T. Soukupová and J. Čech, “Real-time eye blink detection using facial landmarks,” in Proc. 21st Comput. Vis. Winter Workshop, 2016.
[22] M. Tan and Q. V. Le, “EfficientNetV2: Smaller models and faster training,” in Proc. 38th Int. Conf. Mach. Learn. (ICML), PMLR vol. 139, 2021, pp. 10096–10106.
[23] Danyaro, K. U., Nasser, M., Zakari, A., Abdullahi, S., Khanzada, A., Yakubu, M. M., & Shoaib, S. (2025). LLM-based code generation: a systematic literature review with technical and demographic insights. IEEE Access, 13, 194915-194939.
[24] Nasser, M. (2024). Personalized learning through AI: enhancing student engagement and teacher effectiveness. International Journal of Teaching, Learning and Education, 3(6), 23-26.
[25] Nasser, M., Yusof, U. K., & Salim, N. (2023). Deep learning based methods for molecular similarity searching: a systematic review. Processes, 11(5), 1340.
