Multi-Modal Student Engagement Detection Using Eye-Gaze and Posture Analysis under Obstructed Conditions

JOURNAL ARTICLE

Author: Michael Stidi, Maged Nasser

Keywords: Student Engagement, Multimodal Learning analytics, YOLOv11, MoveNet, MediaPipe, FaceMesh, Machine Learning, Obstructions robustness

Abstract: Engagement is a key indicator of effective learning yet remains challenging to measure objectively. Most existing classroom analytics focus solely on facial expressions, making them susceptible to occlusions such as face masks, books or extreme head poses. This paper proposes a lightweight, multimodal pipeline that combines body posture and gaze direction to robustly infer student engagement. The system uses Ultralytics YOLOv11 detector for locating students within each frame. Cropped regions are fed into MoveNet, a fast pose estimator that outputs skeletal keypoints, and into MediaPipe FaceMesh, a 3D face landmark model that estimates eye gaze. These complementary features are concatenated into fixed-length vectors and classified by conventional machine learning algorithms (Multilayer Perceptron, Support Vector Machine and Random Forest). Experiments on the SCB-Dataset a recent benchmark of student and teacher classroom behaviour which demonstrates that the proposed late-fusion approach achieves high accuracy while remaining computationally efficient. Comparisons with posture only and gaze only baselines indicate that fusing posture and gaze cues improves F1 score and AUC, and the fusion is resilient to obstructions. The pipeline is suitable for real time analytics on edge devices and offers a scalable tool for enhancing learning analytics in classrooms.

Article Info: Received: 27 Sep 2025, Received in revised form: 27 Oct 2025, Accepted: 01 Nov 2025, Available online: 06 Nov 2025

DOI: 10.22161/ijtle.4.6.3

Total View: 280

Downloads: 6

Page No: 20-26