An improved student’s facial emotions recognition method using transfer learning

Abstract

Instructors endeavour to encourage active participation and interaction among learners. However, in settings with a large number of students, such as universities or online platforms, obtaining real-time feedback and evaluating teaching methodology presents a significant challenge. In this paper, we introduce a student engagement recognition system based on a hybrid method using handcrafted features and transfer learning. The research is conducted on two databases for emotion detection based on facial cues (FER13) benchmarked dataset and our database. We use the local binary patterns (LBP) method combined with pre-trained MobileNet model for feature extraction and classification. The proposed system adeptly discerns students’ facial expressions and categorizes their engagement states as either ‘engaged’ or ‘disengaged’. We determine the most effective model by evaluating and comparing several deep learning models, including Inception-V3, VGG16, EfficientNet, ResNet, and DenseNet. Experimental results underscore the efficacy of our approach, revealing a remarkable accuracy, surpassing benchmarks set by state-of-the-art models.