Leveraging CNN to analyze facial expressions for academic engagement monitoring with insights from the multi-source academic affective engagement dataset

Abstract

The dynamics of student engagement and emotional states significantly influence learning outcomes. Positive emotions, stemming from successful task completion, contrast with negative emotions arising from learning struggles or failures. Effective transitions to engagement occur upon problem resolution, while unresolved issues lead to frustration and subsequent boredom. Facial engagement monitoring is crucial for assessing students’ attention, interest, and emotional responses during learning. Recent advancements in convolutional neural networks (CNNs) show promise in automatically analyzing facial expressions to infer engagement levels. This study proposes a CNN-based approach utilizing the multi-source academic affective engagement dataset (MAAED) to categorize facial expressions into boredom, confusion, frustration, and yawning. By extracting features from facial images, this method offers an efficient and objective means to gauge student engagement. Recognizing and addressing negative affective states, such as confusion and boredom, is fundamental in creating supportive learning environments. Through automated frame extraction and model comparison, this study demonstrates reduced loss values with improving accuracy, showcasing the effectiveness of this method in objectively evaluating student engagement. Facial engagement monitoring with CNNs, using the MAAED dataset, is pivotal in understanding human behavior and enhancing educational experiences. The CNN model, trained on MAAED annotated facial expressions, accurately classifies engagement categories. Experimental results underscore the CNN-based approach’s efficacy in monitoring facial engagement, highlighting its potential to enrich educational environments and personalized learning experiences.