Efficient object detection for augmented reality based english learning with YOLOv8 optimization

Abstract

This study develops a mobile-based augmented reality (AR) application with machine learning for elementary school students to enhance basic English vocabulary learning. The application integrates an optimized YOLOv8 object detection model, designed to recognize 20 common classroom objects in real-time. The model optimization involves replacing standard Conv layers with GhostConv and the C2f block with the C2fCIB block that has significantly improved computational efficiency. Evaluation results show the optimized model reduces the parameters by 22.003% and decreases the file size from 6.2 MB to 4.9 MB. The model performance improved by achieving precision of 83.7%, recall of 73.5% and a mean Average Precision (mAP) of 81.4%. The model was integrated into the Unity platform via the Barracuda library, enabling real-time detection and interactive display of 3D objects. This aplication also complete with English text, translations, example sentences also audio pronunciation. 3D objects representing classroom vocabulary were specifically created to support AR-based learning. Performance testing on a Samsung A14 showed an improved frame rate of 6–12 FPS compared to the original model’s 5–10 FPS. These results demonstrate that the optimized YOLO model effectively integrates with AR technology, creating a more interactive and enjoyable vocabulary learning experience.