Analysis of Student Engagement in Online Classroom using Convolutional Neural Networks (CNN)
Article Sidebar
Main Article Content
Abstract
The pandemic of the coronavirus (COVID-19) has affected the education system, disrupting all from traditional classrooms to online classrooms. This complicates tracking involvement in online classes even further. A student's separation from education is the most serious scenario that can occur, in addition to damaging the efficiency of the learners. Teachers should indeed be regularly informed about student participation so that they may customize their instruction to the online learning environment. This paper presents a model for analyzing and tracking student participation in online classrooms using neural networks. The approach utilizes Convolutional Neural Networks (CNN) to analyze learners' faces, employing a pre-trained base model obtained from the Keras website. The model categorizes student engagement into three levels disengagement, normal engagement, and high engagement. The experiments were divided into three groups: adjusting the image feature extraction layer, analyzing individual parameters (Learning Rate, Batch Size, Optimizer, Fully connected), and conducting a sequential two-parameter matching test. The best-performing models from each test were applied to subsequent trials, leading to progressive improvements in the model's performance for monitoring online class participation. The evaluation of the model's performance yielded promising results. By incorporating the Optimizer Ranger and employing a Fully Connected (FC) layer with a configuration of 50-100 units, the accuracy of the model experienced a significant boost, reaching an impressive 82.30%. Simultaneously, the loss was notably reduced to 0.46. These improvements were substantial compared to the baseline model, with accuracy seeing a remarkable enhancement of 16.51% and a reduction in loss by 0.31. These findings showcased the effectiveness of the proposed approach in accurately monitoring and categorizing learner engagement levels based on facial analysis.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
World Health Organization: WHO. (12 October 2020). Q&A on coronaviruses (COVID-19). [Online] Available: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/
COVID-19 CORONAVIRUS PANDEMIC (2020). (29 September 2020). [Online] Available: https://www.worldometers.info/coronavirus
Dewan. M. Lin. F, Wen. D. Murshed. M and Uddin. Z, “A Deep Learning Approach to Detecting Engagement of Online Learners,” pp. 1895 - 1902, 2018.
Murshed. M, Dewan. M, Lin. F, and Wen. D, “Engagement Detection in e-Learning Environments using Convolutional Neural Networks,” IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech)., pp. 80–86., 2019.
Keras. (28 September 2020). [Online] Available: https://keras.io/
Qiu. G, Yu. X, Sun. B, Wang. Y and Zhang. L, “Metastatic Cancer Image Classification Based on Deep Learning Method,”pp. 658-661, 2020.
Zhong. Z, Zheng. M, Mai. H, Zhao. J and Liu. X, “Cancer image classification based on DenseNet model,”, pp. 1-6, 2020.
Igiri. Chinwe, Uzoma. Anyama and Ita. Silas, “Effect of Learning Rate on Artificial Neural Network in Machine Learning,” International Journal of Engineering Research., Vol. 4, pp. 359-363, 2021.
E. M. Dogo, O. J. Afolabi, N. I. Nwulu, B. Twala and C. O. Aigbavboa, "A Comparative Analysis of Gradient Descent-Based Optimization Algorithms on Convolutional Neural Networks," International Conference on Computational Techniques, Electronics and Mechanical Systems (CTEMS)., pp. 92-99, 2018.
Z. Wan, Z. Yuxiang, X. Gong, Zhanghuali and B. Yu, "DenseNet model with RAdam optimization algorithm for cancer image classification," IEEE International Conference on Consumer Electronics and Computer Engineering (ICCECE)., pp. 771-775, 2021.
Melinte. Octavian and Vladareanu. Luige, “Facial Expressions Recognition for Human-Robot Interaction Using Deep Convolutional Neural Networks with Rectified Adam Optimizer,” Sensors., Vol. 20, pp. 1-21, 2020.
H. S. Suresha and S. S. Parthasarathy, "Alzheimer Disease Detection Based on Deep Neural Network with Rectified Adam Optimization Technique using MRI Analysis," Third International Conference on Advances in Electronics, Computers and Communications (ICAECC)., pp. 1-6, 2020.
Kandel. Ibrahem and Castelli. Mauro, “The effect of batch size on the generalizability of the convolutional neural networks on a histopathology dataset,” ICT Express., Vol. 6, pp. 312-315, 2020.
Altuwairqi, K., Jarraya, S. K., Allinjawi, A and Hammami, M., “A new emotion–based affective model to detect student’s engagement,”, Vol. 33, pp. 1-11, 2021.
Dewan. M, Lin. F, Wen. D, Murshed. M and Uddin. Z, “A Deep Learning Approach to Detecting Engagement of Online Learners,”, pp. 1895-1902, 2020.
Dash. S, Akber Dewan. M. A, Murshed. M, Lin. F, Abdullah-Al-Wadud. M and Das. A, “A Two-Stage Algorithm for Engagement Detection in Online Learning,” 2019 International Conference on Sustainable Technologies for Industry 4.0 (STI)., pp. 1–4, 2019.
Mane. S. S and Surve. A. R, “Engagement Detection using Video-based Estimation of Head Movement,”, pp. 1745-1749, 2018.
Nezami. O. M, Dras. M, Hamey. L, Richards. D, Wan. S and Paris. C, “Automatic Recognition of Student Engagement using Deep Learning and Facial Expression,”, pp. 1-18, 2018.
Zhang. Z, Li. Z, Liu. H, Cao. T and Liu. S, “Data-driven Online Learning Engagement Detection via Facial Expression and Mouse Behavior Recognition Technology,”,Vol. 58, pp. 63-86, 2019.
Chang. C, Zhang. C, Chen. L and Liu. Y, “An Ensemble Model Using Face and Body Tracking for Engagement Detection,”, pp. 616-622, 2018.
Huang. T, Mei. Y, Zhang. H, Liu. S and Yang. H, “Fine-grained Engagement Recognition in Online Learning Environment,”, pp. 338-341, 2019.
Sümer. Ö, Goldberg. P, Mello. S, Gerjets. P, Trautwein. U and Kasneci. E, “Multimodal Engagement Analysis from Facial Videos in the Classroom,”, pp. 1-15, 2021.
Sharma. P, Joshi S, Gautam. S, Maharjan. S, Filipe. V and Reis. M. J. S, “Student Engagement Detection Using Emotion Analysis, Eye Tracking and Head Movement with Machine Learning,”, pp. 1-18, 2019.
Booth. B. M, Ali. A. M, Narayanan. S. S, Bennett. I, and Farag. A. A, “Toward active and unobtrusive engagement assessment of distance learners,”, pp. 470-476, 2017.
Liu. Y, Chen. J, Zhang. M and Rao. C, “Student engagement study based on multi-cue detection and recognition in an intelligent learning environment. Multimedia Tools and Applications,”, Vol. 77, pp. 28749-28775, 2018.
T. S. A and Guddeti. R. M. R, “Automatic detection of students’ affective states in classroom environment using hybrid convolutional neural networks,”, Vol. 25, pp. 1387-1415, 2021.
Kaur. A, Mustafa. A, Mehta. L and Dhall. A, “Prediction and Localization of Student Engagement in the Wild,”, pp. 1-9, 2018.
Li. Y. Y and Hung. Y. P, “Feature Fusion of Face and Body for Engagement Intensity Detection,”, pp. 3312-3316, 2019.
Thiruthuvanathan. M, Krishnan. B and Rangaswamy. M, “Engagement Detection through Facial Emotional Recognition Using a Shallow Residual Convolutional Neural Networks,”, Vol. 14, pp.236-247, 2021.
Chang. C, Zhang. C, Chen. L and Liu. Y, “An Ensemble Model Using Face and Body Tracking for Engagement Detection,”, pp. 616-622, 2018.
Liao. J, Liang. Y and Pan. J, “Deep facial spatiotemporal network for engagement prediction in online learning,”, Vol. 51, pp. 6609-6621, 2021.
Gupta. A, Jaiswal. R, Adhikari. S and Balasubramanian. V, “DAISEE: Dataset for Affective States in E-Learning Environments,”, Vol. 14, pp.1-12, 2016.
Alnafjan. A, Hosny. M, Al-Wabil. A and Al-Ohali. Y, “Classification of Human Emotions from Electroencephalogram (EEG) Signal using Deep Neural Network,”, Vol. 8, pp. 419-425, 2017.
Abedi. A and Khan. S. S, “Improving state-of-the-art in Detecting Student Engagement with Resnet and TCN Hybrid Network,”, pp. 1-7, 2021.
Li. J, Ngai. G, Leong. H. V and Chan. S. C. F, “Multimodal Human Attention Detection for Reading from Facial Expression, Eye Gaze, and Mouse Dynamics,”, Vol. 16, pp. 37-49, 2016.
Sengupta. S, Basak. S, Saikia. P, Paul. S, Tsalavoutis. V, Atiah. F, Ravi. V and Peters. A, “A review of deep learning with special emphasis on architectures, applications and recent trends,”, pp. 1-29, 2021.
Jalal. M. A, Mihaylova. L and Moore. R. K, “An End-to-End Deep Neural Network for Facial Emotion Classification,” 2019 22th International Conference on Information Fusion (FUSION)., pp. 1–7., 2019.
Mollahosseini. A, Hasani. B and Mahoor. M. H, “AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild,”, pp. 1-18, 2017.
Simple MNIST convnet. (28 September 2020). [Online] Available : https://keras.io/examples/vision/mnist_convnet