Evaluating the Performance of YOLO Architectures for Effective Gun and Knife Detection

Suradej Intagorn; Mathuros Panmuang; Chonnikarn Rodmorn; Suriya Pinitkan

Authors

Suradej Intagorn Kasetsart University, Thailand
Mathuros Panmuang Rajamangala University of Technology Thanyaburi, Thailand
Chonnikarn Rodmorn King Mongkut’s University of Technology North Bangkok, Thailand
Suriya Pinitkan Kasetsart University, Thailand

Keywords:

YOLO, machine learning, weapon detection, image processing, Thailand

Abstract

In recent years, the rise in mass shootings in Thailand has highlighted the need for more comprehensive and cost-effective security solutions. One approach is using artificial intelligence to assist human security personnel, particularly for weapon detection through security cameras. Although advancements in deep learning and computer vision have made it possible to deploy such systems on edge computing devices, real-time weapon detection still faces challenges like accuracy and latency. This study addresses the gap in weapon detection research specific to Thailand by utilizing a dataset featuring local environments and weapons, which differ from those in existing datasets. We compare the performance of YOLO versions 5 through 8, focusing on their mean average precision (mAP) in detecting guns and knives. Since each YOLO version is developed by different research teams and may perform differently under specific conditions, our evaluation considers these variations. The findings indicate that YOLOv8 achieves the highest mAP, with scores of 0.874 on the validation set and 0.848 on the test set, demonstrating its effectiveness in the Thai context.

Author Biographies

Suradej Intagorn, Kasetsart University, Thailand

Department of Computational Science and Digital Technology, Kasetsart University, Kamphaeng Saen Campus, Kamphaeng Saen Sub-District, Kamphaeng Saen District, Nakhon Pathom, Thailand

Mathuros Panmuang, Rajamangala University of Technology Thanyaburi, Thailand

Department of Educational Technology and Communications, Faculty of Technical Education, Rajamangala University of Technology Thanyaburi, Pathum Thani, Thailand

Chonnikarn Rodmorn, King Mongkut’s University of Technology North Bangkok, Thailand

Department of Applied Statistics, Faculty of Applied Science, King Mongkut’s University of Technology North Bangkok, Bangkok, Thailand

Suriya Pinitkan, Kasetsart University, Thailand

Department of Computational Science and Digital Technology, Kasetsart University, Kamphaeng Saen Campus,Kamphaeng Saen Sub-District, Kamphaeng Saen District, Nakhon Pathom, Thailand

References

M. E. Atik, Z. Duran and R. Ozgunluk, “Comparison of YOLO versions for object detection from aerial images,” International Journal of Environment and Geoinformatics, vol. 9, no. 2, pp. 87-93, Jun. 2022, doi: 10.30897/ijegeo.1010741.

Y. Liu, P. Sun, N. Wergeles and Y. Shang, “A survey and performance evaluation of deep learning methods for small object detection,” Expert Systems with Applications, vol. 172, no. 2021, pp. 114602: 1-14, Jun. 2021, doi: 10.1016/j.eswa.2021.114602.

Budi, Yasmine Syifa Nabila, et al. "Daycare shooting in Thailand: time to address firearm violence as a global health issue." Public health challenges 2.2 (2023): e93.

M. M. Fernandez-Carrobles, O. Deniz and F. Maroto, “Gun and knife detection based on faster R-CNN for video surveillance,” In: Morales, A., Fierrez, J., Sánchez, J., Ribeiro, B. (eds) Pattern Recognition and Image Analysis. IbPRIA 2019. Lecture Notes in Computer Science(), vol. 11868, Springer, 2019 Cham, pp. 441-452, Sep. 2019, doi: 10.1007/978-3-030-31321-0_38.

H. Hu, C. Tang, C. Shi and Y. Qian, “Detection of residual feed in aquaculture using YOLO and mask RCNN,” Aquacultural Engineering, vol. 100, pp. 102304: 1 -8, Feb. 2023, doi: 10.1016/j.aquaeng.2022.102304.

P. Jiang, D. Ergu, F. Liu, Y. Cai and B. Ma, “A review of Yolo algorithm developments,” Procedia Computer Science, vol. 199, pp. 1066-1073, 2022, doi: 10.1016/j.procs.2022.01.135.

Li, Chuyi, et al. "Yolov6 v3. 0: A full-scale reloading." arXiv preprint arXiv:2301.05586 (2023).

M. Pullakandam, K. Loya, P. Salota, R. M. R. Yanamala and P. K. Javvaji, “Weapon object detection using quantized YOLOv8,” 2023 5th International Conference on Energy, Power and Environment: Towards Flexible Green Energy Technologies (ICEPE), Shillong, India, 2023, pp. 1-5, Jun. 2023, doi: 10.1109/ICEPE57949.2023.10201506.

T. Chompookham and O. Surinta, “Ensemble methods with deep convolution neural networks for plant leaf recognition, ICIC Express Letters, vol. 15, no. 6, pp. 553-565, Jun. 2021, doi: 10.24507/icicel.15.06.553.

S. Intagorn, S. Pinitkan, M. Panmuang and C. Rodmorn, “Helmet detection system for motorcycle riders with explainable artificial intelligence using convolutional neural network and Grad-CAM,” In: Surinta, O., Kam Fung Yuen, K. (eds) Multi-disciplinary Trends in Artificial Intelligence. MIWAI 2022. Lecture Notes in Computer Science(), vol. 13651, Springer, Cham, pp. 40-51, Nov. 2022, doi: 10.1007/978-3-031-20992-5_4.

S. Luo, J. Yu, Y. Xi and X. Liao, “Aircraft target detection in remote sensing images based on improved YOLOv5,” IEEE Access, vol. 10, pp. 5184-5192, Jan. 2022, doi: 10.1109/ACCESS.2022.3140876.

L. Cheng, J. Li, P. Duan and M. Wang, “A small attentional YOLO model for landslide detection from satellite remote sensing images,” Landslides, vol. 18, no. 8, pp. 2751-2765, May 2021, doi: 10.1007/s10346-021-01694-6.

A. Dumitriu, F. Tatui, F. Miron, R. T. Ionescu and R. Timofte, “Rip current segmentation: A novel benchmark and YOLOv8 baseline results,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops 2023, pp. 1261-1271.

X. Jia, Y. Wang and T. Chen, “Forest fire detection and recognition using YOLOv8 algorithms from UAVs images,” 2023 IEEE 5th International Conference on Power, Intelligent Computing and Systems (ICPICS), Shenyang, China, pp. 646-651, Jul. 2023, doi: 10.1109/ICPICS58376.2023.10235675.

Shah, Syed Atif Ali, Mahmoud Ahmad Al-Khasawneh, and M. Irfan Uddin. "Review of weapon detection techniques within the scope of street-crimes." 2021 2nd International Conference on Smart Computing and Electronic Enterprise (ICSCEE). IEEE, 2021.

Narejo, Sanam, et al. "Weapon detection using YOLO V3 for smart surveillance system." Mathematical Problems in Engineering 2021.1 (2021): 9975700.

Grega, Michał, et al. "Automated detection of firearms and knives in a CCTV image." Sensors 16.1 (2016): 47.

Bhatti, Muhammad Tahir, et al. "Weapon detection in real-time cctv videos using deep learning." Ieee Access 9 (2021): 34366-34382.

Redmon, Joseph, et al. "You only look once: Unified, real-time object detection." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.

Kang, Junhyung, et al. "A survey of deep learning-based object detection methods and datasets for overhead imagery." IEEE Access 10 (2022): 20118-20134.

J. Du, “Understanding of object detection based on CNN family and YOLO,” Journal of Physics: Conference Series, 2nd International Conference on Machine Vision and Information Technology (CMVIT 2018) 23–25 February 2018, Hong Kong, vol. 1004, Feb. 2018, doi: 10.1088/1742-6596/1004/1/012029.

J. R. Terven and D. M. Cordova-Esparza, “A comprehensive review of YOLO: From YOLOv1 and beyond,” Under Review in ACM Computing Surveys, 1-27, Apr. 2023.

C. Li, L. Li, H. Jiang, K. Weng, Y. Geng, L. Li, Z. Ke, Q. Li, M. Cheng, W. Nie, Y. Li, B. Zhang, Y. Liang, L. Zhou, X. Xu, X. Chu, X. Wei and X. Wei, “YOLOv6: A single-stage object detection framework for industrial applications,” pp. 1-17, Sep. 2022, doi: 10.48550/arXiv.2209.02976.

CY. Wang, A. Bochkovskiy and HY. Mark Liao, “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7464-7475, 2023.

A. Malta, M. Mendes and T. Farinha, “Augmented reality maintenance assistant using YOLOv5,” Applied Sciences, vol. 11 no. 11, pp. 4758, May 2021, doi: 10.3390/app11114758.

M. Lippi, N. Bonucci, R. F. Carpio, M. Contarini, S. Speranza and A. Gasparri, “A yolo-based pest detection system for precision agriculture,” In 2021 29th Mediterranean Conference on Control and Automation (MED), PUGLIA, Italy, IEEE, pp. 342-347, Jun. 2021, doi: 10.1109/MED51440.2021.9480344.

A. Aboah, B. Wang, U. Bagci and Y. Adu-Gyamfi, “Real-time multi-class helmet violation detection using few-shot data sampling technique and yolov8.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 5350-5358.