Applied Ensemble Technique for Road Damage Detection based on YOLOv8

Zhipeng Tang; Apirak Jirayusakun; Rapeeporn Chamchong

Authors

Zhipeng Tang Mahasarakham University, Thailand
Apirak Jirayusakun Ramkhamhaeng University, Thailand
Rapeeporn Chamchong Mahasarakham University, Thailand

Keywords:

ensemble technique, object detection, road damage detection, deep learning

Abstract

The presence of road damage poses significant risks to pedestrians and traffic. Although deep learning-based object detection is widely used, the effectiveness of various detections shows considerable variation, and there remains substantial potential for enhancement. This paper proposes an ensemble technique using YOLOv8 for road damage detection. A comparison of object detection models, including YOLOv5, YOLOv8, Faster R-CNN, and SSD, is conducted to determine the best baseline. The study focuses on single-class detection to enhance accuracy in identifying specific types of road damage. Each model predicts class probabilities and bounding box locations. The predictions are then ensembled, with Non-Maximum Suppression applied to filter out overlapping detection boxes. The ensemble YOLOv8 model outperforms the standard one, especially in detecting alligator cracks and potholes, with detection accuracy improved by up to 3%. The method balances precision and recall effectively, suitable for complex road environments.

References

E. H. Manurung, K. Sawito, A. Satoto, and N. Tuanany, “Analysis of the Causes of Road Damage,” civilla, vol. 7, no. 1, p. 87, Apr. 2022, doi: 10.30736/cvl.v7i1.793.

T. Tsubota, C. Fernando, T. Yoshii, and H. Shirayanagi, “Effect of Road Pavement Types and Ages on Traffic Accident Risks,” Transportation Research Procedia, vol. 34, pp. 211–218, 2018, doi: 10.1016/j.trpro.2018.11.034.

J. Wu, Y. Zhang, and X. Zhao, “A Review of Image-Based Pavement Crack Detection Algorithms,” in 2021 40th Chinese Control Conference (CCC), Shanghai, China: IEEE, Jul. 2021, pp. 7300–7306. doi: 10.23919/CCC52363.2021.9549966.

Z. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye, “Object Detection in 20 Years: A Survey,” Proceedings of the IEEE, vol. 111, no. 3, pp. 257–276, 2023, doi: 10.1109/JPROC.2023.3238524.

X. Wu, D. Sahoo, and S. C. H. Hoi, “Recent Advances in Deep Learning for Object Detection,” Neurocomputing, vol. 396, pp. 39–64, Jul. 5, 2020, doi: 10.1016/j.neucom.2020.01.085.

G. Shen, “Road crack detection based on video image processing,” in 2016 3rd International Conference on Systems and Informatics (ICSAI), Shanghai, China: IEEE, Nov. 2016, pp. 912–917. doi: 10.1109/ICSAI.2016.7811081.

N. Ma et al., “Computer vision for road imaging and pothole detection: a state-of-the-art review of systems and algorithms,” Transp Safety Env, vol. 4, no. 4, p. tdac026, Nov. 2022, doi: 10.1093/tse/tdac026.

M. Carranza-García, J. Torres-Mateo, P. Lara-Benítez, and J. García-Gutiérrez, “On the Performance of One-Stage and Two-Stage Object Detectors in Autonomous Vehicles Using Camera Data,” Remote Sensing, vol. 13, no. 1, Art. no. 1, Jan. 2021, doi: 10.3390/rs13010089.

A. A. Mustapha and M. S. Yoosuf, “Exploring the efficacy and comparative analysis of one-stage object detectors for computer vision: a review,” Multimed Tools Appl, vol. 83, no. 20, pp. 59143–59168, Jun. 2024, doi: 10.1007/s11042-023-17751-2.

S. K. Pal, A. Pramanik, J. Maiti, and P. Mitra, “Deep learning in multi-object detection and tracking: state of the art,” Appl Intell, vol. 51, no. 9, pp. 6400–6429, Sep. 2021, doi: 10.1007/s10489-021-02293-7.

L. Zhang, F. Yang, Y. Daniel Zhang, and Y. J. Zhu, “Road crack detection using deep convolutional neural network,” in 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA: IEEE, Sep. 2016, pp. 3708–3712. doi: 10.1109/ICIP.2016.7533052.

H. Maeda, Y. Sekimoto, T. Seto, T. Kashiyama, and H. Omata, “Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images: Road damage detection and classification,” Comput-aided Civ Inf, vol. 33, no. 12, pp. 1127–1141, Dec. 2018, doi: 10.1111/mice.12387.

A. C. Aqsa, H. Mahmudah, and R. W. Sudibyo, “Detection and Classification of Road Damage Using CNN with Hyperparameter Optimization,” in 2022 6th International Conference on Informatics and Computational Sciences (ICICoS), Semarang, Indonesia: IEEE, Sep. 2022, pp. 101–104. doi: 10.1109/ICICoS56336.2022.9930607.

V. Mandal, L. Uong, and Y. Adu-Gyamfi, “Automated Road Crack Detection Using Deep Convolutional Neural Networks,” in 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA: IEEE, Dec. 2018, pp. 5212–5215. doi: 10.1109/BigData.2018.8622327.

A. Alfarrarjeh, D. Trivedi, S. H. Kim, and C. Shahabi, “A Deep Learning Approach for Road Damage Detection from Smartphone Images,” in 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA: IEEE, Dec. 2018, pp. 5201–5204. doi: 10.1109/BigData.2018.8621899.

S. A. Hassan, S. H. Han, and S. Y. Shin, “Real-time Road Cracks Detection based on Improved Deep Convolutional Neural Network,” in 2020 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE), London, ON, Canada: IEEE, Aug. 2020, pp. 1–4. doi: 10.1109/CCECE47787.2020.9255771.

N. Camilleri and T. Gatt, “Detecting road potholes using computer vision techniques,” in 2020 IEEE 16th International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania: IEEE, Sep. 2020, pp. 343–350. doi: 10.1109/ICCP51029.2020.9266138.

A. Angulo, J. A. Vega-Fernández, L. M. Aguilar-Lobo, S. Natraj, and G. Ochoa-Ruiz, “Road Damage Detection Acquisition System Based on Deep Neural Networks for Physical Asset Management,” in Advances in Soft Computing, vol. 11835, L. Martínez-Villaseñor, I. Batyrshin, and A. Marín-Hernández, Eds., in Lecture Notes in Computer Science, vol. 11835. , Cham: Springer International Publishing, 2019, pp. 3–14. doi: 10.1007/978-3-030-33749-0_1.

D. Jeong, “Road Damage Detection Using YOLO with Smartphone Images,” in 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA: IEEE, Dec. 2020, pp. 5559–5562. doi: 10.1109/BigData50022.2020.9377847.

S. Anand, S. Gupta, V. Darbari, and S. Kohli, “Crack-pot: Autonomous Road Crack and Pothole Detection,” in 2018 Digital Image Computing: Techniques and Applications (DICTA), Canberra, Australia: IEEE, Dec. 2018, pp. 1–6. doi: 10.1109/DICTA.2018.8615819.

R. Bibi et al., “Edge AI-Based Automated Detection and Classification of Road Anomalies in VANET Using Deep Learning,” Computational Intelligence and Neuroscience, vol. 2021, pp. 1–16, Sep. 2021, doi: 10.1155/2021/6262194.

Q. Chen, X. Gan, W. Huang, J. Feng, and H. Shim, “Road Damage Detection and Classification Using Mask R-CNN with DenseNet Backbone,” Computers, Materials & Continua, vol. 65, no. 3, pp. 2201–2215, 2020, doi: 10.32604/cmc.2020.011191.

R. Fan, H. Wang, Y. Wang, M. Liu, and I. Pitas, “Graph Attention Layer Evolves Semantic Segmentation for Road Pothole Detection: A Benchmark and Algorithms,” Ieee T Image Process, vol. 30, pp. 8144–8154, 2021, doi: 10.1109/TIP.2021.3112316.

Y. Liu, X. Zhang, B. Zhang, and Z. Chen, “Deep Network For Road Damage Detection,” in 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA: IEEE, Dec. 2020, pp. 5572–5576. doi: 10.1109/BigData50022.2020.9377991.

V. Pham, C. Pham, and T. Dang, “Road Damage Detection and Classification with Detectron2 and Faster R-CNN,” in 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA: IEEE, Dec. 2020, pp. 5592–5601. doi: 10.1109/BigData50022.2020.9378027.

T. Rateke, K. A. Justen, and A. Von Wangenheim, “Road Surface Classification with Images Captured From Low-cost Camera - Road Traversing Knowledge (RTK) Dataset,” RITA, vol. 26, no. 3, pp. 50–64, Nov. 2019, doi: 10.22456/2175-2745.91522.

M. Hussain, “YOLO-v1 to YOLO-v8, the Rise of YOLO and Its Complementary Nature toward Digital Manufacturing and Industrial Defect Detection,” Machines, vol. 11, no. 7, Art. no. 7, Jul. 2023, doi: 10.3390/machines11070677.

X. Dong, Z. Yu, W. Cao, Y. Shi, and Q. Ma, “A survey on ensemble learning,” Frontiers of Computer Science, vol. 14, no. 2, pp. 241–258, Apr. 2020, doi: 10.1007/s11704-019-8208-z.

W. Ding et al., “An Ensemble of One-Stage and Two-Stage Detectors Approach for Road Damage Detection,” in 2022 IEEE International Conference on Big Data (Big Data), Dec. 2022, pp. 6395–6400. doi: 10.1109/BigData55660.2022.10021000.

K. Doshi and Y. Yilmaz, “Road Damage Detection using Deep Ensemble Learning,” in 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA: IEEE, Dec. 2020, pp. 5540–5544. doi: 10.1109/BigData50022.2020.9377774.

S. Wang et al., “An Ensemble Learning Approach with Multi-depth Attention Mechanism for Road Damage Detection,” in 2022 IEEE International Conference on Big Data (Big Data), Dec. 2022, pp. 6439–6444. doi: 10.1109/BigData55660.2022.10021018.

V. Hegde, D. Trivedi, A. Alfarrarjeh, A. Deepak, S. Ho Kim, and C. Shahabi, “Yet Another Deep Learning Approach for Road Damage Detection using Ensemble Learning,” in 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA: IEEE, Dec. 2020, pp. 5553–5558.

A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "YOLOv4: Optimal speed and accuracy of object detection," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 6626-6635.

D. Arya, H. Maeda, S. K. Ghosh, D. Toshniwal, and Y. Sekimoto, “Rdd2022: A multi-national image dataset for automatic road damage detection,’’ Geoscience Data Journal, vol. 11, no. 4, pp. 846–862, 2024.

Sanjeewani, P., Verma, B., “Single class detection-based deep learning approach for identification of road safety attributes,” Neural Comput & Applic 33, 9691–9702. Feb. 2021.