HORL_2OPT: A Hybrid Reinforcement Learning and Hippopotamus Optimization Algorithm for Bottled Water Delivery Route Optimization

Article Sidebar

PDF
Published: Dec 17, 2025
Keywords:
Bottled water logistics Hybrid Metaheuristic algorithms Reinforcement learning Route optimization

Main Article Content

Wanatchapong Kongkaew
Department of Industrial and Manufacturing Engineering, Faculty of Engineering, Prince of Songkla University, Songkhla 90110 Thailand
Phattara Khumprom
Graduate School of Management and Innovation, King Mongkut’s University of Technology Thonburi, Bangkok 10140 Thailand
Thanathip Limna
Department of Computer Engineering, Faculty of Engineering, Prince of Songkla University, Songkhla 90110 Thailand
Sirirat Suwatcharachaitiwong
Department of Industrial and Manufacturing Engineering, Faculty of Engineering, Prince of Songkla University, Songkhla 90110 Thailand
Dollaya Buakum
Department of Industrial and Manufacturing Engineering, Faculty of Engineering, Prince of Songkla University, Songkhla 90110 Thailand

Abstract

This study presents HORL_2OPT, a hybrid optimization framework developed to address the bottled water delivery routing problem modeled as a Traveling Salesman Problem (TSP). The framework aims to minimize travel distance, enhance computational efficiency, and ensure consistent solutions. HORL_2OPT combines three key components: 𝑄-learning for guided initialization, the Hippopotamus Optimization Algorithm (HOA) for global exploration, and a 2-opt heuristic for local route refinement. Tested on 15 TSPLIB benchmarks and 26 real-world cases from a bottled water distributor in southern Thailand, HORL_2OPT consistently produced the best or near-best results. For instance, it achieved a total distance of 8,034.2 in the berlin52 problem, outperforming HOA (12,953.2), DE (25,215.2), and PSO (23,187.0); and in lin318, it achieved 56,695.0 compared to HOA’s 85,286.2 and DFA’s 122,910.4. In real applications, it generated the shortest or equally optimal routes in 18 of 26 cases, occasionally surpassing LINGO, with most runs completed within 20 seconds. By integrating machine learning, metaheuristics, and local search, HORL_2OPT delivers robust, high-quality solutions suitable for practical logistics and dynamic routing scenarios.

Article Details

How to Cite
Kongkaew, W., Phattara Khumprom, Thanathip Limna, Sirirat Suwatcharachaitiwong, & Dollaya Buakum. (2025). HORL_2OPT: A Hybrid Reinforcement Learning and Hippopotamus Optimization Algorithm for Bottled Water Delivery Route Optimization. Science & Technology Asia, 30(4), 28–54. retrieved from https://ph02.tci-thaijo.org/index.php/SciTechAsia/article/view/258971
Issue
Vol.30 No.4 (October-December 2025)
Section
Physical sciences
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

References

Oonchokdee N, Pichitlamken J, Wangwatcharakul W. Cost reduction by fleet planning for parcel delivery service. Sci Technol Asia. 2024;29(2):74–84.

Rajwar K, Deep K, Das S. An exhaustive review of the metaheuristic algorithms for search and optimization: taxonomy, applications, and open challenges. Artif Intell Rev. 2023;56(11):13187–257.

Toaza B, EsztergĂĄr-Kiss D. A review of metaheuristic algorithms for solving TSPbased scheduling optimization problems. Appl Soft Comput. 2023Íž148:110908.

Abualigah L, Elaziz MA, Khasawneh AM, Alshinwan M, Ibrahim RA, Al-Qaness MA, et al. Meta-heuristic optimization algorithms for solving real-world mechanical engineering design problems: a comprehensive survey, applications, comparative analysis, and results. Neural Comput Appl. 2022;34(6):4081–110.

Saidi AKAA, Ayadi N. Enhancing logistics efficiency: a comprehensive study of Royal Solidex company. IJRDO [Internet]. 2025 [cited 2025 Jun 3]. Available from: https://doi.org/10.53555/bm.v11i1.6217

Tagaro JC, Valda DTS, Villa SE III, Yasuda MD. Logistics Optimization: A Literature Review of Techniques for Streamlining Land Transportation in Supply Chain Operations [Internet]. 2025 [cited 2025 Jun 3]. Available from: https://www.researchgate.net/publication/384246072

Hamrouni C, Alutaybi A, Ouerfelli G. Multi-agent mapping and tracking-based electrical vehicles with unknown environment exploration. World Electr Veh J. 2025Íž16(3):162.

Malathy V, Al-Jawahry HM, Madhura G K, Suganya G, Rashmi P. A reinforcement learning method in cooperative multi-agent system for production control system. In: Proceedings of the 2024 International Conference on Data Science and Network Security (ICDSNS); 2024. p. 1–4.

Lin C, Han G, Zhang T, Shah SBH, Peng Y. Smart underwater pollution detection based on graph-based multi-agent reinforcement learning towards AUV-based network ITS. IEEE Trans Intell Transp Syst. 2023;24(7):7494–505.

Van der Zwan M. Multi-agent task allocation and path planning for autonomous ground support equipment [master’s thesis]. Delft: TU Delft; 2023.

Cano JA, Gómez-Montoya RA, Salazar F, Cortés P. Disruptive and conventional technologies for the support of logistics processes: a literature review. Int J Technol. 2021;12(3):448–60.

Feng B, Ye Q. Operations management of smart logistics: a literature review and future research. Front Eng Manag. 2021;8(3):344–55.

Al Zadajali A, Ullah A. The effectiveness of logistics services on firms’ performances– a literature review. AJEBI. 2024;3(1):125–32.

Rymarczyk P, Bogacki S, Figura C, Rutkowski M, Staliński P. Optimizing order picking processes in warehouses: strategies for efficient routing and clustering. J Mod Sci. 2024;57(3):467–84.

Xiaoshan Y, Weiwei G. Research on logistics distribution route optimization based on deep learning model and block chain technology. 3C Empresa. 2023;12(1):68–85.

Toro OEM, Escobar ZAH, Granada EM. Literature review on the vehicle routing problem in the green transportation context. Luna Azul. 2016;42:362–87.

Singh H. Logistics optimization for circular economy: a vehicle route planning based analysis [master’s thesis]. Vaasa: University of Vaasa; 2024.

Liu L, Lee LS, Seow HV, Chen CY. Logistics center location-inventory-routing problem optimization: a systematic review using PRISMA method. Sustainability. 2022Íž14(23):15853.

Tan K, Liu W, Xu F, Li C. Optimization model and algorithm of logistics vehicle routing problem under major emergency. Mathematics. 2023Íž11(5):1274.

Indrianti N, Leuveano RAC, Abdul-Rashid SH, Ridho MI. Green vehicle routing problem optimization for LPG distribution: genetic algorithms for complex constraints and emission reduction. Sustainability. 2025Íž17(3):1144.

Okulewicz M, Mańdziuk J. A metaheuristic approach to solve dynamic vehicle routing problem in continuous search space. Swarm Evol Comput. 2019;48:44–61.

Abualigah L. Particle swarm optimization: advances, applications, and experimental insights. Comput Mater Contin. 2025;82(2):1539–92.

Gao M, Fu X, Dong G, Li H. An adaptive mutation multi-particle swarm optimization for traveling salesman prob-lem. In: Proceedings of the 3rd International Conference on Material, Mechanical and Manufacturing Engineering; 2015. p. 1003–1007.

Iliopoulou C, Kepaptsoglou K, Vlahogianni E. Metaheuristics for the transit route network design problem: a review and comparative analysis. Public Transp. 2019;11(3):487–521.

Rahman MA, Sokkalingam R, Othman M, Biswas K, Abdullah L, Kadir EA. Nature-inspired metaheuristic techniques for combinatorial optimization problems: overview and recent advances. Mathematics. 2021Íž9(20):2633.

Pan W, Liu SQ. Deep reinforcement learning for the dynamic and uncertain vehicle routing problem. Appl Intell. 2022;53(1):405–22.

Wang C, Cao Z, Wu Y, Teng L, Wu G. Deep reinforcement learning for solving vehicle routing problems with backhauls. IEEE Trans Neural Netw Learn Syst. 2025;36(3):4779–93.

Raza SM, Sajid M, Singh J. Vehicle routing problem using reinforcement learning: recent advancements. In: Gupta D, Sambyo K, Prasad M, Agarwal S, editors. Lect Notes Electr Eng. 2022. p. 269–80.

Chen YR, Rezapour A, Tzeng WG, Tsai SC. RL-routing: an SDN routing algorithm based on deep reinforcement learning. IEEE Trans Netw Sci Eng. 2020;7(4):3185–99.

Zhao J, Mao M, Zhao X, Zou J. A hybrid of deep reinforcement learning and local search for the vehicle routing problems. IEEE Trans Intell Transp Syst. 2021;22(11):7208–18.

Amiri MH, Hashjin NM, Montazeri M, Mirjalili S, Khodadadi N. Hippopotamus optimization algorithm: a novel natureinspired optimization algorithm. Sci Rep. 2024Íž14:5032.

Mehta P, Sait S, Yildiz B, Yildiz A. Enhanced hippopotamus optimization algorithm and artificial neural network for mechanical component design. Mater Test. 2025;67(4):655–62.

Han T, Wang H, Li T, Liu Q, Huang Y. MHO: a modified hippopotamus optimization algorithm for global optimization and engineering design problems. Biomimetics. 2025Íž10(2):90.

Maurya P, Tiwari P, Pratap A. Application of the hippopotamus optimization algorithm for distribution network reconfiguration with distributed generation considering different load models for enhancement of power system performance. Electr Eng. 2025;107:3909–46.

Kongkaew W, Koohathongsumrit N, Pongsathornwiwat A, Suwatcharachaitiwong S, Buakum D, Limna T, Khumprom P. Application of hippopotamus optimization algorithm for the routing problem in drinking water factory. In: Proceedings of the 24th Asia Pacific Industrial Engineering and Management Systems Conference (APIEMS); 2024. p. 332–37.

Mamatha MC, Sateesh Kumar HC. Channel estimation using deep learning techniques with hippopotamus optimization algorithm for millimeter wave MIMOOFDM system. Int J Intell Eng Syst. 2024;17(6):1314–24.

Campuzano G, Obreque C, Aguayo MM. Accelerating the Miller–Tucker–Zemlin model for the asymmetric traveling salesman problem. Expert Syst Appl. 2020;148:113229.

Storn R, Price K. Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim. 1997;11:341-59.

Storn R, Price K. Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim. 1997;11:341–59.

Ali IM, Essam D, Kasmarik K. A novel design of differential evolution for solving discrete traveling salesman problems. Swarm Evol Comput. 2019Íž52:100607.

Yang XS. Firefly algorithms for multimodal optimization. In: Proceedings of the 5th International Conference on Stochastic Algorithms: Foundations and Applications; 2009. p. 169–78.

Zhou L, Ding L, Qiang X. A multipopulation discrete firefly algorithm to solve TSP. In: Pan L, Păun G, Pérez-Jiménez MJ, Song T, editors. Bio-inspired Computing – Theories and Applications. Berlin, Heidelberg: Springer; 2014. p. 648–53.

Pitakaso R, Sethanan K, Jamrus T. Hybrid PSO and ALNS algorithm for software and mobile application for transportation in ice manufacturing industry 3.5. Comput Ind Eng. 2020Íž144:106461.

Mirjalili S, Mirjalili SM, Lewis A. Grey wolf optimizer. Adv Eng Softw. 2014;69:46–61.