HORL_2OPT: A Hybrid Reinforcement Learning and Hippopotamus Optimization Algorithm for Bottled Water Delivery Route Optimization
Article Sidebar
Main Article Content
Abstract
This study presents HORL_2OPT, a hybrid optimization framework developed to address the bottled water delivery routing problem modeled as a Traveling Salesman Problem (TSP). The framework aims to minimize travel distance, enhance computational efficiency, and ensure consistent solutions. HORL_2OPT combines three key components: š-learning for guided initialization, the Hippopotamus Optimization Algorithm (HOA) for global exploration, and a 2-opt heuristic for local route refinement. Tested on 15 TSPLIB benchmarks and 26 real-world cases from a bottled water distributor in southern Thailand, HORL_2OPT consistently produced the best or near-best results. For instance, it achieved a total distance of 8,034.2 in the berlin52 problem, outperforming HOA (12,953.2), DE (25,215.2), and PSO (23,187.0)Ķ¾ and in lin318, it achieved 56,695.0 compared to HOAās 85,286.2 and DFAās 122,910.4. In real applications, it generated the shortest or equally optimal routes in 18 of 26 cases, occasionally surpassing LINGO, with most runs completed within 20 seconds. By integrating machine learning, metaheuristics, and local search, HORL_2OPT delivers robust, high-quality solutions suitable for practical logistics and dynamic routing scenarios.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
Oonchokdee N, Pichitlamken J, Wangwatcharakul W. Cost reduction by fleet planning for parcel delivery service. Sci Technol Asia. 2024Ķ¾29(2):74ā84.
Rajwar K, Deep K, Das S. An exhaustive review of the metaheuristic algorithms for search and optimization: taxonomy, applications, and open challenges. Artif Intell Rev. 2023Ķ¾56(11):13187ā257.
Toaza B, EsztergĆ”r-Kiss D. A review of metaheuristic algorithms for solving TSPbased scheduling optimization problems. Appl Soft Comput. 2023Ķ¾148:110908.
Abualigah L, Elaziz MA, Khasawneh AM, Alshinwan M, Ibrahim RA, Al-Qaness MA, et al. Meta-heuristic optimization algorithms for solving real-world mechanical engineering design problems: a comprehensive survey, applications, comparative analysis, and results. Neural Comput Appl. 2022Ķ¾34(6):4081ā110.
Saidi AKAA, Ayadi N. Enhancing logistics efficiency: a comprehensive study of Royal Solidex company. IJRDO [Internet]. 2025 [cited 2025 Jun 3]. Available from: https://doi.org/10.53555/bm.v11i1.6217
Tagaro JC, Valda DTS, Villa SE III, Yasuda MD. Logistics Optimization: A Literature Review of Techniques for Streamlining Land Transportation in Supply Chain Operations [Internet]. 2025 [cited 2025 Jun 3]. Available from: https://www.researchgate.net/publication/384246072
Hamrouni C, Alutaybi A, Ouerfelli G. Multi-agent mapping and tracking-based electrical vehicles with unknown environment exploration. World Electr Veh J. 2025Ķ¾16(3):162.
Malathy V, Al-Jawahry HM, Madhura G K, Suganya G, Rashmi P. A reinforcement learning method in cooperative multi-agent system for production control system. In: Proceedings of the 2024 International Conference on Data Science and Network Security (ICDSNS)Ķ¾ 2024. p. 1ā4.
Lin C, Han G, Zhang T, Shah SBH, Peng Y. Smart underwater pollution detection based on graph-based multi-agent reinforcement learning towards AUV-based network ITS. IEEE Trans Intell Transp Syst. 2023Ķ¾24(7):7494ā505.
Van der Zwan M. Multi-agent task allocation and path planning for autonomous ground support equipment [masterās thesis]. Delft: TU DelftĶ¾ 2023.
Cano JA, GĆ³mez-Montoya RA, Salazar F, CortĆ©s P. Disruptive and conventional technologies for the support of logistics processes: a literature review. Int J Technol. 2021Ķ¾12(3):448ā60.
Feng B, Ye Q. Operations management of smart logistics: a literature review and future research. Front Eng Manag. 2021Ķ¾8(3):344ā55.
Al Zadajali A, Ullah A. The effectiveness of logistics services on firmsā performancesā a literature review. AJEBI. 2024Ķ¾3(1):125ā32.
Rymarczyk P, Bogacki S, Figura C, Rutkowski M, StaliÅski P. Optimizing order picking processes in warehouses: strategies for efficient routing and clustering. J Mod Sci. 2024Ķ¾57(3):467ā84.
Xiaoshan Y, Weiwei G. Research on logistics distribution route optimization based on deep learning model and block chain technology. 3C Empresa. 2023Ķ¾12(1):68ā85.
Toro OEM, Escobar ZAH, Granada EM. Literature review on the vehicle routing problem in the green transportation context. Luna Azul. 2016Ķ¾42:362ā87.
Singh H. Logistics optimization for circular economy: a vehicle route planning based analysis [masterās thesis]. Vaasa: University of VaasaĶ¾ 2024.
Liu L, Lee LS, Seow HV, Chen CY. Logistics center location-inventory-routing problem optimization: a systematic review using PRISMA method. Sustainability. 2022Ķ¾14(23):15853.
Tan K, Liu W, Xu F, Li C. Optimization model and algorithm of logistics vehicle routing problem under major emergency. Mathematics. 2023Ķ¾11(5):1274.
Indrianti N, Leuveano RAC, Abdul-Rashid SH, Ridho MI. Green vehicle routing problem optimization for LPG distribution: genetic algorithms for complex constraints and emission reduction. Sustainability. 2025Ķ¾17(3):1144.
Okulewicz M, MaÅdziuk J. A metaheuristic approach to solve dynamic vehicle routing problem in continuous search space. Swarm Evol Comput. 2019Ķ¾48:44ā61.
Abualigah L. Particle swarm optimization: advances, applications, and experimental insights. Comput Mater Contin. 2025Ķ¾82(2):1539ā92.
Gao M, Fu X, Dong G, Li H. An adaptive mutation multi-particle swarm optimization for traveling salesman prob-lem. In: Proceedings of the 3rd International Conference on Material, Mechanical and Manufacturing EngineeringĶ¾ 2015. p. 1003ā1007.
Iliopoulou C, Kepaptsoglou K, Vlahogianni E. Metaheuristics for the transit route network design problem: a review and comparative analysis. Public Transp. 2019Ķ¾11(3):487ā521.
Rahman MA, Sokkalingam R, Othman M, Biswas K, Abdullah L, Kadir EA. Nature-inspired metaheuristic techniques for combinatorial optimization problems: overview and recent advances. Mathematics. 2021Ķ¾9(20):2633.
Pan W, Liu SQ. Deep reinforcement learning for the dynamic and uncertain vehicle routing problem. Appl Intell. 2022Ķ¾53(1):405ā22.
Wang C, Cao Z, Wu Y, Teng L, Wu G. Deep reinforcement learning for solving vehicle routing problems with backhauls. IEEE Trans Neural Netw Learn Syst. 2025Ķ¾36(3):4779ā93.
Raza SM, Sajid M, Singh J. Vehicle routing problem using reinforcement learning: recent advancements. In: Gupta D, Sambyo K, Prasad M, Agarwal S, editors. Lect Notes Electr Eng. 2022. p. 269ā80.
Chen YR, Rezapour A, Tzeng WG, Tsai SC. RL-routing: an SDN routing algorithm based on deep reinforcement learning. IEEE Trans Netw Sci Eng. 2020Ķ¾7(4):3185ā99.
Zhao J, Mao M, Zhao X, Zou J. A hybrid of deep reinforcement learning and local search for the vehicle routing problems. IEEE Trans Intell Transp Syst. 2021Ķ¾22(11):7208ā18.
Amiri MH, Hashjin NM, Montazeri M, Mirjalili S, Khodadadi N. Hippopotamus optimization algorithm: a novel natureinspired optimization algorithm. Sci Rep. 2024Ķ¾14:5032.
Mehta P, Sait S, Yildiz B, Yildiz A. Enhanced hippopotamus optimization algorithm and artificial neural network for mechanical component design. Mater Test. 2025Ķ¾67(4):655ā62.
Han T, Wang H, Li T, Liu Q, Huang Y. MHO: a modified hippopotamus optimization algorithm for global optimization and engineering design problems. Biomimetics. 2025Ķ¾10(2):90.
Maurya P, Tiwari P, Pratap A. Application of the hippopotamus optimization algorithm for distribution network reconfiguration with distributed generation considering different load models for enhancement of power system performance. Electr Eng. 2025Ķ¾107:3909ā46.
Kongkaew W, Koohathongsumrit N, Pongsathornwiwat A, Suwatcharachaitiwong S, Buakum D, Limna T, Khumprom P. Application of hippopotamus optimization algorithm for the routing problem in drinking water factory. In: Proceedings of the 24th Asia Pacific Industrial Engineering and Management Systems Conference (APIEMS)Ķ¾ 2024. p. 332ā37.
Mamatha MC, Sateesh Kumar HC. Channel estimation using deep learning techniques with hippopotamus optimization algorithm for millimeter wave MIMOOFDM system. Int J Intell Eng Syst. 2024Ķ¾17(6):1314ā24.
Campuzano G, Obreque C, Aguayo MM. Accelerating the MillerāTuckerāZemlin model for the asymmetric traveling salesman problem. Expert Syst Appl. 2020Ķ¾148:113229.
Storn R, Price K. Differential evolution ā a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim. 1997Ķ¾11:341-59.
Storn R, Price K. Differential evolution ā a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim. 1997Ķ¾11:341ā59.
Ali IM, Essam D, Kasmarik K. A novel design of differential evolution for solving discrete traveling salesman problems. Swarm Evol Comput. 2019Ķ¾52:100607.
Yang XS. Firefly algorithms for multimodal optimization. In: Proceedings of the 5th International Conference on Stochastic Algorithms: Foundations and ApplicationsĶ¾ 2009. p. 169ā78.
Zhou L, Ding L, Qiang X. A multipopulation discrete firefly algorithm to solve TSP. In: Pan L, PÄun G, PĆ©rez-JimĆ©nez MJ, Song T, editors. Bio-inspired Computing ā Theories and Applications. Berlin, Heidelberg: SpringerĶ¾ 2014. p. 648ā53.
Pitakaso R, Sethanan K, Jamrus T. Hybrid PSO and ALNS algorithm for software and mobile application for transportation in ice manufacturing industry 3.5. Comput Ind Eng. 2020Ķ¾144:106461.
Mirjalili S, Mirjalili SM, Lewis A. Grey wolf optimizer. Adv Eng Softw. 2014Ķ¾69:46ā61.