Nelder–Mead Method with Local Selection Using Memory for Discrete Stochastic Optimization

Abstract

We consider the Nelder–Mead (NM) simplex algorithm for optimization of discrete–event stochastic simulation models. We propose seven new modified algorithms based on NM to reduce computational time and to improve quality of the estimated optimal solutions. Our modifications include utilizing past information of already seen solutions, using adaptive sample size, and using Restart. We compare the performance of these extensions on four test functions with 5 levels of random variations. We find that using past information leads to reduction of computational efforts by up to 50%. The adaptive modifications need fewer resources than the non-adaptive counterparts by up to 30% but there is no difference in quality. Although the Restart consumes 60% more resources than non–Restart, the Restart gives up to 50% improvement in the optimal solution. In summary, we find that a Restart with memory algorithm gives the best performance either with or without adaptivity.