Actor-Critic–Like Stochastic Adaptive Search for Continuous Simulation Optimization

We propose a random search method for solving a class of simulation optimization problems with Lipschitz continuity properties. The algorithm samples candidate solutions from a parameterized probability distribution over the solution space and estimates the performance of the sampled points through an asynchronous learning procedure based on the so-called shrinking ball method. A distinctive feature of the algorithm is that it fully retains the previous simulation information and incorporates an approximation architecture to exploit knowledge of the objective function in searching for improved solutions. Each step of the algorithm involves simultaneous adaptation of a parameterized distribution and an approximator of the objective function, which is akin to the actor-critic structure used in reinforcement learning. We establish a finite-time probability bound on the algorithm’s performance and show its global convergence when only a single simulation observation is collected at each iteration. Empirical results indicate that the algorithm is promising and may outperform some of the existing procedures in terms of efficiency and reliability.