Gradient-Based Adaptive Stochastic Search for Simulation Optimization Over Continuous Space

We extend the idea of model-based algorithms for deterministic optimization to simulation optimization over continuous space. Model-based algorithms iteratively generate a population of candidate solutions from a sampling distribution and use the performance of the candidate solutions to update the sampling distribution. By viewing the original simulation optimization problem as another optimization problem over the parameter space of the sampling distribution, we propose to use a direct gradient search on the parameter space to update the sampling distribution. To improve the computational efficiency, we further develop a two-timescale updating scheme that updates the parameter on a slow timescale and estimates the quantities involved in the parameter updating on the fast timescale. We analyze the convergence properties of our algorithms through techniques from stochastic approximation, and demonstrate the good empirical performance by comparing with two state-of-the-art model-based simulation optimization methods.