An extended Two-Stage Sequential Optimization approach: Properties and performance

This paper looks into nonlinear non convex stochastic unconstrained optimization with finite simulation budget. Our work builds upon the Two-Stage Sequential Optimization (TSSO) algorithm that addresses the class of problems of interest by using the modified nugget effect kriging (MNEK) meta-model and proposing a budget allocation followed by a two-stage sequential procedure. Despite its efficiency and performance, we have observed that, given a finite budget, the choice of the number of replications per iteration, currently left to the user, is particularly critical for the algorithm performance. A fixed a-priori assignment can affect the ability to control the algorithm making it particularly sensitive to the initial settings. In this paper, we propose the extended TSSO (eTSSO). Specifically, a general simulation budget allocation scheme is proposed with the objective to balance the need of accurate function estimations to improve the selection in the search stage, with the need to explore the solution space. The new scheme adaptively, and recursively, increases the simulation budget based upon information iteratively returned by the optimizer itself. We analyze the asymptotic properties of eTSSO. Subsequently, we propose four alternative variants of the general allocation that we empirically analyze by comparing the quality of the estimated optimum input combination and the corresponding estimated optimum output against TSSO and other state of the art algorithms.