Minimization of a class of rare event probabilities and buffered probabilities of exceedance

We consider the problem of choosing design parameters to minimize the probability of an undesired rare event that is described through the average of n i.i.d. random variables. Since the probability of interest for near optimal design parameters is very small, one needs to develop suitable accelerated Monte-Carlo methods for estimating its value. One of the challenges in the study is that simulating from exponential twists of the laws of the summands may be computationally demanding since these transformed laws may be non-standard and intractable. We consider a setting where the summands are given as a nonlinear functional of random variables, the exponential twists of whose distributions take a simpler form than those for the original summands. We use techniques from Dupuis and Wang (Stochastics 76(6):481–508, 2004, Math Oper Res 32(3):723–757, 2007) to identify the appropriate Issacs equations whose subsolutions are used to construct tractable importance sampling (IS) schemes. We also study the closely related problem of estimating buffered probability of exceedance and provide the first rigorous results that relate the asymptotics of buffered probability and that of the ordinary probability under a large deviation scaling. The analogous minimization problem for buffered probability, under conditions, can be formulated as a convex optimization problem. We show that, under conditions, changes of measures that are asymptotically efficient (under the large deviation scaling) for estimating ordinary probability are also asymptotically efficient for estimating the buffered probability of exceedance. We embed the constructed IS scheme in gradient descent algorithms to solve the optimization problems, and illustrate these schemes through computational experiments.