Applications to Random Search Methods with Joint Normal Search Variates

As an application of the previous description of a general method to accelerate random search algorithms, in the following we consider search variates $$Z_{n+1}$$ Z n + 1 at an iteration point $$X_n=x_n$$ X n = x n having a joint normal conditional distribution with mean and covariance matrix $$(\mu ,\Lambda )= (\mu (x_n),\Lambda (x_n))$$ ( μ , Λ ) = ( μ ( x n ) , Λ ( x n ) ) . The mean search gain for a step $$X_n \rightarrow X_{n+1}$$ X n → X n + 1 is determined by means of the mean decrease of the objective function. For simplification, instead of the infinite-stage optimal decision process for the selection of the parameters of the joint normal distribution, only the optimal one-step, $$X_n \rightarrow X_{n+1}$$ X n → X n + 1 , gains are taken into account, where the convergence rate of the fixed parameter and the optimized search method is evaluated. Since the optimal parameters of the normal distribution depend on the gradient and Hesse matrix of the objective function F, in a numerical realization of the optimal RSM, Quasi-Newton methods can be applied.