Application of Monte Carlo stochastic optimization (MOST) to deep learning

In this paper, we propose a new optimization method based on the Monte Carlo method. The proposed method is applied to several benchmark problems, and the result of applying it to the optimization of neural network is reported. Deep machine learning using neural networks is one of the important keywords to promote innovation in today’s advanced information society. Therefore, research on large-scale, high-speed, and high-precision algorithms has been actively conducted. The author has developed an optimization method in which the search region for multivariate parameters constituting the objective function is divided into two regions for each parameter, the integral values of each divided region are numerically calculated by the Monte Carlo method, the magnitude of each integral value is compared, and the optimum point is judged to be in a small region. The proposed method was applied to 50 variable benchmark functions (Schwefel and Ackley Functions), and was compared with the results of genetic algorithm (GA) which was a representative of existing optimization methods. As a result, it was confirmed that the proposed method is faster and more accurate than GA. In addition, the proposed method is applied to machine learning by neural networks, specifically XOR gate circuits and IRS classification problems, and verified. The neural network optimized by MOST reproduced teacher data and test data faster and more accurately than conventional Adam and genetic algorithms.