Chaos Theory, Advanced Metaheuristic Algorithms And Their Newfangled Deep Learning Architecture Optimization Applications: A Review

Metaheuristic techniques are capable of representing optimization frames with their specific theories as well as objective functions owing to their being adjustable and effective in various applications. Through the optimization of deep learning models, metaheuristic algorithms inspired by nature, imitating the behavior of living and non-living beings, have been used for about four decades to solve challenging, complex, and chaotic problems. These algorithms can be categorized as evolution-based, swarm-based, nature-based, human-based, hybrid, or chaos-based. Chaos theory, as a useful approach to understanding neural network optimization, has the basic idea of viewing the neural network optimization as a dynamical system in which the equation schemes are utilized from the space pertaining to learnable parameters, namely optimization trajectory, to itself, which enables the description of the evolution of the system by understanding the training behavior, which is to say the number of iterations over time. The examination of the recent studies reveals the importance of chaos theory, which is sensitive to initial conditions with randomness and dynamical properties that are principally emerging on the complex multimodal landscape. Chaotic optimization, in this regard, accelerates the speed of the algorithm while also enhancing the variety of movement patterns. The significance of hybrid algorithms developed through their applications in different domains concerning real-world phenomena and well-known benchmark problems in the literature is also evident. Metaheuristic optimization algorithms have also been applied to deep learning or deep neural networks (DNNs), a branch of machine learning. In this respect, the basic features of deep learning and DNNs and the extensive use of metaheuristic algorithms are overviewed and explained. Accordingly, the current review aims at providing new insights into the studies that deal with metaheuristic algorithms, hybrid-based metaheuristics, chaos-based metaheuristics as well as deep learning besides presenting recent information on the development of the essence of this branch of science with emerging opportunities, applicability-based optimization aspects and generation of well-informed decisions.