IDEAS home Printed from https://ideas.repec.org/a/spr/joptap/v180y2019i3d10.1007_s10957-018-1397-z.html
   My bibliography  Save this article

Gradient Surfing: A New Deterministic Approach for Low-Dimensional Global Optimization

Author

Listed:
  • Efrat Taig

    (Ben-Gurion University of the Negev)

  • Ohad Ben-Shahar

    (Ben-Gurion University of the Negev)

Abstract

We describe a novel global optimization technique which utilizes global minima basins of attraction in order to quickly converge to a global minima. A key to the proposed method is the “steeper goes deeper” heuristic: coupling between magnitudes of gradients on different level sets of a basin of attraction and the depth of its minima. Local minima are avoided with a combination of local optimization and a heuristic-based leaping step. Gradient surfing performance is evaluated across a set of small-scale problems from the literature, and results are compared to those of 12 previously published methods. A practical six-dimensional non-convex image registration application is presented as well, where GS performance exceeds that of classic global optimization methods in both speed and accuracy. Additionally, we validate the optimization method by applying a new Gaussian mixture model benchmark for non-convex function. Finally, the “steeper goes deeper” heuristic is validated empirically on five different classes of non-convex functions using two different evaluation approaches. In all cases, steeper gradients are shown to lead to deeper optima with a high probability.

Suggested Citation

  • Efrat Taig & Ohad Ben-Shahar, 2019. "Gradient Surfing: A New Deterministic Approach for Low-Dimensional Global Optimization," Journal of Optimization Theory and Applications, Springer, vol. 180(3), pages 855-878, March.
  • Handle: RePEc:spr:joptap:v:180:y:2019:i:3:d:10.1007_s10957-018-1397-z
    DOI: 10.1007/s10957-018-1397-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10957-018-1397-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10957-018-1397-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. B Suman & P Kumar, 2006. "A survey of simulated annealing as a tool for single and multiobjective optimization," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 57(10), pages 1143-1160, October.
    2. Dimitris Fouskakis & David Draper, 2002. "Stochastic Optimization: a Review," International Statistical Review, International Statistical Institute, vol. 70(3), pages 315-349, December.
    3. Dezhen Xue & Prasanna V. Balachandran & John Hogden & James Theiler & Deqing Xue & Turab Lookman, 2016. "Accelerated search for materials with targeted properties by adaptive design," Nature Communications, Nature, vol. 7(1), pages 1-9, September.
    4. E. L. Lawler & D. E. Wood, 1966. "Branch-and-Bound Methods: A Survey," Operations Research, INFORMS, vol. 14(4), pages 699-719, August.
    5. Goffe, William L. & Ferrier, Gary D. & Rogers, John, 1994. "Global optimization of statistical functions with simulated annealing," Journal of Econometrics, Elsevier, vol. 60(1-2), pages 65-99.
    6. L. Ingber, 1996. "Adaptive simulated annealing (ASA): Lessons learned," Lester Ingber Papers 96as, Lester Ingber.
    7. Zoubin Ghahramani, 2015. "Probabilistic machine learning and artificial intelligence," Nature, Nature, vol. 521(7553), pages 452-459, May.
    8. F. Lampariello & G. Liuzzi, 2015. "A filling function method for unconstrained global optimization," Computational Optimization and Applications, Springer, vol. 61(3), pages 713-729, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Altay, Elif Varol & Alatas, Bilal, 2020. "Randomness as source for inspiring solution search methods: Music based approaches," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 537(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Paolo Postiglione & Maria Simona Andreano & Roberto Benedetti, 2017. "Spatial Clusters in EU Productivity Growth," Growth and Change, Wiley Blackwell, vol. 48(1), pages 40-60, March.
    2. Nilsson, Birger & Hansson, Björn, 2004. "A Two-State Capital Asset Pricing Model with Unobservable States," Working Papers 2004:28, Lund University, Department of Economics.
    3. Paolo Postiglione & M. Andreano & Roberto Benedetti, 2013. "Using Constrained Optimization for the Identification of Convergence Clubs," Computational Economics, Springer;Society for Computational Economics, vol. 42(2), pages 151-174, August.
    4. Thomas Baudin & Robert Stelter, 2022. "The rural exodus and the rise of Europe," Journal of Economic Growth, Springer, vol. 27(3), pages 365-414, September.
    5. Masafumi Nakano & Akihiko Takahashi & Soichiro Takahashi, 2018. "State Space Approach to Adaptive Artificial Intelligence Modeling: Application to Financial Portfolio with Fuzzy System," CARF F-Series CARF-F-422, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo.
    6. Luca Benati & Paolo Surico, 2009. "VAR Analysis and the Great Moderation," American Economic Review, American Economic Association, vol. 99(4), pages 1636-1652, September.
    7. John M. Abowd & Francis Kramarz & Sébastien Pérez-Duarte & Ian M. Schmutte, 2018. "Sorting Between and Within Industries: A Testable Model of Assortative Matching," Annals of Economics and Statistics, GENES, issue 129, pages 1-32.
    8. Asma Khalil Alkhamis & Manar Hosny, 2023. "A Multi-Objective Simulated Annealing Local Search Algorithm in Memetic CENSGA: Application to Vaccination Allocation for Influenza," Sustainability, MDPI, vol. 15(21), pages 1-37, October.
    9. Jason Matthew DeBacker, 2015. "Flip‐Flopping: Ideological Adjustment Costs In The United States Senate," Economic Inquiry, Western Economic Association International, vol. 53(1), pages 108-128, January.
    10. Haan, Peter & Prowse, Victoria L., 2010. "The Design of Unemployment Transfers: Evidence from a Dynamic Structural Life-Cycle Model," IZA Discussion Papers 4792, Institute of Labor Economics (IZA).
    11. Green, Rikard & Larsson, Karl & Lunina, Veronika & Nilsson, Birger, 2018. "Cross-commodity news transmission and volatility spillovers in the German energy markets," Journal of Banking & Finance, Elsevier, vol. 95(C), pages 231-243.
    12. Kapetanios, George & Marcellino, Massimiliano & Papailias, Fotis, 2016. "Forecasting inflation and GDP growth using heuristic optimisation of information criteria and variable reduction methods," Computational Statistics & Data Analysis, Elsevier, vol. 100(C), pages 369-382.
    13. Roman Sustek, 2011. "Monetary Business Cycle Accounting," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 14(4), pages 592-612, October.
    14. Li, Yi & Liu, Kailong & Foley, Aoife M. & Zülke, Alana & Berecibar, Maitane & Nanini-Maury, Elise & Van Mierlo, Joeri & Hoster, Harry E., 2019. "Data-driven health estimation and lifetime prediction of lithium-ion batteries: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 113(C), pages 1-1.
    15. Kezong Tang & Xiong-Fei Wei & Yuan-Hao Jiang & Zi-Wei Chen & Lihua Yang, 2023. "An Adaptive Ant Colony Optimization for Solving Large-Scale Traveling Salesman Problem," Mathematics, MDPI, vol. 11(21), pages 1-26, October.
    16. Deb, Partha & Trivedi, Pravin K., 2002. "The structure of demand for health care: latent class versus two-part models," Journal of Health Economics, Elsevier, vol. 21(4), pages 601-625, July.
    17. Catherine Kyrtsou & Michel Terraza, 2003. "Is it Possible to Study Chaotic and ARCH Behaviour Jointly? Application of a Noisy Mackey–Glass Equation with Heteroskedastic Errors to the Paris Stock Exchange Returns Series," Computational Economics, Springer;Society for Computational Economics, vol. 21(3), pages 257-276, June.
    18. Felipe, Ángel & Ortuño, M. Teresa & Righini, Giovanni & Tirado, Gregorio, 2014. "A heuristic approach for the green vehicle routing problem with multiple technologies and partial recharges," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 71(C), pages 111-128.
    19. Pudney, Stephen, 2011. "Perception and retrospection: The dynamic consistency of responses to survey questions on wellbeing," Journal of Public Economics, Elsevier, vol. 95(3), pages 300-310.
    20. Szafranek, Karol, 2019. "Bagged neural networks for forecasting Polish (low) inflation," International Journal of Forecasting, Elsevier, vol. 35(3), pages 1042-1059.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joptap:v:180:y:2019:i:3:d:10.1007_s10957-018-1397-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.