IDEAS home Printed from https://ideas.repec.org/a/spr/joptap/v180y2019i3d10.1007_s10957-018-1397-z.html
   My bibliography  Save this article

Gradient Surfing: A New Deterministic Approach for Low-Dimensional Global Optimization

Author

Listed:
  • Efrat Taig

    (Ben-Gurion University of the Negev)

  • Ohad Ben-Shahar

    (Ben-Gurion University of the Negev)

Abstract

We describe a novel global optimization technique which utilizes global minima basins of attraction in order to quickly converge to a global minima. A key to the proposed method is the “steeper goes deeper” heuristic: coupling between magnitudes of gradients on different level sets of a basin of attraction and the depth of its minima. Local minima are avoided with a combination of local optimization and a heuristic-based leaping step. Gradient surfing performance is evaluated across a set of small-scale problems from the literature, and results are compared to those of 12 previously published methods. A practical six-dimensional non-convex image registration application is presented as well, where GS performance exceeds that of classic global optimization methods in both speed and accuracy. Additionally, we validate the optimization method by applying a new Gaussian mixture model benchmark for non-convex function. Finally, the “steeper goes deeper” heuristic is validated empirically on five different classes of non-convex functions using two different evaluation approaches. In all cases, steeper gradients are shown to lead to deeper optima with a high probability.

Suggested Citation

  • Efrat Taig & Ohad Ben-Shahar, 2019. "Gradient Surfing: A New Deterministic Approach for Low-Dimensional Global Optimization," Journal of Optimization Theory and Applications, Springer, vol. 180(3), pages 855-878, March.
  • Handle: RePEc:spr:joptap:v:180:y:2019:i:3:d:10.1007_s10957-018-1397-z
    DOI: 10.1007/s10957-018-1397-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10957-018-1397-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10957-018-1397-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. B Suman & P Kumar, 2006. "A survey of simulated annealing as a tool for single and multiobjective optimization," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 57(10), pages 1143-1160, October.
    2. Dimitris Fouskakis & David Draper, 2002. "Stochastic Optimization: a Review," International Statistical Review, International Statistical Institute, vol. 70(3), pages 315-349, December.
    3. Dezhen Xue & Prasanna V. Balachandran & John Hogden & James Theiler & Deqing Xue & Turab Lookman, 2016. "Accelerated search for materials with targeted properties by adaptive design," Nature Communications, Nature, vol. 7(1), pages 1-9, September.
    4. E. L. Lawler & D. E. Wood, 1966. "Branch-and-Bound Methods: A Survey," Operations Research, INFORMS, vol. 14(4), pages 699-719, August.
    5. Goffe, William L. & Ferrier, Gary D. & Rogers, John, 1994. "Global optimization of statistical functions with simulated annealing," Journal of Econometrics, Elsevier, vol. 60(1-2), pages 65-99.
    6. L. Ingber, 1996. "Adaptive simulated annealing (ASA): Lessons learned," Lester Ingber Papers 96as, Lester Ingber.
    7. Zoubin Ghahramani, 2015. "Probabilistic machine learning and artificial intelligence," Nature, Nature, vol. 521(7553), pages 452-459, May.
    8. F. Lampariello & G. Liuzzi, 2015. "A filling function method for unconstrained global optimization," Computational Optimization and Applications, Springer, vol. 61(3), pages 713-729, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Altay, Elif Varol & Alatas, Bilal, 2020. "Randomness as source for inspiring solution search methods: Music based approaches," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 537(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nilsson, Birger & Hansson, Björn, 2004. "A Two-State Capital Asset Pricing Model with Unobservable States," Working Papers 2004:28, Lund University, Department of Economics.
    2. Paolo Postiglione & M. Andreano & Roberto Benedetti, 2013. "Using Constrained Optimization for the Identification of Convergence Clubs," Computational Economics, Springer;Society for Computational Economics, vol. 42(2), pages 151-174, August.
    3. Paolo Postiglione & Maria Simona Andreano & Roberto Benedetti, 2017. "Spatial Clusters in EU Productivity Growth," Growth and Change, Wiley Blackwell, vol. 48(1), pages 40-60, March.
    4. Thomas Baudin & Robert Stelter, 2022. "The rural exodus and the rise of Europe," Journal of Economic Growth, Springer, vol. 27(3), pages 365-414, September.
    5. Masafumi Nakano & Akihiko Takahashi & Soichiro Takahashi, 2018. "State Space Approach to Adaptive Artificial Intelligence Modeling: Application to Financial Portfolio with Fuzzy System," CARF F-Series CARF-F-422, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo.
    6. Luca Benati & Paolo Surico, 2009. "VAR Analysis and the Great Moderation," American Economic Review, American Economic Association, vol. 99(4), pages 1636-1652, September.
    7. Asgharian, Hossein & Hess, Wolfgang & Liu, Lu, 2013. "A spatial analysis of international stock market linkages," Journal of Banking & Finance, Elsevier, vol. 37(12), pages 4738-4754.
    8. Luca Benati & Paolo Surico, 2008. "Evolving U.S. Monetary Policy and The Decline of Inflation Predictability," Journal of the European Economic Association, MIT Press, vol. 6(2-3), pages 634-646, 04-05.
    9. John M. Abowd & Francis Kramarz & Sébastien Pérez-Duarte & Ian M. Schmutte, 2018. "Sorting Between and Within Industries: A Testable Model of Assortative Matching," Annals of Economics and Statistics, GENES, issue 129, pages 1-32.
    10. Asma Khalil Alkhamis & Manar Hosny, 2023. "A Multi-Objective Simulated Annealing Local Search Algorithm in Memetic CENSGA: Application to Vaccination Allocation for Influenza," Sustainability, MDPI, vol. 15(21), pages 1-37, October.
    11. Minford, Patrick & Ou, Zhirong, 2013. "Taylor Rule or optimal timeless policy? Reconsidering the Fed's behavior since 1982," Economic Modelling, Elsevier, vol. 32(C), pages 113-123.
    12. Vo Le & Kent Matthews & David Meenagh & Patrick Minford & Zhiguo Xiao, 2014. "Banking and the Macroeconomy in China: A Banking Crisis Deferred?," Open Economies Review, Springer, vol. 25(1), pages 123-161, February.
    13. Ye, Hong & Lin, Zhiping, 2006. "Speed-up simulated annealing by parallel coordinates," European Journal of Operational Research, Elsevier, vol. 173(1), pages 59-71, August.
    14. Liu, Chunping & Minford, Patrick, 2014. "Comparing behavioural and rational expectations for the US post-war economy," Economic Modelling, Elsevier, vol. 43(C), pages 407-415.
    15. Coşar Gözükırmızı & Metin Demiralp, 2019. "Solving ODEs by Obtaining Purely Second Degree Multinomials via Branch and Bound with Admissible Heuristic," Mathematics, MDPI, vol. 7(4), pages 1-23, April.
    16. Jason Matthew DeBacker, 2015. "Flip‐Flopping: Ideological Adjustment Costs In The United States Senate," Economic Inquiry, Western Economic Association International, vol. 53(1), pages 108-128, January.
    17. Luca Benati & Pierpaolo Benigno, 2023. "Gibson s Paradox and the Natural Rate of Interest," Diskussionsschriften dp2303, Universitaet Bern, Departement Volkswirtschaft.
    18. Haan, Peter & Prowse, Victoria L., 2010. "The Design of Unemployment Transfers: Evidence from a Dynamic Structural Life-Cycle Model," IZA Discussion Papers 4792, Institute of Labor Economics (IZA).
    19. Wilson, Christopher & van der Velden, Maja, 2022. "Sustainable AI: An integrated model to guide public sector decision-making," Technology in Society, Elsevier, vol. 68(C).
    20. Dufour, Jean-Marie, 2006. "Monte Carlo tests with nuisance parameters: A general approach to finite-sample inference and nonstandard asymptotics," Journal of Econometrics, Elsevier, vol. 133(2), pages 443-477, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joptap:v:180:y:2019:i:3:d:10.1007_s10957-018-1397-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.