IDEAS home Printed from https://ideas.repec.org/a/spr/jglopt/v91y2025i1d10.1007_s10898-024-01434-9.html
   My bibliography  Save this article

Global optimization: a machine learning approach

Author

Listed:
  • Dimitris Bertsimas

    (Massachusetts Institute of Technology)

  • Georgios Margaritis

    (Massachusetts Institute of Technology)

Abstract

Many approaches for addressing global optimization problems typically rely on relaxations of nonlinear constraints over specific mathematical primitives. This is restricting in applications with constraints that are implicit or consist of more general primitives. Trying to address such limitations, Bertsimas and Ozturk (2023) proposed OCTHaGOn as a way of solving very general global optimization problems by approximating the nonlinear constraints using hyperplane-based decision-trees and then using those trees to construct a unified MIO approximation of the original problem. We provide extensions to this approach, by (i) approximating the original problem using other MIO-representable ML models besides decision trees, such as gradient boosted trees, multi layer perceptrons and suport vector machines (ii) proposing adaptive sampling procedures for more accurate ML-based constraint approximations, (iii) utilizing robust optimization to account for the uncertainty of the sample-dependent training of the ML models, (iv) leveraging a family of relaxations to address the infeasibilities of the final MIO approximation. We then test the enhanced framework in 81 global optimization instances. We show improvements in solution feasibility and optimality in the majority of instances. We also compare against BARON, showing improved optimality gaps and solution times in more than 9 instances.

Suggested Citation

  • Dimitris Bertsimas & Georgios Margaritis, 2025. "Global optimization: a machine learning approach," Journal of Global Optimization, Springer, vol. 91(1), pages 1-37, January.
  • Handle: RePEc:spr:jglopt:v:91:y:2025:i:1:d:10.1007_s10898-024-01434-9
    DOI: 10.1007/s10898-024-01434-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10898-024-01434-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10898-024-01434-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Arne Stolbjerg Drud, 1994. "CONOPT—A Large-Scale GRG Code," INFORMS Journal on Computing, INFORMS, vol. 6(2), pages 207-216, May.
    2. Robert L. Smith, 1984. "Efficient Monte Carlo Procedures for Generating Points Uniformly Distributed over Bounded Regions," Operations Research, INFORMS, vol. 32(6), pages 1296-1308, December.
    3. Miten Mistry & Dimitrios Letsios & Gerhard Krennrich & Robert M. Lee & Ruth Misener, 2021. "Mixed-Integer Convex Nonlinear Optimization with Gradient-Boosted Trees Embedded," INFORMS Journal on Computing, INFORMS, vol. 33(3), pages 1103-1119, July.
    4. Michael R. Bussieck & Arne Stolbjerg Drud & Alexander Meeraus, 2003. "MINLPLib—A Collection of Test Models for Mixed-Integer Nonlinear Programming," INFORMS Journal on Computing, INFORMS, vol. 15(1), pages 114-119, February.
    5. Gambella, Claudio & Ghaddar, Bissan & Naoum-Sawaya, Joe, 2021. "Optimization problems for machine learning: A survey," European Journal of Operational Research, Elsevier, vol. 290(3), pages 807-828.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ni, Yuanming & Steinshamn, Stein I. & Kvamsdal, Sturla F., 2022. "Negative shocks in an age-structured bioeconomic model and how to deal with them," Economic Analysis and Policy, Elsevier, vol. 76(C), pages 15-30.
    2. Huiyi Cao & Kamil A. Khan, 2023. "General convex relaxations of implicit functions and inverse functions," Journal of Global Optimization, Springer, vol. 86(3), pages 545-572, July.
    3. Duarte, Belmiro P.M. & Sagnol, Guillaume & Wong, Weng Kee, 2018. "An algorithm based on semidefinite programming for finding minimax optimal designs," Computational Statistics & Data Analysis, Elsevier, vol. 119(C), pages 99-117.
    4. Artur M. Schweidtmann & Alexander Mitsos, 2019. "Deterministic Global Optimization with Artificial Neural Networks Embedded," Journal of Optimization Theory and Applications, Springer, vol. 180(3), pages 925-948, March.
    5. Santos, Lucas F. & Costa, Caliane B.B. & Caballero, José A. & Ravagnani, Mauro A.S.S., 2022. "Framework for embedding black-box simulation into mathematical programming via kriging surrogate model applied to natural gas liquefaction process optimization," Applied Energy, Elsevier, vol. 310(C).
    6. Durand-Lasserve, Olivier & Almutairi, Hossa & Aljarboua, Abdullah & Pierru, Axel & Pradhan, Shreekar & Murphy, Frederic, 2023. "Hard-linking a top-down economic model with a bottom-up energy system for an oil-exporting country with price controls," Energy, Elsevier, vol. 266(C).
    7. Emmanuel Ogbe & Xiang Li, 2019. "A joint decomposition method for global optimization of multiscenario nonconvex mixed-integer nonlinear programs," Journal of Global Optimization, Springer, vol. 75(3), pages 595-629, November.
    8. Marian Leimbach & Anselm Schultes & Lavinia Baumstark & Anastasis Giannousakis & Gunnar Luderer, 2017. "Solution algorithms for regional interactions in large-scale integrated assessment models of climate change," Annals of Operations Research, Springer, vol. 255(1), pages 29-45, August.
    9. Xu, Jianwei & Liang, Yingzong & Luo, Xianglong & Chen, Jianyong & Yang, Zhi & Chen, Ying, 2023. "Techno-economic-environmental analysis of direct-contact membrane distillation systems integrated with low-grade heat sources: A multi-objective optimization approach," Applied Energy, Elsevier, vol. 349(C).
    10. Victor Reyes & Ignacio Araya, 2021. "AbsTaylor: upper bounding with inner regions in nonlinear continuous global optimization problems," Journal of Global Optimization, Springer, vol. 79(2), pages 413-429, February.
    11. Fuentes-Cortés, Luis Fabián & Flores-Tlacuahuac, Antonio, 2018. "Integration of distributed generation technologies on sustainable buildings," Applied Energy, Elsevier, vol. 224(C), pages 582-601.
    12. Cignitti, Stefano & Andreasen, Jesper G. & Haglind, Fredrik & Woodley, John M. & Abildskov, Jens, 2017. "Integrated working fluid-thermodynamic cycle design of organic Rankine cycle power systems for waste heat recovery," Applied Energy, Elsevier, vol. 203(C), pages 442-453.
    13. Jianhui Xie & Qiwei Xie & Yongjun Li & Liang Liang, 2021. "Solving data envelopment analysis models with sum-of-fractional objectives: a global optimal approach based on the multiparametric disaggregation technique," Annals of Operations Research, Springer, vol. 304(1), pages 453-480, September.
    14. Gao, Lei & Hwang, Yunho & Cao, Tao, 2019. "An overview of optimization technologies applied in combined cooling, heating and power systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 114(C), pages 1-1.
    15. Ibrić, Nidret & Ahmetović, Elvis & Kravanja, Zdravko & Maréchal, François & Kermani, Maziar, 2017. "Simultaneous synthesis of non-isothermal water networks integrated with process streams," Energy, Elsevier, vol. 141(C), pages 2587-2612.
    16. Chernonog, Tatyana & Goldberg, Noam, 2018. "On the multi-product newsvendor with bounded demand distributions," International Journal of Production Economics, Elsevier, vol. 203(C), pages 38-47.
    17. Klaus Mittenzwei, 2020. "Greenhouse Gas Emissions in Norwegian Agriculture: The Regional and Structural Dimension," Sustainability, MDPI, vol. 12(6), pages 1-13, March.
    18. Rutherford, Thomas F. & Böhringer, Christoph, 2021. "Will COVID-19 change the calculus of climate policy?," VfS Annual Conference 2021 (Virtual Conference): Climate Economics 242432, Verein für Socialpolitik / German Economic Association.
    19. Dominik Bongartz & Alexander Mitsos, 2017. "Deterministic global optimization of process flowsheets in a reduced space using McCormick relaxations," Journal of Global Optimization, Springer, vol. 69(4), pages 761-796, December.
    20. Yang, Yudi & Fan, Yueyue & Wets, Roger J.B., 2018. "Stochastic travel demand estimation: Improving network identifiability using multi-day observation sets," Transportation Research Part B: Methodological, Elsevier, vol. 107(C), pages 192-211.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jglopt:v:91:y:2025:i:1:d:10.1007_s10898-024-01434-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.