IDEAS home Printed from https://ideas.repec.org/a/inm/orijoc/v30y2018i3p608-624.html
   My bibliography  Save this article

What Works Best When? A Systematic Evaluation of Heuristics for Max-Cut and QUBO

Author

Listed:
  • Iain Dunning

    (DeepMind, London N1C 4AG, United Kingdom)

  • Swati Gupta

    (Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332)

  • John Silberholz

    (Ross School of Business, University of Michigan, Ann Arbor, Michigan 48109)

Abstract

Though empirical testing is broadly used to evaluate heuristics, there are shortcomings with how it is often applied in practice. In a systematic review of Max-Cut and quadratic unconstrained binary optimization (QUBO) heuristics papers, we found only 4% publish source code, only 14% compare heuristics with identical termination criteria, and most experiments are performed with an artificial, homogeneous set of problem instances. To address these limitations, we implement and release as open-source a code-base of 10 Max-Cut and 27 QUBO heuristics. We perform heuristic evaluation using cloud computing on a library of 3,296 instances. This large-scale evaluation provides insight into the types of problem instances for which each heuristic performs well or poorly. Because no single heuristic outperforms all others across all problem instances, we use machine learning to predict which heuristic will work best on a previously unseen problem instance, a key question facing practitioners.

Suggested Citation

  • Iain Dunning & Swati Gupta & John Silberholz, 2018. "What Works Best When? A Systematic Evaluation of Heuristics for Max-Cut and QUBO," INFORMS Journal on Computing, INFORMS, vol. 30(3), pages 608-624, August.
    • Handle: RePEc:inm:orijoc:v:30:y:2018:i:3:p:608-624
    DOI: 10.1287/ijoc.2017.0798
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/ijoc.2017.0798
    Download Restriction: no
    File URL: https://libkey.io/10.1287/ijoc.2017.0798?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. John Silberholz & Bruce Golden, 2010. "Comparison of Metaheuristics," International Series in Operations Research & Management Science, in: Michel Gendreau & Jean-Yves Potvin (ed.), Handbook of Metaheuristics, chapter 0, pages 625-640, Springer.
    2. Thomas E. Vollmann & Elwood S. Buffa, 1966. "The Facilities Layout Problem in Perspective," Management Science, INFORMS, vol. 12(10), pages 450-468, June.
    3. Rafael Martí & Abraham Duarte & Manuel Laguna, 2009. "Advanced Scatter Search for the Max-Cut Problem," INFORMS Journal on Computing, INFORMS, vol. 21(1), pages 26-38, February.
    4. Raymond R. Hill & Charles H. Reilly, 2000. "The Effects of Coefficient Correlation Structure in Two-Dimensional Knapsack Problems on Solution Procedure Performance," Management Science, INFORMS, vol. 46(2), pages 302-317, February.
    5. Edmund K. Burke & Matthew Hyde & Graham Kendall & Gabriela Ochoa & Ender Özcan & John R. Woodward, 2010. "A Classification of Hyper-heuristic Approaches," International Series in Operations Research & Management Science, in: Michel Gendreau & Jean-Yves Potvin (ed.), Handbook of Metaheuristics, chapter 0, pages 449-468, Springer.
    6. Francisco Barahona & Martin Grötschel & Michael Jünger & Gerhard Reinelt, 1988. "An Application of Combinatorial Optimization to Statistical Physics and Circuit Layout Design," Operations Research, INFORMS, vol. 36(3), pages 493-513, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Fred Glover & Gary Kochenberger & Rick Hennig & Yu Du, 2022. "Quantum bridge analytics I: a tutorial on formulating and using QUBO models," Annals of Operations Research, Springer, vol. 314(1), pages 141-183, July.
    2. Ricardo N. Liang & Eduardo A. J. Anacleto & Cláudio N. Meneses, 2022. "Data structures for speeding up Tabu Search when solving sparse quadratic unconstrained binary optimization problems," Journal of Heuristics, Springer, vol. 28(4), pages 433-479, August.
    3. Cheng Lu & Zhibin Deng & Shu-Cherng Fang & Wenxun Xing, 2023. "A New Global Algorithm for Max-Cut Problem with Chordal Sparsity," Journal of Optimization Theory and Applications, Springer, vol. 197(2), pages 608-638, May.
    4. Byron Tasseff & Tameem Albash & Zachary Morrell & Marc Vuffray & Andrey Y. Lokhov & Sidhant Misra & Carleton Coffrin, 2024. "On the emerging potential of quantum annealing hardware for combinatorial optimization," Journal of Heuristics, Springer, vol. 30(5), pages 325-358, December.
    5. Timotej Hrga & Janez Povh, 2021. "MADAM: a parallel exact solver for max-cut based on semidefinite programming and ADMM," Computational Optimization and Applications, Springer, vol. 80(2), pages 347-375, November.
    6. Theja Tulabandhula & Deeksha Sinha & Saketh Reddy Karra & Prasoon Patidar, 2020. "Multi-Purchase Behavior: Modeling, Estimation and Optimization," Papers 2006.08055, arXiv.org, revised Aug 2023.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wenxing Zhu & Geng Lin & M. M. Ali, 2013. "Max- k -Cut by the Discrete Dynamic Convexized Method," INFORMS Journal on Computing, INFORMS, vol. 25(1), pages 27-40, February.
    2. Fuda Ma & Jin-Kao Hao, 2017. "A multiple search operator heuristic for the max-k-cut problem," Annals of Operations Research, Springer, vol. 248(1), pages 365-403, January.
    3. Marko Ɖurasević & Domagoj Jakobović, 2019. "Creating dispatching rules by simple ensemble combination," Journal of Heuristics, Springer, vol. 25(6), pages 959-1013, December.
    4. Gahm, Christian & Uzunoglu, Aykut & Wahl, Stefan & Ganschinietz, Chantal & Tuma, Axel, 2022. "Applying machine learning for the anticipation of complex nesting solutions in hierarchical production planning," European Journal of Operational Research, Elsevier, vol. 296(3), pages 819-836.
    5. Dell'Amico, Mauro & Trubian, Marco, 1998. "Solution of large weighted equicut problems," European Journal of Operational Research, Elsevier, vol. 106(2-3), pages 500-521, April.
    6. Bolte, Andreas & Thonemann, Ulrich Wilhelm, 1996. "Optimizing simulated annealing schedules with genetic programming," European Journal of Operational Research, Elsevier, vol. 92(2), pages 402-416, July.
    7. Cheng Lu & Zhibin Deng, 2021. "A branch-and-bound algorithm for solving max-k-cut problem," Journal of Global Optimization, Springer, vol. 81(2), pages 367-389, October.
    8. Jorge A. Sefair & Oscar Guaje & Andrés L. Medaglia, 2021. "A column-oriented optimization approach for the generation of correlated random vectors," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 43(3), pages 777-808, September.
    9. Shenshen Gu & Yue Yang, 2020. "A Deep Learning Algorithm for the Max-Cut Problem Based on Pointer Network Structure with Supervised Learning and Reinforcement Learning Strategies," Mathematics, MDPI, vol. 8(2), pages 1-20, February.
    10. Drexl, Michael & Schneider, Michael, 2015. "A survey of variants and extensions of the location-routing problem," European Journal of Operational Research, Elsevier, vol. 241(2), pages 283-308.
    11. Aleksandra Swiercz & Wojciech Frohmberg & Michal Kierzynka & Pawel Wojciechowski & Piotr Zurkowski & Jan Badura & Artur Laskowski & Marta Kasprzak & Jacek Blazewicz, 2018. "GRASShopPER—An algorithm for de novo assembly based on GPU alignments," PLOS ONE, Public Library of Science, vol. 13(8), pages 1-23, August.
    12. repec:dgr:rugsom:99a17 is not listed on IDEAS
    13. Lu, Yongliang & Benlic, Una & Wu, Qinghua, 2018. "A memetic algorithm for the Orienteering Problem with Mandatory Visits and Exclusionary Constraints," European Journal of Operational Research, Elsevier, vol. 268(1), pages 54-69.
    14. Leo Lopes & Kate Smith-Miles, 2013. "Generating Applicable Synthetic Instances for Branch Problems," Operations Research, INFORMS, vol. 61(3), pages 563-577, June.
    15. A. S. Santos & A. M. Madureira & M. L. R. Varela, 2018. "The Influence of Problem Specific Neighborhood Structures in Metaheuristics Performance," Journal of Mathematics, Hindawi, vol. 2018, pages 1-14, July.
    16. Gili Rosenberg & Mohammad Vazifeh & Brad Woods & Eldad Haber, 2016. "Building an iterative heuristic solver for a quantum annealer," Computational Optimization and Applications, Springer, vol. 65(3), pages 845-869, December.
    17. Pierre Fouilhoux & A. Mahjoub, 2012. "Solving VLSI design and DNA sequencing problems using bipartization of graphs," Computational Optimization and Applications, Springer, vol. 51(2), pages 749-781, March.
    18. Goldengorin, Boris & Ghosh, Diptesh, 2004. "A Multilevel Search Algorithm for the Maximization of Submodular Functions," Research Report 04A20, University of Groningen, Research Institute SOM (Systems, Organisations and Management).
    19. Hongwei Liu & Sanyang Liu & Fengmin Xu, 2003. "A Tight Semidefinite Relaxation of the MAX CUT Problem," Journal of Combinatorial Optimization, Springer, vol. 7(3), pages 237-245, September.
    20. Vidal, Thibaut & Crainic, Teodor Gabriel & Gendreau, Michel & Prins, Christian, 2013. "Heuristics for multi-attribute vehicle routing problems: A survey and synthesis," European Journal of Operational Research, Elsevier, vol. 231(1), pages 1-21.
    21. Jerzy Grobelny & Rafał Michalski, 2020. "Effects of scatter plot initial solutions on regular grid facility layout algorithms in typical production models," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 28(2), pages 601-632, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijoc:v:30:y:2018:i:3:p:608-624. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.