IDEAS home Printed from https://ideas.repec.org/a/inm/orijoc/v34y2022i6p3007-3022.html
   My bibliography  Save this article

Bifidelity Surrogate Modelling: Showcasing the Need for New Test Instances

Author

Listed:
  • Nicolau Andrés-Thió

    (School of Mathematics and Statistics, The University of Melbourne, Parkville, Victoria 3010 Australia)

  • Mario Andrés Muñoz

    (School of Mathematics and Statistics, The University of Melbourne, Parkville, Victoria 3010 Australia)

  • Kate Smith-Miles

    (School of Mathematics and Statistics, The University of Melbourne, Parkville, Victoria 3010 Australia)

Abstract

In recent years, multifidelity expensive black-box (Mf-EBB) methods have received increasing attention due to their strong applicability to industrial design problems. The challenge, however, is that knowledge of the relationship between decisions and objective values is limited to a small set of sample observations of variable quality. In the field of Mf-EBB, a problem instance consists of an expensive yet accurate source of information, and one or more cheap yet less accurate sources of information. The field aims to provide techniques either to accurately explain how decisions affect design outcome, or to find the best decisions to optimise design outcomes. Many techniques that use surrogate models have been developed to provide solutions to both aims. Only in recent years, however, have researchers begun to explore the conditions under which these new techniques are reliable, often focusing on problems with a single low-fidelity function, known as bifidelity expensive black-box (Bf-EBB) problems. This study extends the existing Bf-EBB test instances found in the literature, as well as the features used to determine when the low-fidelity information source should be used. A literature test suite is constructed and augmented with new instances to demonstrate the potentially misleading results that could be reached using only the instances currently found in the literature, and to expose the criticality of a more heterogeneous test suite for algorithm assessment. Addressing the shortcomings of the existing literature, a new set of features is presented, as well as a new instance creation procedure, and a study of their impact on algorithm assessment is conducted. The low-fidelity information source is shown to be valuable if it is often locally accurate, even when its overall accuracy is relatively low. This contradicts the existing literature guidelines, which indicate the low-fidelity information is only useful if it has a high overall accuracy.

Suggested Citation

  • Nicolau Andrés-Thió & Mario Andrés Muñoz & Kate Smith-Miles, 2022. "Bifidelity Surrogate Modelling: Showcasing the Need for New Test Instances," INFORMS Journal on Computing, INFORMS, vol. 34(6), pages 3007-3022, November.
    • Handle: RePEc:inm:orijoc:v:34:y:2022:i:6:p:3007-3022
    DOI: 10.1287/ijoc.2022.1217
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/ijoc.2022.1217
    Download Restriction: no
    File URL: https://libkey.io/10.1287/ijoc.2022.1217?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Juliane Müller & Christine Shoemaker, 2014. "Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems," Journal of Global Optimization, Springer, vol. 60(2), pages 123-144, October.
    2. Dionne M. Aleman & H. Edwin Romeijn & James F. Dempsey, 2009. "A Response Surface Approach to Beam Orientation Optimization in Intensity-Modulated Radiation Therapy Treatment Planning," INFORMS Journal on Computing, INFORMS, vol. 21(1), pages 62-76, February.
    3. Rommel G. Regis & Christine A. Shoemaker, 2007. "A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions," INFORMS Journal on Computing, INFORMS, vol. 19(4), pages 497-509, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Juliane Müller, 2017. "SOCEMO: Surrogate Optimization of Computationally Expensive Multiobjective Problems," INFORMS Journal on Computing, INFORMS, vol. 29(4), pages 581-596, November.
    2. Juliane Müller & Joshua D. Woodbury, 2017. "GOSAC: global optimization with surrogate approximation of constraints," Journal of Global Optimization, Springer, vol. 69(1), pages 117-136, September.
    3. Rommel Regis & Christine Shoemaker, 2013. "A quasi-multistart framework for global optimization of expensive functions using response surface models," Journal of Global Optimization, Springer, vol. 56(4), pages 1719-1753, August.
    4. Tipaluck Krityakierne & Taimoor Akhtar & Christine A. Shoemaker, 2016. "SOP: parallel surrogate global optimization with Pareto center selection for computationally expensive single objective problems," Journal of Global Optimization, Springer, vol. 66(3), pages 417-437, November.
    5. Fani Boukouvala & M. M. Faruque Hasan & Christodoulos A. Floudas, 2017. "Global optimization of general constrained grey-box models: new method and its application to constrained PDEs for pressure swing adsorption," Journal of Global Optimization, Springer, vol. 67(1), pages 3-42, January.
    6. Juliane Müller & Marcus Day, 2019. "Surrogate Optimization of Computationally Expensive Black-Box Problems with Hidden Constraints," INFORMS Journal on Computing, INFORMS, vol. 31(4), pages 689-702, October.
    7. Boukouvala, Fani & Misener, Ruth & Floudas, Christodoulos A., 2016. "Global optimization advances in Mixed-Integer Nonlinear Programming, MINLP, and Constrained Derivative-Free Optimization, CDFO," European Journal of Operational Research, Elsevier, vol. 252(3), pages 701-727.
    8. Lim, Gino J. & Bard, Jonathan F., 2016. "Benders decomposition and an IP-based heuristic for selecting IMRT treatment beam anglesAuthor-Name: Lin, Sifeng," European Journal of Operational Research, Elsevier, vol. 251(3), pages 715-726.
    9. Songhao Wang & Szu Hui Ng & William Benjamin Haskell, 2022. "A Multilevel Simulation Optimization Approach for Quantile Functions," INFORMS Journal on Computing, INFORMS, vol. 34(1), pages 569-585, January.
    10. Juliane Müller & Christine Shoemaker & Robert Piché, 2014. "SO-I: a surrogate model algorithm for expensive nonlinear integer programming problems including global optimization applications," Journal of Global Optimization, Springer, vol. 59(4), pages 865-889, August.
    11. Mehdad, E. & Kleijnen, Jack P.C., 2014. "Global Optimization for Black-box Simulation via Sequential Intrinsic Kriging," Other publications TiSEM 8fa8d96f-a086-4c4b-88ab-9, Tilburg University, School of Economics and Management.
    12. Misic, V.V. & Aleman, D.M. & Sharpe, M.B., 2010. "Neighborhood search approaches to non-coplanar beam orientation optimization for total marrow irradiation using IMRT," European Journal of Operational Research, Elsevier, vol. 205(3), pages 522-527, September.
    13. Zhe Zhou & Fusheng Bai, 2018. "An adaptive framework for costly black-box global optimization based on radial basis function interpolation," Journal of Global Optimization, Springer, vol. 70(4), pages 757-781, April.
    14. Liu, Haoxiang & Wang, David Z.W., 2017. "Locating multiple types of charging facilities for battery electric vehicles," Transportation Research Part B: Methodological, Elsevier, vol. 103(C), pages 30-55.
    15. Taimoor Akhtar & Christine Shoemaker, 2016. "Multi objective optimization of computationally expensive multi-modal functions with RBF surrogates and multi-rule selection," Journal of Global Optimization, Springer, vol. 64(1), pages 17-32, January.
    16. Liu, Haoxiang & Szeto, W.Y. & Long, Jiancheng, 2019. "Bike network design problem with a path-size logit-based equilibrium constraint: Formulation, global optimization, and matheuristic," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 127(C), pages 284-307.
    17. Gino Lim & Laleh Kardar & Wenhua Cao, 2014. "A hybrid framework for optimizing beam angles in radiation therapy planning," Annals of Operations Research, Springer, vol. 217(1), pages 357-383, June.
    18. Marc C. Robini & Feng Yang & Yuemin Zhu, 2020. "A stochastic approach to full inverse treatment planning for charged-particle therapy," Journal of Global Optimization, Springer, vol. 77(4), pages 853-893, August.
    19. Krityakierne, Tipaluck & Baowan, Duangkamon, 2020. "Aggregated GP-based Optimization for Contaminant Source Localization," Operations Research Perspectives, Elsevier, vol. 7(C).
    20. Miriyala, Srinivas Soumitri & Subramanian, Venkat & Mitra, Kishalay, 2018. "TRANSFORM-ANN for online optimization of complex industrial processes: Casting process as case study," European Journal of Operational Research, Elsevier, vol. 264(1), pages 294-309.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijoc:v:34:y:2022:i:6:p:3007-3022. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.