IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v338y2024i2d10.1007_s10479-024-05941-6.html
   My bibliography  Save this article

An experimental design for comparing interactive methods based on their desirable properties

Author

Listed:
  • Bekir Afsar

    (University of Jyvaskyla, Faculty of Information Technology)

  • Johanna Silvennoinen

    (University of Jyvaskyla, Faculty of Information Technology)

  • Francisco Ruiz

    (Universidad de Málaga)

  • Ana B. Ruiz

    (Universidad de Málaga)

  • Giovanni Misitano

    (University of Jyvaskyla, Faculty of Information Technology)

  • Kaisa Miettinen

    (University of Jyvaskyla, Faculty of Information Technology)

Abstract

In multiobjective optimization problems, Pareto optimal solutions representing different tradeoffs cannot be ordered without incorporating preference information of a decision maker (DM). In interactive methods, the DM takes an active part in the solution process and provides preference information iteratively. Between iterations, the DM can learn how achievable the preferences are, learn about the tradeoffs, and adjust the preferences. Different interactive methods have been proposed in the literature, but the question of how to select the best-suited method for a problem to be solved remains partly open. We propose an experimental design for evaluating interactive methods according to several desirable properties related to the cognitive load experienced by the DM, the method’s ability to capture preferences and its responsiveness to changes in the preferences, the DM’s satisfaction in the overall solution process, and their confidence in the final solution. In the questionnaire designed, we connect each questionnaire item to be asked with a relevant research question characterizing these desirable properties of interactive methods. We also conduct a between-subjects experiment to compare three interactive methods and report interesting findings. In particular, we find out that trade-off-free methods may be more suitable for exploring the whole set of Pareto optimal solutions, while classification-based methods seem to work better for fine-tuning the preferences to find the final solution.

Suggested Citation

  • Bekir Afsar & Johanna Silvennoinen & Francisco Ruiz & Ana B. Ruiz & Giovanni Misitano & Kaisa Miettinen, 2024. "An experimental design for comparing interactive methods based on their desirable properties," Annals of Operations Research, Springer, vol. 338(2), pages 835-856, July.
  • Handle: RePEc:spr:annopr:v:338:y:2024:i:2:d:10.1007_s10479-024-05941-6
    DOI: 10.1007/s10479-024-05941-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10479-024-05941-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10479-024-05941-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Buchanan, J. T. & Daellenbach, H. G., 1987. "A comparative evaluation of interactive solution methods for multiple objective decision models," European Journal of Operational Research, Elsevier, vol. 29(3), pages 353-359, June.
    2. Miettinen, Kaisa & Makela, Marko M., 2006. "Synchronous approach in interactive multiobjective optimization," European Journal of Operational Research, Elsevier, vol. 170(3), pages 909-922, May.
    3. Jyrki Wallenius, 1975. "Comparative Evaluation of Some Interactive Approaches to Multicriterion Optimization," Management Science, INFORMS, vol. 21(12), pages 1387-1396, August.
    4. Kaisa Miettinen & Francisco Ruiz, 2016. "NAUTILUS framework: towards trade-off-free interaction in multiobjective optimization," Journal of Business Economics, Springer, vol. 86(1), pages 5-21, January.
    5. Bekir Afsar & Johanna Silvennoinen & Giovanni Misitano & Francisco Ruiz & Ana B. Ruiz & Kaisa Miettinen, 2023. "Designing empirical experiments to compare interactive multiobjective optimization methods," Journal of the Operational Research Society, Taylor & Francis Journals, vol. 74(11), pages 2327-2338, November.
    6. Kaisa Miettinen & Jussi Hakanen & Dmitry Podkopaev, 2016. "Interactive Nonlinear Multiobjective Optimization Methods," International Series in Operations Research & Management Science, in: Salvatore Greco & Matthias Ehrgott & José Rui Figueira (ed.), Multiple Criteria Decision Analysis, edition 2, chapter 0, pages 927-976, Springer.
    7. Francisco Ruiz & Mariano Luque & Kaisa Miettinen, 2012. "Improving the computational efficiency in a global formulation (GLIDE) for interactive multiobjective optimization," Annals of Operations Research, Springer, vol. 197(1), pages 47-70, August.
    8. Brockhoff, Klaus, 1985. "Experimental test of MCDM algorithms in a modular approach," European Journal of Operational Research, Elsevier, vol. 22(2), pages 159-166, November.
    9. Kok, Matthijs, 1986. "The interface with decision makers and some experimental results in interactive multiple objective programming methods," European Journal of Operational Research, Elsevier, vol. 26(1), pages 96-107, July.
    10. Risto Heikkinen & Juha Sipilä & Vesa Ojalehto & Kaisa Miettinen, 2023. "Flexible data driven inventory management with interactive multi-objective lot size optimisation," International Journal of Logistics Systems and Management, Inderscience Enterprises Ltd, vol. 46(2), pages 206-235.
    11. Ruiz, Ana B. & Sindhya, Karthik & Miettinen, Kaisa & Ruiz, Francisco & Luque, Mariano, 2015. "E-NAUTILUS: A decision support system for complex multiobjective optimization problems based on the NAUTILUS method," European Journal of Operational Research, Elsevier, vol. 246(1), pages 218-231.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aloysius, John A. & Davis, Fred D. & Wilson, Darryl D. & Ross Taylor, A. & Kottemann, Jeffrey E., 2006. "User acceptance of multi-criteria decision support systems: The impact of preference elicitation techniques," European Journal of Operational Research, Elsevier, vol. 169(1), pages 273-285, February.
    2. Aksoy, Yasemin & Butler, Timothy W. & Minor, Elliott D., 1996. "Comparative studies in interactive multiple objective mathematical programming," European Journal of Operational Research, Elsevier, vol. 89(2), pages 408-422, March.
    3. Klimberg, Ronald & Cohen, Robert M., 1999. "Experimental evaluation of a graphical display system to visualizing multiple criteria solutions," European Journal of Operational Research, Elsevier, vol. 119(1), pages 191-208, November.
    4. Ana B. Ruiz & Francisco Ruiz & Kaisa Miettinen & Laura Delgado-Antequera & Vesa Ojalehto, 2019. "NAUTILUS Navigator: free search interactive multiobjective optimization without trading-off," Journal of Global Optimization, Springer, vol. 74(2), pages 213-231, June.
    5. Bhupinder Singh Saini & Michael Emmerich & Atanu Mazumdar & Bekir Afsar & Babooshka Shavazipour & Kaisa Miettinen, 2022. "Optimistic NAUTILUS navigator for multiobjective optimization with costly function evaluations," Journal of Global Optimization, Springer, vol. 83(4), pages 865-889, August.
    6. Moez Limayem & Gerardine DeSanctis, 2000. "Providing Decisional Guidance for Multicriteria Decision Making in Groups," Information Systems Research, INFORMS, vol. 11(4), pages 386-401, December.
    7. Zanakis, Stelios H. & Solomon, Anthony & Wishart, Nicole & Dublish, Sandipa, 1998. "Multi-attribute decision making: A simulation comparison of select methods," European Journal of Operational Research, Elsevier, vol. 107(3), pages 507-529, June.
    8. Downing, C. E. & Ringuest, J. L., 1998. "An experimental evaluation of the efficacy of four multi-objective linear programming algorithms," European Journal of Operational Research, Elsevier, vol. 104(3), pages 549-558, February.
    9. Hartikainen, Markus & Miettinen, Kaisa & Klamroth, Kathrin, 2019. "Interactive Nonconvex Pareto Navigator for multiobjective optimization," European Journal of Operational Research, Elsevier, vol. 275(1), pages 238-251.
    10. Sahar Mirzaee & David Fannon & Matthias Ruth, 2019. "A comparison of preference elicitation methods for multi-criteria design decisions about resilient and sustainable buildings," Environment Systems and Decisions, Springer, vol. 39(4), pages 439-453, December.
    11. Jaszkiewicz, Andrzej & Slowinski, Roman, 1999. "The `Light Beam Search' approach - an overview of methodology and applications," European Journal of Operational Research, Elsevier, vol. 113(2), pages 300-314, March.
    12. Maciej Nowak & Tadeusz Trzaskalik, 2022. "A trade-off multiobjective dynamic programming procedure and its application to project portfolio selection," Annals of Operations Research, Springer, vol. 311(2), pages 1155-1181, April.
    13. Michael Brusco & Stephanie Stahl, 2001. "An interactive multiobjective programming approach to combinatorial data analysis," Psychometrika, Springer;The Psychometric Society, vol. 66(1), pages 5-24, March.
    14. Rastegar, Narges & Khorram, Esmaile, 2014. "A combined scalarizing method for multiobjective programming problems," European Journal of Operational Research, Elsevier, vol. 236(1), pages 229-237.
    15. Tunjo Perić & Zoran Babić & Josip Matejaš, 2018. "Comparative analysis of application efficiency of two iterative multi objective linear programming methods (MP method and STEM method)," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 26(3), pages 565-583, September.
    16. Manel Baucells & Rakesh K. Sarin, 2003. "Group Decisions with Multiple Criteria," Management Science, INFORMS, vol. 49(8), pages 1105-1118, August.
    17. Ruiz, Francisco & Luque, Mariano & Miguel, Francisca & del Mar Munoz, Maria, 2008. "An additive achievement scalarizing function for multiobjective programming problems," European Journal of Operational Research, Elsevier, vol. 188(3), pages 683-694, August.
    18. Argyris, Nikolaos & Karsu, Özlem & Yavuz, Mirel, 2022. "Fair resource allocation: Using welfare-based dominance constraints," European Journal of Operational Research, Elsevier, vol. 297(2), pages 560-578.
    19. Barbati, Maria & Greco, Salvatore & Kadziński, Miłosz & Słowiński, Roman, 2018. "Optimization of multiple satisfaction levels in portfolio decision analysis," Omega, Elsevier, vol. 78(C), pages 192-204.
    20. Andrea Baldin & Trine Bille & Andrea Ellero & Daniela Favaretto, 2016. "Multiobjective optimization model for pricing and seat allocation problem in non profit performing arts organization," Working Papers 20, Venice School of Management - Department of Management, Università Ca' Foscari Venezia.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:338:y:2024:i:2:d:10.1007_s10479-024-05941-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.