IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v253y2016i3p746-760.html
   My bibliography  Save this article

Preference stability over time with multiple elicitation methods to support wastewater infrastructure decision-making

Author

Listed:
  • Lienert, Judit
  • Duygan, Mert
  • Zheng, Jun

Abstract

We used a multi-method and repeated elicitation approach across different stakeholder groups to explore possible differences in the outcome of an environmental decision. We compared different preference elicitation procedures based on Multi Criteria Decision Analysis (MCDA) over time for a water infrastructure decision in Switzerland. We implemented the SWING and SMART/SWING weight elicitation methods and also compared results with earlier stakeholder interviews. In all procedures, the weights for environmental protection and well-functioning (waste-)water systems were higher than for cost reduction. The SMART/SWING variant produced statistically significantly different weights than SWING. Weights changed over time with both elicitation methods. Weights were more stable with the SWING method, which was also perceived as slightly more difficult than the SMART/SWING variant. We checked whether the difference in weights produced by the two elicitation methods and the difference in their stability affects the ranking of six alternatives. Overall an unconventional decentralized alternative ranked first or second in 92 percent of all elicitation procedures, which were the online surveys or interviews. For practical decision-making, using multiple methods across different stakeholder groups and repeating elicitation can increase our confidence that the results reflect the true opinions of the decision makers and stakeholders.

Suggested Citation

  • Lienert, Judit & Duygan, Mert & Zheng, Jun, 2016. "Preference stability over time with multiple elicitation methods to support wastewater infrastructure decision-making," European Journal of Operational Research, Elsevier, vol. 253(3), pages 746-760.
    • Handle: RePEc:eee:ejores:v:253:y:2016:i:3:p:746-760
    DOI: 10.1016/j.ejor.2016.03.010
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221716301382
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2016.03.010?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. John Hey & Andrea Morone & Ulrich Schmidt, 2009. "Noise and bias in eliciting preferences," Journal of Risk and Uncertainty, Springer, vol. 39(3), pages 213-235, December.
    2. Scholten, Lisa & Schuwirth, Nele & Reichert, Peter & Lienert, Judit, 2015. "Tackling uncertainty in multi-criteria decision analysis – An application to water supply infrastructure planning," European Journal of Operational Research, Elsevier, vol. 242(1), pages 243-260.
    3. Roy Brouwer & Thijs Dekker & John Rolfe & Jill Windle, 2010. "Choice Certainty and Consistency in Repeated Choice Experiments," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 46(1), pages 93-109, May.
    4. Schaafsma, Marije & Brouwer, Roy & Liekens, Inge & De Nocker, Leo, 2014. "Temporal stability of preferences and willingness to pay for natural areas in choice experiments: A test–retest," Resource and Energy Economics, Elsevier, vol. 38(C), pages 243-260.
    5. Katrin Borcherding & Thomas Eppel & Detlof von Winterfeldt, 1991. "Comparison of Weighting Judgments in Multiattribute Utility Measurement," Management Science, INFORMS, vol. 37(12), pages 1603-1619, December.
    6. Van Ittersum, Koert & Pennings, Joost M.E. & Wansink, Brian & van Trijp, Hans C.M., 2007. "The validity of attribute-importance measurement: A review," Journal of Business Research, Elsevier, vol. 60(11), pages 1177-1190, November.
    7. Weber, Martin & Borcherding, Katrin, 1993. "Behavioral influences on weight judgments in multiattribute decision making," European Journal of Operational Research, Elsevier, vol. 67(1), pages 1-12, May.
    8. Hämäläinen, Raimo P. & Luoma, Jukka & Saarinen, Esa, 2013. "On the importance of behavioral operational research: The case of understanding and communicating about dynamic systems," European Journal of Operational Research, Elsevier, vol. 228(3), pages 623-634.
    9. San Miguel, Fernando & Ryan, Mandy & Scott, Anthony, 2002. "Are preferences stable? The case of health care," Journal of Economic Behavior & Organization, Elsevier, vol. 48(1), pages 1-14, May.
    10. Sarah K. Jacobi & Benjamin F. Hobbs, 2007. "Quantifying and Mitigating the Splitting Bias and Other Value Tree-Induced Weighting Biases," Decision Analysis, INFORMS, vol. 4(4), pages 194-210, December.
    11. Doyle, John R. & Green, Rodney H. & Bottomley, Paul A., 1997. "Judging Relative Importance: Direct Rating and Point Allocation Are Not Equivalent," Organizational Behavior and Human Decision Processes, Elsevier, vol. 70(1), pages 65-72, April.
    12. Bottomley, Paul A. & Doyle, John R., 2001. "A comparison of three weight elicitation methods: good, better, and best," Omega, Elsevier, vol. 29(6), pages 553-560, December.
    13. Huizingh, Eelko K. R. E. & Vrolijk, Hans C. J., 1997. "A Comparison of Verbal and Numerical Judgments in the Analytic Hierarchy Process," Organizational Behavior and Human Decision Processes, Elsevier, vol. 70(3), pages 237-247, June.
    14. A Morton & B Fasolo, 2009. "Behavioural decision theory for multi-criteria decision analysis: a guided tour," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(2), pages 268-275, February.
    15. Stefan A. Hajkowicz, 2012. "For the Greater Good? A Test for Strategic Bias in Group Environmental Decisions," Group Decision and Negotiation, Springer, vol. 21(3), pages 331-344, May.
    16. Schuwirth, N. & Reichert, P. & Lienert, J., 2012. "Methodological aspects of multi-criteria decision analysis for policy support: A case study on pharmaceutical removal from hospital wastewater," European Journal of Operational Research, Elsevier, vol. 220(2), pages 472-483.
    17. Ulf Liebe & Jürgen Meyerhoff & Volkmar Hartje, 2012. "Test–Retest Reliability of Choice Experiments in Environmental Valuation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 53(3), pages 389-407, November.
    18. Belton, Valerie, 1986. "A comparison of the analytic hierarchy process and a simple multi-attribute value function," European Journal of Operational Research, Elsevier, vol. 26(1), pages 7-21, July.
    19. Veronesi, Marcella & Chawla, Fabienne & Maurer, Max & Lienert, Judit, 2014. "Climate change and the willingness to pay to reduce ecological and health risks from wastewater flooding in urban centers and the environment," Ecological Economics, Elsevier, vol. 98(C), pages 1-10.
    20. Gregory, Robin & Lichtenstein, Sarah & Slovic, Paul, 1993. "Valuing Environmental Resources: A Constructive Approach," Journal of Risk and Uncertainty, Springer, vol. 7(2), pages 177-197, October.
    21. Jorgensen, Bradley S. & Syme, Geoffrey J. & Smith, Leigh M. & Bishop, Brian J., 2004. "Random error in willingness to pay measurement: A multiple indicators, latent variable approach to the reliability of contingent values," Journal of Economic Psychology, Elsevier, vol. 25(1), pages 41-59, February.
    22. Stefan Zeisberger & Dennis Vrecko & Thomas Langer, 2012. "Measuring the time stability of Prospect Theory preferences," Theory and Decision, Springer, vol. 72(3), pages 359-386, March.
    23. Brouwer, Roy, 2006. "Do stated preference methods stand the test of time? A test of the stability of contingent values and models for health risks when facing an extreme event," Ecological Economics, Elsevier, vol. 60(2), pages 399-406, December.
    24. Bottomley, Paul A. & Doyle, John R., 2013. "Comparing the validity of numerical judgements elicited by direct rating and point allocation: Insights from objectively verifiable perceptual tasks," European Journal of Operational Research, Elsevier, vol. 228(1), pages 148-157.
    25. Hämäläinen, Raimo P. & Alaja, Susanna, 2008. "The threat of weighting biases in environmental decision analysis," Ecological Economics, Elsevier, vol. 68(1-2), pages 556-569, December.
    26. Stefan Hajkowicz & Kerry Collins, 2007. "A Review of Multiple Criteria Analysis for Water Resource Planning and Management," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 21(9), pages 1553-1566, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ahmad, Salman & Ouenniche, Jamal & Kolosz, Ben W. & Greening, Philip & Andresen, John M. & Maroto-Valer, M. Mercedes & Xu, Bing, 2021. "A stakeholders’ participatory approach to multi-criteria assessment of sustainable aviation fuels production pathways," International Journal of Production Economics, Elsevier, vol. 238(C).
    2. Luis C. Dias & Gabriela D. Oliveira & Paula Sarabando, 2021. "Choice-based preference disaggregation concerning vehicle technologies," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 29(1), pages 177-200, March.
    3. Mónica D. Oliveira & Inês Mataloto & Panos Kanavos, 2019. "Multi-criteria decision analysis for health technology assessment: addressing methodological challenges to improve the state of the art," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(6), pages 891-918, August.
    4. Kuller, M. & Beutler, P. & Lienert, J., 2023. "Preference change in stakeholder group-decision processes in the public sector: Extent, causes and implications," European Journal of Operational Research, Elsevier, vol. 308(3), pages 1268-1285.
    5. Zheng, Jun & Lienert, Judit, 2018. "Stakeholder interviews with two MAVT preference elicitation philosophies in a Swiss water infrastructure decision: Aggregation using SWING-weighting and disaggregation using UTAGMS," European Journal of Operational Research, Elsevier, vol. 267(1), pages 273-287.
    6. Marttunen, Mika & Belton, Valerie & Lienert, Judit, 2018. "Are objectives hierarchy related biases observed in practice? A meta-analysis of environmental and energy applications of Multi-Criteria Decision Analysis," European Journal of Operational Research, Elsevier, vol. 265(1), pages 178-194.
    7. Franco, L. Alberto & Hämäläinen, Raimo P. & Rouwette, Etiënne A.J.A. & Leppänen, Ilkka, 2021. "Taking stock of behavioural OR: A review of behavioural studies with an intervention focus," European Journal of Operational Research, Elsevier, vol. 293(2), pages 401-418.
    8. Fernández, Rafael & Calvo, Andrés & Correal, Juan Francisco & D'Ayala, Dina & Medaglia, Andrés L., 2024. "Large-scale school building infrastructure improvement: The case of the city of Cali, Colombia," Socio-Economic Planning Sciences, Elsevier, vol. 93(C).
    9. Perni, Ángel & Barreiro-Hurlé, Jesús & Martínez-Paz, José Miguel, 2020. "When policy implementation failures affect public preferences for environmental goods: Implications for economic analysis in the European water policy," Ecological Economics, Elsevier, vol. 169(C).
    10. Aubert, Alice H. & Schmid, Sara & Lienert, Judit, 2024. "Can online interfaces enhance learning for public decision-making? Eliciting citizens’ preferences for multicriteria decision analysis," European Journal of Operational Research, Elsevier, vol. 314(2), pages 760-775.
    11. Aubert, Alice H. & Esculier, Fabien & Lienert, Judit, 2020. "Recommendations for online elicitation of swing weights from citizens in environmental decision-making," Operations Research Perspectives, Elsevier, vol. 7(C).
    12. Aubert, Alice H. & Lienert, Judit, 2024. "Operational Research for, with, and by citizens: An overview," European Journal of Operational Research, Elsevier, vol. 316(3), pages 800-814.
    13. Bana e Costa, Carlos A. & Oliveira, Mónica D. & Vieira, Ana C.L. & Freitas, Liliana & Rodrigues, Teresa C. & Bana e Costa, João & Freitas, Ângela & Santana, Paula, 2023. "Collaborative development of composite indices from qualitative value judgements: The EURO-HEALTHY Population Health Index model," European Journal of Operational Research, Elsevier, vol. 305(1), pages 475-492.
    14. Haag, Fridolin & Zürcher, Sara & Lienert, Judit, 2019. "Enhancing the elicitation of diverse decision objectives for public planning," European Journal of Operational Research, Elsevier, vol. 279(3), pages 912-928.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marttunen, Mika & Belton, Valerie & Lienert, Judit, 2018. "Are objectives hierarchy related biases observed in practice? A meta-analysis of environmental and energy applications of Multi-Criteria Decision Analysis," European Journal of Operational Research, Elsevier, vol. 265(1), pages 178-194.
    2. Zheng, Jun & Lienert, Judit, 2018. "Stakeholder interviews with two MAVT preference elicitation philosophies in a Swiss water infrastructure decision: Aggregation using SWING-weighting and disaggregation using UTAGMS," European Journal of Operational Research, Elsevier, vol. 267(1), pages 273-287.
    3. Marttunen, Mika & Haag, Fridolin & Belton, Valerie & Mustajoki, Jyri & Lienert, Judit, 2019. "Methods to inform the development of concise objectives hierarchies in multi-criteria decision analysis," European Journal of Operational Research, Elsevier, vol. 277(2), pages 604-620.
    4. Scholten, Lisa & Schuwirth, Nele & Reichert, Peter & Lienert, Judit, 2015. "Tackling uncertainty in multi-criteria decision analysis – An application to water supply infrastructure planning," European Journal of Operational Research, Elsevier, vol. 242(1), pages 243-260.
    5. Ewa Roszkowska, 2020. "The extention rank ordering criteria weighting methods in fuzzy enviroment," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 30(2), pages 91-114.
    6. Kuller, M. & Beutler, P. & Lienert, J., 2023. "Preference change in stakeholder group-decision processes in the public sector: Extent, causes and implications," European Journal of Operational Research, Elsevier, vol. 308(3), pages 1268-1285.
    7. Gilberto Montibeller & Detlof von Winterfeldt, 2015. "Cognitive and Motivational Biases in Decision and Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 35(7), pages 1230-1251, July.
    8. J. Price & D. Dupont & W. Adamowicz, 2017. "As Time Goes By: Examination of Temporal Stability Across Stated Preference Question Formats," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 68(3), pages 643-662, November.
    9. Matthews, Yvonne & Scarpa, Riccardo & Marsh, Dan, 2017. "Stability of Willingness-to-Pay for Coastal Management: A Choice Experiment Across Three Time Periods," Ecological Economics, Elsevier, vol. 138(C), pages 64-73.
    10. Lahtinen, Tuomas J. & Hämäläinen, Raimo P., 2016. "Path dependence and biases in the even swaps decision analysis method," European Journal of Operational Research, Elsevier, vol. 249(3), pages 890-898.
    11. Suk, Kwanho & Yoon, Song-Oh, 2012. "The moderating role of decision task goals in attribute weight convergence," Organizational Behavior and Human Decision Processes, Elsevier, vol. 118(1), pages 37-45.
    12. Dan Rigby & Michael Burton & Jo Pluske, 2016. "Preference Stability and Choice Consistency in Discrete Choice Experiments," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 65(2), pages 441-461, October.
    13. Garmendia, Eneko & Gamboa, Gonzalo, 2012. "Weighting social preferences in participatory multi-criteria evaluations: A case study on sustainable natural resource management," Ecological Economics, Elsevier, vol. 84(C), pages 110-120.
    14. Roy Brouwer & Ivana Logar & Oleg Sheremet, 2017. "Choice Consistency and Preference Stability in Test-Retests of Discrete Choice Experiment and Open-Ended Willingness to Pay Elicitation Formats," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 68(3), pages 729-751, November.
    15. Eneko Garmendia & Gonzalo Gamboa, 2012. "Weighting social preferences in participatory multi-criteria evaluations: a case study on sustainable natural resource management," Working Papers 2012-06, BC3.
    16. Madson Bruno da Silva Monte & Danielle Costa Morais, 2019. "A Decision Model for Identifying and Solving Problems in an Urban Water Supply System," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 33(14), pages 4835-4848, November.
    17. Wulf, David & Bertsch, Valentin, 2016. "A natural language generation approach to support understanding and traceability of multi-dimensional preferential sensitivity analysis in multi-criteria decision making," MPRA Paper 75025, University Library of Munich, Germany.
    18. Denise Doiron & Hong Il Yoo, 2020. "Stated preferences over job characteristics: A panel study," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 53(1), pages 43-82, February.
    19. Scholz, Michael & Dorner, Verena & Schryen, Guido & Benlian, Alexander, 2017. "A configuration-based recommender system for supporting e-commerce decisions," European Journal of Operational Research, Elsevier, vol. 259(1), pages 205-215.
    20. Lahtinen, Tuomas J. & Hämäläinen, Raimo P. & Jenytin, Cosmo, 2020. "On preference elicitation processes which mitigate the accumulation of biases in multi-criteria decision analysis," European Journal of Operational Research, Elsevier, vol. 282(1), pages 201-210.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:253:y:2016:i:3:p:746-760. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.