IDEAS home Printed from https://ideas.repec.org/a/kap/enreec/v69y2018i2d10.1007_s10640-016-0083-6.html
   My bibliography  Save this article

Single-Choice, Repeated-Choice, and Best-Worst Scaling Elicitation Formats: Do Results Differ and by How Much?

Author

Listed:
  • Daniel R. Petrolia

    (Mississippi State University)

  • Matthew G. Interis

    (Mississippi State University)

  • Joonghyun Hwang

    (Florida Fish and Wildlife Conservation Commission)

Abstract

This paper presents what we believe to be the most comprehensive suite of comparison criteria regarding multinomial discrete-choice experiment elicitation formats to date. We administer a choice experiment focused on ecosystem-service valuation to three independent samples: single-choice, repeated-choice, and best-worst scaling elicitation. We test whether results differ by parameter estimates, scale factors, preference heterogeneity, status-quo effects, attribute non-attendance, and magnitude and precision of welfare measures. Overall, we find limited evidence of differences in attribute parameter estimates, scale factors, and attribute increment values across elicitation treatments. However, we find significant differences in status-quo effects across elicitation treatments, with repeated-choice resulting in greater proportions of “action” votes, and consequently, higher program-level welfare estimates. Also, we find that single-choice yields drastically less-precise welfare estimates. Finally, we find some differences in attribute non-attendance behavior across elicitation formats, although there appears to be little consistency in class shares even within a given elicitation treatment.

Suggested Citation

  • Daniel R. Petrolia & Matthew G. Interis & Joonghyun Hwang, 2018. "Single-Choice, Repeated-Choice, and Best-Worst Scaling Elicitation Formats: Do Results Differ and by How Much?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 69(2), pages 365-393, February.
    • Handle: RePEc:kap:enreec:v:69:y:2018:i:2:d:10.1007_s10640-016-0083-6
    DOI: 10.1007/s10640-016-0083-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10640-016-0083-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10640-016-0083-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jürgen Meyerhoff & Ulf Liebe, 2009. "Status Quo Effect in Choice Experiments: Empirical Evidence on Attitudes and Choice Task Complexity," Land Economics, University of Wisconsin Press, vol. 85(3), pages 515-528.
    2. Hanemann, W. Michael, . "Some Issues In Continuous - And Discrete - Response Contingent Valuation Studies," Northeastern Journal of Agricultural and Resource Economics, Northeastern Agricultural and Resource Economics Association, vol. 14(1), pages 1-9.
    3. Ladenburg, Jacob & Olsen, Søren Bøye, 2008. "Gender-specific starting point bias in choice experiments: Evidence from an empirical study," Journal of Environmental Economics and Management, Elsevier, vol. 56(3), pages 275-285, November.
    4. Christian A. Vossler & Maurice Doyon & Daniel Rondeau, 2012. "Truth in Consequentiality: Theory and Field Evidence on Discrete Choice Experiments," American Economic Journal: Microeconomics, American Economic Association, vol. 4(4), pages 145-171, November.
    5. Mohammed Alemu & Morten Mørkbak & Søren Olsen & Carsten Jensen, 2013. "Attending to the Reasons for Attribute Non-attendance in Choice Experiments," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 54(3), pages 333-359, March.
    6. Collins, Andrew T. & Rose, John M. & Hensher, David A., 2013. "Specification issues in a generalised random parameters attribute nonattendance model," Transportation Research Part B: Methodological, Elsevier, vol. 56(C), pages 234-253.
    7. Newell, Laurie W. & Swallow, Stephen K., 2013. "Real-payment choice experiments: Valuing forested wetlands and spatial attributes within a landscape context," Ecological Economics, Elsevier, vol. 92(C), pages 37-47.
    8. Danny Campbell & W. Hutchinson & Riccardo Scarpa, 2008. "Incorporating Discontinuous Preferences into the Analysis of Discrete Choice Experiments," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 41(3), pages 401-417, November.
    9. McNair, Ben J. & Bennett, Jeff & Hensher, David A., 2011. "A comparison of responses to single and repeated discrete choice questions," Resource and Energy Economics, Elsevier, vol. 33(3), pages 554-571, September.
    10. Flynn, Terry N. & Louviere, Jordan J. & Peters, Tim J. & Coast, Joanna, 2007. "Best-worst scaling: What it can do for health care research and how to do it," Journal of Health Economics, Elsevier, vol. 26(1), pages 171-189, January.
    11. Richard Carson & Jordan Louviere, 2011. "A Common Nomenclature for Stated Preference Elicitation Approaches," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 49(4), pages 539-559, August.
    12. Olivier Beaumais & Dominique Prunetti & Anne Casabianca & Xavier Pieri, 2015. "Improving Solid Waste Management in the Island of Beauty (Corsica): a Latent-Class Rank-Ordered Logit Approach with Observed Heterogenous Ranking Capabilities," Revue d'économie politique, Dalloz, vol. 125(2), pages 209-231.
    13. Riccardo Scarpa & Mara Thiene & David A. Hensher, 2010. "Monitoring Choice Task Attribute Attendance in Nonmarket Valuation of Multiple Park Management Services: Does It Matter?," Land Economics, University of Wisconsin Press, vol. 86(4), pages 817-839.
    14. Carson, Richard T. & Czajkowski, Mikołaj, 2019. "A new baseline model for estimating willingness to pay from discrete choice models," Journal of Environmental Economics and Management, Elsevier, vol. 95(C), pages 57-61.
    15. List John A. & Sinha Paramita & Taylor Michael H., 2006. "Using Choice Experiments to Value Non-Market Goods and Services: Evidence from Field Experiments," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 6(2), pages 1-39, January.
    16. Potoglou, Dimitris & Burge, Peter & Flynn, Terry & Netten, Ann & Malley, Juliette & Forder, Julien & Brazier, John E., 2011. "Best-worst scaling vs. discrete choice experiments: An empirical comparison using social care data," Social Science & Medicine, Elsevier, vol. 72(10), pages 1717-1727, May.
    17. Richard Carson & Theodore Groves, 2007. "Incentive and informational properties of preference questions," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 37(1), pages 181-210, May.
    18. David Hensher & William Greene, 2003. "The Mixed Logit model: The state of practice," Transportation, Springer, vol. 30(2), pages 133-176, May.
    19. Ben McNair & David Hensher & Jeff Bennett, 2012. "Modelling Heterogeneity in Response Behaviour Towards a Sequence of Discrete Choice Questions: A Probabilistic Decision Process Model," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 51(4), pages 599-616, April.
    20. David Hensher & William Greene, 2010. "Non-attendance and dual processing of common-metric attributes in choice analysis: a latent class specification," Empirical Economics, Springer, vol. 39(2), pages 413-426, October.
    21. Bateman, Ian J. & Cole, Matthew & Cooper, Philip & Georgiou, Stavros & Hadley, David & Poe, Gregory L., 2004. "On visible choice sets and scope sensitivity," Journal of Environmental Economics and Management, Elsevier, vol. 47(1), pages 71-93, January.
    22. Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521766555.
    23. Timothy C. Haab & Kenneth E. McConnell, 2002. "Valuing Environmental and Natural Resources," Books, Edward Elgar Publishing, number 2427.
    24. John Pattison & Peter C. Boxall & Wiktor L. Adamowicz, 2011. "The Economic Benefits of Wetland Retention and Restoration in Manitoba," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 59, pages 223-244, June.
    25. Laura Taylor & Mark Morrison & Kevin Boyle, 2010. "Exchange Rules and the Incentive Compatibility of Choice Experiments," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 47(2), pages 197-220, October.
    26. Gregory L. Poe & Kelly L. Giraud & John B. Loomis, 2005. "Computational Methods for Measuring the Difference of Empirical Distributions," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 87(2), pages 353-365.
    27. Matthew G. Interis & Daniel R. Petrolia, 2016. "Location, Location, Habitat: How the Value of Ecosystem Services Varies across Location and by Habitat," Land Economics, University of Wisconsin Press, vol. 92(2), pages 292-307.
    28. David Hensher & Andrew Collins & William Greene, 2013. "Accounting for attribute non-attendance and common-metric aggregation in a probabilistic decision process mixed multinomial logit model: a warning on potential confounding," Transportation, Springer, vol. 40(5), pages 1003-1020, September.
    29. R.K. Blamey & J.W. Bennett & J.J. Louviere & M.D. Morrison & J.C. Rolfe, 2002. "Attribute Causality in Environmental Choice Modelling," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 23(2), pages 167-186, October.
    30. Dan Rigby & Michael Burton & Jayson L. Lusk, 2015. "Journals, Preferences, and Publishing in Agricultural and Environmental Economics," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 97(2), pages 490-509.
    31. Day, Brett & Pinto Prades, Jose-Luis, 2010. "Ordering anomalies in choice experiments," Journal of Environmental Economics and Management, Elsevier, vol. 59(3), pages 271-285, May.
    32. Richard T. Carson, 2012. "Contingent Valuation: A Practical Alternative When Prices Aren't Available," Journal of Economic Perspectives, American Economic Association, vol. 26(4), pages 27-42, Fall.
    33. David Hensher & John Rose & William Greene, 2012. "Inferring attribute non-attendance from stated choice data: implications for willingness to pay estimates and a warning for stated choice experiment design," Transportation, Springer, vol. 39(2), pages 235-245, March.
    34. Daniel R. Petrolia & Matthew G. Interis & Joonghyun Hwang, 2014. "America's Wetland? A National Survey of Willingness to Pay for Restoration of Louisiana's Coastal Wetlands," Marine Resource Economics, University of Chicago Press, vol. 29(1), pages 17-37.
    35. Thomas P. Holmes & Kevin J. Boyle, 2005. "Dynamic Learning and Context-Dependence in Sequential, Attribute-Based, Stated-Preference Valuation Questions," Land Economics, University of Wisconsin Press, vol. 81(1).
    36. Gabriela Scheufele & Jeff Bennett, 2012. "Response Strategies and Learning in Discrete Choice Experiments," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 52(3), pages 435-453, July.
    37. Danny Campbell & David A. Hensher & Riccardo Scarpa, 2011. "Non-attendance to attributes in environmental choice analysis: a latent class specification," Journal of Environmental Planning and Management, Taylor & Francis Journals, vol. 54(8), pages 1061-1076, December.
    38. Stephane Hess & Amanda Stathopoulos & Danny Campbell & Vikki O’Neill & Sebastian Caussade, 2013. "It’s not that I don’t care, I just don’t care very much: confounding between attribute non-attendance and taste heterogeneity," Transportation, Springer, vol. 40(3), pages 583-607, May.
    39. Day, Brett & Bateman, Ian J. & Carson, Richard T. & Dupont, Diane & Louviere, Jordan J. & Morimoto, Sanae & Scarpa, Riccardo & Wang, Paul, 2012. "Ordering effects and choice set awareness in repeat-response stated preference studies," Journal of Environmental Economics and Management, Elsevier, vol. 63(1), pages 73-91.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Patrick Lloyd-Smith & Ewa Zawojska & Wiktor Adamowicz, 2020. "Moving beyond the Contingent Valuation versus Choice Experiment Debate: Presentation Effects in Stated Preference," Land Economics, University of Wisconsin Press, vol. 96(1), pages 1-24.
    2. Palm-Forster, Leah H. & Taylor, Mykel & Banerjee, Simanti & Xie, Lusi, 2023. "Factors influencing enrollment of leased cropland in the Conservation Stewardship Program in Kansas," Land Use Policy, Elsevier, vol. 135(C).
    3. John Robinson, Peter & van Beukering, Pieter & Brander, Luke & Brouwer, Roy & Haider, W. & Taylor, Michael & Mau, Paulus, 2022. "Understanding the determinants of biodiversity non-use values in the context of climate change: Stated preferences for the Hawaiian coral reefs," Ecosystem Services, Elsevier, vol. 53(C).
    4. Boxebeld, Sander, 2024. "Ordering effects in discrete choice experiments: A systematic literature review across domains," Journal of choice modelling, Elsevier, vol. 51(C).
    5. Robert Huber & Robert Finger, 2020. "A Meta‐analysis of the Willingness to Pay for Cultural Services from Grasslands in Europe," Journal of Agricultural Economics, Wiley Blackwell, vol. 71(2), pages 357-383, June.
    6. Samare P. I. Huls & Emily Lancsar & Bas Donkers & Jemimah Ride, 2022. "Two for the price of one: If moving beyond traditional single‐best discrete choice experiments, should we use best‐worst, best‐best or ranking for preference elicitation?," Health Economics, John Wiley & Sons, Ltd., vol. 31(12), pages 2630-2647, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Petrolia, Daniel & Interis, Matthew & Hwang, Joonghyun, 2015. "Single-Choice, Repeated-Choice, and Best-Worst Elicitation Formats: Do Results Differ and by How Much?," Working Papers 212479, Mississippi State University, Department of Agricultural Economics.
    2. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    3. Nguyen, Thanh Cong & Le, Hoa Thu & Nguyen, Hang Dieu & Ngo, Mai Thanh & Nguyen, Hong Quang, 2021. "Examining ordering effects and strategic behaviour in a discrete choice experiment," Economic Analysis and Policy, Elsevier, vol. 70(C), pages 394-413.
    4. Logar, Ivana & Brouwer, Roy & Campbell, Danny, 2020. "Does attribute order influence attribute-information processing in discrete choice experiments?," Resource and Energy Economics, Elsevier, vol. 60(C).
    5. Sandorf, Erlend Dancke & Crastes dit Sourd, Romain & Mahieu, Pierre-Alexandre, 2018. "The effect of attribute-alternative matrix displays on preferences and processing strategies," Journal of choice modelling, Elsevier, vol. 29(C), pages 113-132.
    6. Gonçalves, Tânia & Lourenço-Gomes, Lina & Pinto, Lígia M. Costa, 2022. "The role of attribute non-attendance on consumer decision-making: Theoretical insights and empirical evidence," Economic Analysis and Policy, Elsevier, vol. 76(C), pages 788-805.
    7. Daniel R. Petrolia & Matthew G. Interis & Joonghyun Hwang, 2014. "America's Wetland? A National Survey of Willingness to Pay for Restoration of Louisiana's Coastal Wetlands," Marine Resource Economics, University of Chicago Press, vol. 29(1), pages 17-37.
    8. Hole, Arne Risa & Kolstad, Julie Riise & Gyrd-Hansen, Dorte, 2013. "Inferred vs. stated attribute non-attendance in choice experiments: A study of doctors’ prescription behaviour," Journal of Economic Behavior & Organization, Elsevier, vol. 96(C), pages 21-31.
    9. Mohammed H. Alemu & Søren B. Olsen, 2017. "Can a Repeated Opt-Out Reminder remove hypothetical bias in discrete choice experiments? An application to consumer valuation of novel food products," IFRO Working Paper 2017/05, University of Copenhagen, Department of Food and Resource Economics.
    10. Espinosa-Goded, María & Rodriguez-Entrena, Macario & Salazar-Ordóñez, Melania, 2021. "A straightforward diagnostic tool to identify attribute non-attendance in discrete choice experiments," Economic Analysis and Policy, Elsevier, vol. 71(C), pages 211-226.
    11. Klaus Glenk & Julia Martin-Ortega & Manuel Pulido-Velazquez & Jacqueline Potts, 2015. "Inferring Attribute Non-attendance from Discrete Choice Experiments: Implications for Benefit Transfer," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 60(4), pages 497-520, April.
    12. Colombo, Sergio & Christie, Michael & Hanley, Nick, 2013. "What are the consequences of ignoring attributes in choice experiments? Implications for ecosystem service valuation," Ecological Economics, Elsevier, vol. 96(C), pages 25-35.
    13. Meyerhoff, Jürgen & Glenk, Klaus, 2015. "Learning how to choose—effects of instructional choice sets in discrete choice experiments," Resource and Energy Economics, Elsevier, vol. 41(C), pages 122-142.
    14. Collins, Andrew T. & Rose, John M. & Hensher, David A., 2013. "Specification issues in a generalised random parameters attribute nonattendance model," Transportation Research Part B: Methodological, Elsevier, vol. 56(C), pages 234-253.
    15. Glenk, Klaus & Meyerhoff, Jürgen & Akaichi, Faical & Martin-Ortega, Julia, 2019. "Revisiting cost vector effects in discrete choice experiments," Resource and Energy Economics, Elsevier, vol. 57(C), pages 135-155.
    16. Weller, Priska & Oehlmann, Malte & Mariel, Petr & Meyerhoff, Jürgen, 2014. "Stated and inferred attribute non-attendance in a design of designs approach," Journal of choice modelling, Elsevier, vol. 11(C), pages 43-56.
    17. Chen, Xuqi & Shen, Meng & Gao, Zhifeng, 2017. "Impact of Intra-respondent Variations in Attribute Attendance on Consumer Preference in Food Choice," 2017 Annual Meeting, July 30-August 1, Chicago, Illinois 258509, Agricultural and Applied Economics Association.
    18. David Hensher & Andrew Collins & William Greene, 2013. "Accounting for attribute non-attendance and common-metric aggregation in a probabilistic decision process mixed multinomial logit model: a warning on potential confounding," Transportation, Springer, vol. 40(5), pages 1003-1020, September.
    19. Mørkbak, Morten Raun & Olsen, Søren Bøye & Campbell, Danny, 2014. "Behavioral implications of providing real incentives in stated choice experiments," Journal of Economic Psychology, Elsevier, vol. 45(C), pages 102-116.
    20. Campbell, Danny & Boeri, Marco & Doherty, Edel & George Hutchinson, W., 2015. "Learning, fatigue and preference formation in discrete choice experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 119(C), pages 345-363.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:enreec:v:69:y:2018:i:2:d:10.1007_s10640-016-0083-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.