IDEAS home Printed from https://ideas.repec.org/a/wly/hlthec/v20y2011i3p273-286.html
   My bibliography  Save this article

Does the number of choice sets matter? Results from a web survey applying a discrete choice experiment

Author

Listed:
  • Mickael Bech
  • Trine Kjaer
  • Jørgen Lauridsen

Abstract

Optimising the design of discrete choice experiments (DCE) involves maximising not only the statistical efficiency, but also how the nature and complexity of the experiment itself affects model parameters and variance. The present paper contributes by investigating the impact of the number of DCE choice sets presented to each respondent on response rate, self‐reported choice certainty, perceived choice difficulty, willingness‐to‐pay (WTP) estimates, and response variance. A sample of 1053 respondents was exposed to 5, 9 or 17 choice sets in a DCE eliciting preferences for dental services. Our results showed no differences in response rates and no systematic differences in the respondents' self‐reported perception of the uncertainty of their DCE answers. There were some differences in WTP estimates suggesting that estimated preferences are to some extent context‐dependent, but no differences in standard deviations for WTP estimates or goodness‐of‐fit statistics. Respondents exposed to 17 choice sets had somewhat higher response variance compared to those exposed to 5 choice sets, indicating that cognitive burden may increase with the number of choice sets beyond a certain threshold. Overall, our results suggest that respondents are capable of managing multiple choice sets – in this case 17 choice sets – without problems. Copyright © 2010 John Wiley & Sons, Ltd.

Suggested Citation

  • Mickael Bech & Trine Kjaer & Jørgen Lauridsen, 2011. "Does the number of choice sets matter? Results from a web survey applying a discrete choice experiment," Health Economics, John Wiley & Sons, Ltd., vol. 20(3), pages 273-286, March.
    • Handle: RePEc:wly:hlthec:v:20:y:2011:i:3:p:273-286
    DOI: 10.1002/hec.1587
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/hec.1587
    Download Restriction: no
    File URL: https://libkey.io/10.1002/hec.1587?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. David Hensher, 2006. "Revealing Differences in Willingness to Pay due to the Dimensionality of Stated Choice Designs: An Initial Assessment," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 34(1), pages 7-44, May.
    2. Mickael Bech & Dorte Gyrd‐Hansen, 2005. "Effects coding in discrete choice experiments," Health Economics, John Wiley & Sons, Ltd., vol. 14(10), pages 1079-1083, October.
    3. Swait, Joffre & Adamowicz, Wiktor, 2001. "The Influence of Task Complexity on Consumer Choice: A Latent Class Model of Decision Strategy Switching," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 28(1), pages 135-148, June.
    4. Caussade, Sebastián & Ortúzar, Juan de Dios & Rizzi, Luis I. & Hensher, David A., 2005. "Assessing the influence of design dimensions on stated choice experiment estimates," Transportation Research Part B: Methodological, Elsevier, vol. 39(7), pages 621-640, August.
    5. Arne Risa Hole, 2006. "Small-sample properties of tests for heteroscedasticity in the conditional logit model," Economics Bulletin, AccessEcon, vol. 3(18), pages 1-14.
    6. Carlsson, Fredrik & Martinsson, Peter, 2006. "How much is too much? - An investigation of the effect of the number of choice sets, starting point and the choice of bid vectors in choice experiments," Working Papers in Economics 191, University of Gothenburg, Department of Economics.
    7. Emily Lancsar & Jordan Louviere, 2006. "Deleting ‘irrational’ responses from discrete choice experiments: a case of investigating or imposing preferences?," Health Economics, John Wiley & Sons, Ltd., vol. 15(8), pages 797-811, August.
    8. Johnson, F. Reed & Desvousges, William H., 1997. "Estimating Stated Preferences with Rated-Pair Data: Environmental, Health, and Employment Effects of Energy Programs," Journal of Environmental Economics and Management, Elsevier, vol. 34(1), pages 79-99, September.
    9. Hanley, Nick & Adamowicz, Wiktor & Wright, Robert E., 2005. "Price vector effects in choice experiments: an empirical test," Resource and Energy Economics, Elsevier, vol. 27(3), pages 227-234, October.
    10. Hensher, David A. & Stopher, Peter R. & Louviere, Jordan J., 2001. "An exploratory analysis of the effect of numbers of choice sets in designed choice experiments: an airline choice application," Journal of Air Transport Management, Elsevier, vol. 7(6), pages 373-379.
    11. Scott, Anthony, 2002. "Identifying and analysing dominant preferences in discrete choice experiments: An application in health care," Journal of Economic Psychology, Elsevier, vol. 23(3), pages 383-398, June.
    12. Swait, Joffre & Adamowicz, Wiktor, 2001. "Choice Environment, Market Complexity, and Consumer Behavior: A Theoretical and Empirical Approach for Incorporating Decision Complexity into Models of Consumer Choice," Organizational Behavior and Human Decision Processes, Elsevier, vol. 86(2), pages 141-167, November.
    13. Laura O. Taylor & Ronald G. Cummings, 1999. "Unbiased Value Estimates for Environmental Goods: A Cheap Talk Design for the Contingent Valuation Method," American Economic Review, American Economic Association, vol. 89(3), pages 649-665, June.
    14. DeShazo, J. R. & Fermo, German, 2002. "Designing Choice Sets for Stated Preference Methods: The Effects of Complexity on Choice Consistency," Journal of Environmental Economics and Management, Elsevier, vol. 44(1), pages 123-143, July.
    15. Louviere,Jordan J. & Hensher,David A. & Swait,Joffre D., 2000. "Stated Choice Methods," Cambridge Books, Cambridge University Press, number 9780521788304, September.
    16. David A. Hensher, 2004. "Identifying the Influence of Stated Choice Design Dimensionality on Willingness to Pay for Travel Time Savings," Journal of Transport Economics and Policy, University of Bath, vol. 38(3), pages 425-446, September.
    17. Heiner, Ronald A, 1983. "The Origin of Predictable Behavior," American Economic Review, American Economic Association, vol. 73(4), pages 560-595, September.
    18. Mandy Ryan & Sarah Wordsworth, 2000. "Sensitivity of Willingness to Pay Estimates to the Level of Attributes in Discrete Choice Experiments," Scottish Journal of Political Economy, Scottish Economic Society, vol. 47(5), pages 504-524, November.
    19. Nick Hanley & Robert Wright & Gary Koop, 2002. "Modelling Recreation Demand Using Choice Experiments: Climbing in Scotland," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 22(3), pages 449-466, July.
    20. Marisa J. Mazzotta & James J. Opaluch, 1995. "Decision Making When Choices Are Complex: A Test of Heiner's Hypothesis," Land Economics, University of Wisconsin Press, vol. 71(4), pages 500-515.
    21. Weenig, Mieneke W. H. & Maarleveld, Marleen, 2002. "The impact of time constraint on information search strategies in complex choice tasks," Journal of Economic Psychology, Elsevier, vol. 23(6), pages 689-702, December.
    22. repec:bla:scotjp:v:47:y:2000:i:5:p:504-24 is not listed on IDEAS
    23. Krinsky, Itzhak & Robb, A Leslie, 1986. "On Approximating the Statistical Properties of Elasticities," The Review of Economics and Statistics, MIT Press, vol. 68(4), pages 715-719, November.
    24. Ruby, Melissa C. & Johnson, F. Reed & Mathews, Kristy E., 1998. "Assessing Opt-Out Formats For Discrete-Choice Stated Preferences: Results From A Saltwater Angling Survey," 1998 Annual meeting, August 2-5, Salt Lake City, UT 20807, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    25. Ulla Slothuus Skjoldborg & Dorte Gyrd‐Hansen, 2003. "Conjoint analysis. The cost variable: an Achilles' heel?," Health Economics, John Wiley & Sons, Ltd., vol. 12(6), pages 479-491, June.
    26. Tara Maddala & Kathryn A. Phillips & F. Reed Johnson, 2003. "An experiment on simplifying conjoint analysis designs for measuring preferences," Health Economics, John Wiley & Sons, Ltd., vol. 12(12), pages 1035-1047, December.
    27. Stirling Bryan & Martin Buxton & Robert Sheldon & Alison Grant, 1998. "Magnetic resonance imaging for the investigation of knee injuries: an investigation of preferences," Health Economics, John Wiley & Sons, Ltd., vol. 7(7), pages 595-603, November.
    28. Louviere, Jordan J, 2001. "What If Consumer Experiments Impact Variances as Well as Means? Response Variability as a Behavioral Phenomenon," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 28(3), pages 506-511, December.
    29. Jeff Bennett & Russell Blamey (ed.), 2001. "The Choice Modelling Approach to Environmental Valuation," Books, Edward Elgar Publishing, number 2028.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jürgen Meyerhoff & Malte Oehlmann & Priska Weller, 2015. "The Influence of Design Dimensions on Stated Choices in an Environmental Context," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 61(3), pages 385-407, July.
    2. Rulleau, Bénédicte & Dachary-Bernard, Jeanne, 2012. "Preferences, rational choices and economic valuation: Some empirical tests," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 41(2), pages 198-206.
    3. Mandy Ryan & Mabelle Amaya‐Amaya, 2005. "‘Threats’ to and hopes for estimating benefits," Health Economics, John Wiley & Sons, Ltd., vol. 14(6), pages 609-619, June.
    4. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    5. Boxall, Peter C. & Adamowicz, Wiktor L. & Moon, Amanda, 2009. "Complexity in choice experiments: choice of the status quo alternative and implications for welfare measurement," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 53(4), pages 1-17.
    6. Haghani, Milad & Bliemer, Michiel C.J. & Hensher, David A., 2021. "The landscape of econometric discrete choice modelling research," Journal of choice modelling, Elsevier, vol. 40(C).
    7. Rolfe, John & Bennett, Jeff, 2009. "The impact of offering two versus three alternatives in choice modelling experiments," Ecological Economics, Elsevier, vol. 68(4), pages 1140-1148, February.
    8. Yang, Jui-Chen & Johnson, F. Reed & Kilambi, Vikram & Mohamed, Ateesha F., 2015. "Sample size and utility-difference precision in discrete-choice experiments: A meta-simulation approach," Journal of choice modelling, Elsevier, vol. 16(C), pages 50-57.
    9. Marit Kragt & Jeffrey Bennett, 2012. "Attribute Framing in Choice Experiments: How Do Attribute Level Descriptions Affect Value Estimates?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 51(1), pages 43-59, January.
    10. Caussade, Sebastián & Ortúzar, Juan de Dios & Rizzi, Luis I. & Hensher, David A., 2005. "Assessing the influence of design dimensions on stated choice experiment estimates," Transportation Research Part B: Methodological, Elsevier, vol. 39(7), pages 621-640, August.
    11. Ladenburg, Jacob & Olsen, Søren Bøye, 2014. "Augmenting short Cheap Talk scripts with a repeated Opt-Out Reminder in Choice Experiment surveys," Resource and Energy Economics, Elsevier, vol. 37(C), pages 39-63.
    12. Bujosa Bestard, Angel & Riera Font, Antoni, 2021. "Attribute range effects: Preference anomaly or unexplained variance?," Journal of choice modelling, Elsevier, vol. 41(C).
    13. Kosenius, Anna-Kaisa, 2013. "Preference discontinuity in choice experiment: Determinants and implications," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 45(C), pages 138-145.
    14. Moser, Riccarda & Raffaelli, Roberta, 2014. "Does attribute cut-off elicitation affect choice consistency? Contrasting hypothetical and real-money choice experiments," Journal of choice modelling, Elsevier, vol. 11(C), pages 16-29.
    15. Mandy Ryan & Karen Gerard & Gillian Currie, 2012. "Using Discrete Choice Experiments in Health Economics," Chapters, in: Andrew M. Jones (ed.), The Elgar Companion to Health Economics, Second Edition, chapter 41, Edward Elgar Publishing.
    16. Emily Lancsar & Jordan Louviere, 2008. "Conducting Discrete Choice Experiments to Inform Healthcare Decision Making," PharmacoEconomics, Springer, vol. 26(8), pages 661-677, August.
    17. Hoyos, David, 2010. "The state of the art of environmental valuation with discrete choice experiments," Ecological Economics, Elsevier, vol. 69(8), pages 1595-1603, June.
    18. Shiwen Quan & Yinchu Zeng & Xiaohua Yu & Te Bao, 2018. "WTP for baby milk formula in China: Using attribute nonattendance as a priori information to select attributes in choice experiment," Agribusiness, John Wiley & Sons, Ltd., vol. 34(2), pages 300-320, March.
    19. Scarpa, Riccardo & Rose, John M., 2008. "Design efficiency for non-market valuation with choice modelling: how to measure it, what to report and why," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 52(3), pages 1-30.
    20. Amilon, Anna & Ladenburg, Jacob & Siren, Anu & Vernstrøm Østergaard, Stine, 2020. "Willingness to pay for long-term home care services: Evidence from a stated preferences analysis," The Journal of the Economics of Ageing, Elsevier, vol. 17(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:hlthec:v:20:y:2011:i:3:p:273-286. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/5749 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.