IDEAS home Printed from https://ideas.repec.org/a/eee/transa/v46y2012i3p626-644.html
   My bibliography  Save this article

Not bored yet – Revisiting respondent fatigue in stated choice experiments

Author

Listed:
  • Hess, Stephane
  • Hensher, David A.
  • Daly, Andrew

Abstract

Stated choice surveys are used extensively in the study of choice behaviour across many different areas of research, notably in transport. One of their main characteristics in comparison with most types of revealed preference (RP) surveys is the ability to capture behaviour by the same respondent under varying choice scenarios. While this ability to capture multiple choices is generally seen as an advantage, there is a certain amount of unease about survey length. The precise definition about what constitutes a large number of choice tasks however varies across disciplines, and it is not uncommon to see surveys with up to twenty tasks per respondent in some areas. The argument against this practice has always been one of reducing respondent engagement, which could be interpreted as a result of fatigue or boredom, with frequent reference to the findings of Bradley and Daly (1994) who showed a significant drop in utility scale, i.e. an increase in error, as a respondent moved from one choice experiment to the next, an effect they related to respondent fatigue. While the work by Bradley and Daly has become a standard reference in this context, it should be recognised that not only was the fatigue part of the work based on a single dataset, but the state-of-the-art and the state-of-practice in stated choice survey design and implementation has moved on significantly since their study. In this paper, we review other literature and present a more comprehensive study investigating evidence of respondent fatigue across a larger number of different surveys. Using a comprehensive testing framework employing both Logit and mixed Logit structures, we provide strong evidence that the concerns about fatigue in the literature are possibly overstated, with no clear decreasing trend in scale across choice tasks in any of our studies. For the data sets tested, we find that accommodating any scale heterogeneity has little or no impact on substantive model results, that the role of constants generally decreases as the survey progresses, and that there is evidence of significant attribute level (as opposed to scale) heterogeneity across choice tasks.

Suggested Citation

  • Hess, Stephane & Hensher, David A. & Daly, Andrew, 2012. "Not bored yet – Revisiting respondent fatigue in stated choice experiments," Transportation Research Part A: Policy and Practice, Elsevier, vol. 46(3), pages 626-644.
    • Handle: RePEc:eee:transa:v:46:y:2012:i:3:p:626-644
    DOI: 10.1016/j.tra.2011.11.008
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0965856411001777
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.tra.2011.11.008?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Wiktor Adamowicz & Peter Boxall & Michael Williams & Jordan Louviere, 1998. "Stated Preference Approaches for Measuring Passive Use Values: Choice Experiments and Contingent Valuation," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 80(1), pages 64-75.
    2. Ladenburg, Jacob & Olsen, Søren Bøye, 2008. "Gender-specific starting point bias in choice experiments: Evidence from an empirical study," Journal of Environmental Economics and Management, Elsevier, vol. 56(3), pages 275-285, November.
    3. Roy Brouwer & Thijs Dekker & John Rolfe & Jill Windle, 2010. "Choice Certainty and Consistency in Repeated Choice Experiments," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 46(1), pages 93-109, May.
    4. Fosgerau, Mogens, 2006. "Investigating the distribution of the value of travel time savings," Transportation Research Part B: Methodological, Elsevier, vol. 40(8), pages 688-707, September.
    5. Nick Hanley & Robert Wright & Gary Koop, 2002. "Modelling Recreation Demand Using Choice Experiments: Climbing in Scotland," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 22(3), pages 449-466, July.
    6. Scott J. Savage & Donald M. Waldman, 2008. "Learning and fatigue during choice experiments: a comparison of online and mail survey modes," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(3), pages 351-371.
    7. Hu, Wuyang, 2006. "Effects of Endogenous Task Complexity and the Endowed Bundle on Stated Choice," 2006 Annual meeting, July 23-26, Long Beach, CA 21437, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    8. Risa Hole, Arne, 2004. "Forecasting the demand for an employee Park and Ride service using commuters' stated choices," Transport Policy, Elsevier, vol. 11(4), pages 355-362, October.
    9. Mickael Bech & Trine Kjaer & Jørgen Lauridsen, 2011. "Does the number of choice sets matter? Results from a web survey applying a discrete choice experiment," Health Economics, John Wiley & Sons, Ltd., vol. 20(3), pages 273-286, March.
    10. Bateman, Ian J. & Burgess, Diane & Hutchinson, W. George & Matthews, David I., 2008. "Learning design contingent valuation (LDCV): NOAA guidelines, preference learning and coherent arbitrariness," Journal of Environmental Economics and Management, Elsevier, vol. 55(2), pages 127-141, March.
    11. Caussade, Sebastián & Ortúzar, Juan de Dios & Rizzi, Luis I. & Hensher, David A., 2005. "Assessing the influence of design dimensions on stated choice experiment estimates," Transportation Research Part B: Methodological, Elsevier, vol. 39(7), pages 621-640, August.
    12. Guilkey, David K. & Murphy, James L., 1993. "Estimation and testing in the random effects probit model," Journal of Econometrics, Elsevier, vol. 59(3), pages 301-317, October.
    13. Thomas P. Holmes & Kevin J. Boyle, 2005. "Dynamic Learning and Context-Dependence in Sequential, Attribute-Based, Stated-Preference Valuation Questions," Land Economics, University of Wisconsin Press, vol. 81(1).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mikołaj Czajkowski & Marek Giergiczny & William H. Greene, 2012. "Learning and Fatigue Effects Revisited. The Impact of Accounting for Unobservable Preference and Scale Heterogeneity on Perceived Ordering Effects in Multiple Choice Task Discrete Choice Experiments," Working Papers 2012-08, Faculty of Economic Sciences, University of Warsaw.
    2. Thijs Dekker & Paul Koster & Roy Brouwer, 2014. "Changing with the Tide: Semiparametric Estimation of Preference Dynamics," Land Economics, University of Wisconsin Press, vol. 90(4), pages 717-745.
    3. Mikolaj Czajkowski & Marek Giergiczny & William H. Greene, 2014. "Learning and Fatigue Effects Revisited: Investigating the Effects of Accounting for Unobservable Preference and Scale Heterogeneity," Land Economics, University of Wisconsin Press, vol. 90(2), pages 324-351.
    4. Boxebeld, Sander, 2024. "Ordering effects in discrete choice experiments: A systematic literature review across domains," Journal of choice modelling, Elsevier, vol. 51(C).
    5. Ladenburg, Jacob & Olsen, Søren Bøye, 2014. "Augmenting short Cheap Talk scripts with a repeated Opt-Out Reminder in Choice Experiment surveys," Resource and Energy Economics, Elsevier, vol. 37(C), pages 39-63.
    6. Campbell, Danny & Boeri, Marco & Doherty, Edel & George Hutchinson, W., 2015. "Learning, fatigue and preference formation in discrete choice experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 119(C), pages 345-363.
    7. Jacobsen, Jette Bredahl & Thorsen, Bo Jellesmark, 2010. "Preferences for site and environmental functions when selecting forthcoming national parks," Ecological Economics, Elsevier, vol. 69(7), pages 1532-1544, May.
    8. Gabriela D. Oliveira & Luis C. Dias, 2020. "The potential learning effect of a MCDA approach on consumer preferences for alternative fuel vehicles," Annals of Operations Research, Springer, vol. 293(2), pages 767-787, October.
    9. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    10. Mamine, Fateh & Fares, M'hand & Minviel, Jean Joseph, 2020. "Contract Design for Adoption of Agrienvironmental Practices: A Meta-analysis of Discrete Choice Experiments," Ecological Economics, Elsevier, vol. 176(C).
    11. Day, Brett & Bateman, Ian J. & Carson, Richard T. & Dupont, Diane & Louviere, Jordan J. & Morimoto, Sanae & Scarpa, Riccardo & Wang, Paul, 2012. "Ordering effects and choice set awareness in repeat-response stated preference studies," Journal of Environmental Economics and Management, Elsevier, vol. 63(1), pages 73-91.
    12. Ben McNair & David Hensher & Jeff Bennett, 2012. "Modelling Heterogeneity in Response Behaviour Towards a Sequence of Discrete Choice Questions: A Probabilistic Decision Process Model," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 51(4), pages 599-616, April.
    13. Tomasz Gajderowicz & Gabriela Grotkowska, 2019. "Polarization of Tastes: Stated Preference Stability in Sequential Discrete Choices," European Research Studies Journal, European Research Studies Journal, vol. 0(4), pages 70-87.
    14. Tenaw G. Abate & Morten R. Mørkbak & Søren B. Olsen, 2018. "Inducing value and institutional learning effects in stated choice experiments using advanced disclosure and instructional choice set treatments," Agricultural Economics, International Association of Agricultural Economists, vol. 49(3), pages 339-351, May.
    15. McNair, Ben J. & Bennett, Jeff & Hensher, David A., 2011. "A comparison of responses to single and repeated discrete choice questions," Resource and Energy Economics, Elsevier, vol. 33(3), pages 554-571, September.
    16. McNair, Ben J. & Hensher, David A. & Bennett, Jeff, 2010. "Modelling heterogeneity in response behaviour towards a sequence of discrete choice questions: a latent class approach," MPRA Paper 23427, University Library of Munich, Germany.
    17. Nguyen, Thanh Cong & Le, Hoa Thu & Nguyen, Hang Dieu & Ngo, Mai Thanh & Nguyen, Hong Quang, 2021. "Examining ordering effects and strategic behaviour in a discrete choice experiment," Economic Analysis and Policy, Elsevier, vol. 70(C), pages 394-413.
    18. Carlsson, Fredrik & Raun Mørkbak, Morten & Bøye Olsen, Søren, 2010. "The first time is the hardest: A test of ordering effects in choice experiments," Working Papers in Economics 470, University of Gothenburg, Department of Economics.
    19. Svenningsen, Lea S. & Jacobsen, Jette Bredahl, 2018. "Testing the effect of changes in elicitation format, payment vehicle and bid range on the hypothetical bias for moral goods," Journal of choice modelling, Elsevier, vol. 29(C), pages 17-32.
    20. Mikołaj Czajkowski & Nick Hanley & Jacob LaRiviere, 2016. "Controlling for the Effects of Information in a Public Goods Discrete Choice Model," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 63(3), pages 523-544, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:transa:v:46:y:2012:i:3:p:626-644. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/547/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.