IDEAS home Printed from https://ideas.repec.org/a/kap/enreec/v84y2023i4d10.1007_s10640-022-00742-6.html
   My bibliography  Save this article

Adjusting and Calibrating Elicited Values Based on Follow-up Certainty Questions: A Meta-analysis

Author

Listed:
  • Jerrod Penn

    (Louisiana State University & LSU Agricultural Center)

  • Wuyang Hu

    (The Ohio State University)

Abstract

Researchers have proposed many methods to reduce hypothetical bias (HB) in stated preference studies. One of the earliest and most popular is Certainty follow-up, in which the respondent states how sure they are of their response to the valuation question they just responded to. Certainty follow-up enables the use of several cutoffs to calibrate for HB, whereas the efficacy of other popular HB mitigation methods, such as Cheap Talk, have no such flexibility. Even given a cutoff level, its ability to reduce HB may vary with characteristics of the Certainty follow-up and of the study. Using a meta-analysis, we find that Certainty follow-up is more effective than Cheap Talk at adjusting for potential HB and that value elicitation method, mode of data collection, as well as whether other HB mitigation methods are used in a study could affect Certainty follow-up efficacy. Using and recoding Certainty follow-up questions quantitatively or qualitatively can be equally effective when compared to unadjusted hypothetical values where potential HB may occur or to values elicited with real binding conditions in which the actual magnitude of the HB is known. There is strong evidence that HB can be completely calibrated for or even overcorrected, but we encourage more Certainty follow-up studies with binding elicitations to fully explore the potential of this method.

Suggested Citation

  • Jerrod Penn & Wuyang Hu, 2023. "Adjusting and Calibrating Elicited Values Based on Follow-up Certainty Questions: A Meta-analysis," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 84(4), pages 919-946, April.
    • Handle: RePEc:kap:enreec:v:84:y:2023:i:4:d:10.1007_s10640-022-00742-6
    DOI: 10.1007/s10640-022-00742-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10640-022-00742-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10640-022-00742-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Craig D. Broadbent, 2014. "Evaluating mitigation and calibration techniques for hypothetical bias in choice experiments," Journal of Environmental Planning and Management, Taylor & Francis Journals, vol. 57(12), pages 1831-1848, December.
    2. Johannesson, Magnus & Johansson, Per-Olov & Kristrom, Bengt & Gerdtham, Ulf-G., 1993. "Willingness to pay for antihypertensive therapy -- further results," Journal of Health Economics, Elsevier, vol. 12(1), pages 95-108, April.
    3. James Murphy & P. Allen & Thomas Stevens & Darryl Weatherhead, 2005. "A Meta-analysis of Hypothetical Bias in Stated Preference Valuation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 30(3), pages 313-325, March.
    4. Tunçel, Tuba & Hammitt, James K., 2014. "A new meta-analysis on the WTP/WTA disparity," Journal of Environmental Economics and Management, Elsevier, vol. 68(1), pages 175-187.
    5. Yohei Mitani & Nicholas Flores, 2014. "Hypothetical Bias Reconsidered: Payment and Provision Uncertainties in a Threshold Provision Mechanism," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 59(3), pages 433-454, November.
    6. Jerrod M Penn & Wuyang Hu, 2018. "Understanding Hypothetical Bias: An Enhanced Meta-Analysis," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 100(4), pages 1186-1206.
    7. Champ, Patricia A. & Alberini, Anna & Correas, Ignacio, 2005. "Using contingent valuation to value a noxious weeds control program: the effects of including an unsure response category," Ecological Economics, Elsevier, vol. 55(1), pages 47-60, October.
    8. Matteo Mattmann & Ivana Logar & Roy Brouwer, 2019. "Choice certainty, consistency, and monotonicity in discrete choice experiments," Journal of Environmental Economics and Policy, Taylor & Francis Journals, vol. 8(2), pages 109-127, April.
    9. Richard C. Ready & Ståle Navrud & RW. Richard Dubourg, 2001. "How Do Respondents with Uncertain Willingness to Pay Answer Contingent Valuation Questions?," Land Economics, University of Wisconsin Press, vol. 77(3), pages 315-326.
    10. Champ, Patricia A. & Moore, Rebecca & Bishop, Richard C., 2009. "A Comparison of Approaches to Mitigate Hypothetical Bias," Agricultural and Resource Economics Review, Cambridge University Press, vol. 38(2), pages 166-180, October.
    11. Christian A. Vossler & Robert G. Ethier & Gregory L. Poe & Michael P. Welsh, 2003. "Payment Certainty in Discrete Choice Contingent Valuation Responses: Results from a Field Validity Test," Southern Economic Journal, John Wiley & Sons, vol. 69(4), pages 886-902, April.
    12. Dan Ariely & George Loewenstein & Drazen Prelec, 2003. ""Coherent Arbitrariness": Stable Demand Curves Without Stable Preferences," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(1), pages 73-106.
    13. Akter, Sonia & Brouwer, Roy & Brander, Luke & van Beukering, Pieter, 2009. "Respondent uncertainty in a contingent market for carbon offsets," Ecological Economics, Elsevier, vol. 68(6), pages 1858-1863, April.
    14. Nick Hanley & Bengt Kriström & Jason F. Shogren, 2009. "Coherent Arbitrariness: On Value Uncertainty for Environmental Goods," Land Economics, University of Wisconsin Press, vol. 85(1), pages 41-50.
    15. Champ, Patricia A. & Bishop, Richard C. & Brown, Thomas C. & McCollum, Daniel W., 1997. "Using Donation Mechanisms to Value Nonuse Benefits from Public Goods," Journal of Environmental Economics and Management, Elsevier, vol. 33(2), pages 151-162, June.
    16. Rhodes, R. J. & Whitehead, J. C. & Smith, T. I. J. & Denson, M. R., 2018. "A benefit-cost analysis of a red drum stock enhancement program in South Carolina," Journal of Benefit-Cost Analysis, Cambridge University Press, vol. 9(2), pages 323-341, July.
    17. Fifer, Simon & Rose, John M., 2016. "Can you ever be certain? Reducing hypothetical bias in stated choice experiments via respondent reported choice certaintyAuthor-Name: Beck, Matthew J," Transportation Research Part B: Methodological, Elsevier, vol. 89(C), pages 149-167.
    18. Richard C. Ready & Patricia A. Champ & Jennifer L. Lawton, 2010. "Using Respondent Uncertainty to Mitigate Hypothetical Bias in a Stated Choice Experiment," Land Economics, University of Wisconsin Press, vol. 86(2), pages 363-381.
    19. Pierre-Alexandre Mahieu & Pere Riera & Bengt Kriström & Runar Brännlund & Marek Giergiczny, 2014. "Exploring the determinants of uncertainty in contingent valuation surveys," Journal of Environmental Economics and Policy, Taylor & Francis Journals, vol. 3(2), pages 186-200, July.
    20. Johannesson, Magnus & Blomquist, Glenn C. & Blumenschein, Karen & Johansson, Per-olov & Liljas, Bengt & O'Conor, Richard M., 1999. "Calibrating Hypothetical Willingness to Pay Responses," Journal of Risk and Uncertainty, Springer, vol. 18(1), pages 21-32, April.
    21. Karen Blumenschein & Magnus Johannesson & Glenn C. Blomquist & Bengt Liljas & Richard M. O'Conor, 1998. "Experimental Results on Expressed Certainty and Hypothetical Bias in Contingent Valuation," Southern Economic Journal, John Wiley & Sons, vol. 65(1), pages 169-177, July.
    22. Roe, Brian & Boyle, Kevin J. & Teisl, Mario F., 1996. "Using Conjoint Analysis to Derive Estimates of Compensating Variation," Journal of Environmental Economics and Management, Elsevier, vol. 31(2), pages 145-159, September.
    23. Jerrod M. Penn & Wuyang Hu, 2021. "The Extent of Hypothetical Bias in Willingness to Accept," American Journal of Agricultural Economics, John Wiley & Sons, vol. 103(1), pages 126-141, January.
    24. Welsh, Michael P. & Poe, Gregory L., 1998. "Elicitation Effects in Contingent Valuation: Comparisons to a Multiple Bounded Discrete Choice Approach," Journal of Environmental Economics and Management, Elsevier, vol. 36(2), pages 170-185, September.
    25. Christian A. Vossler & Robert G. Ethier & Gregory L. Poe & Michael P. Welsh, 2003. "Payment Certainty in Discrete Choice Contingent Valuation Responses: Results from a Field Validity Test," Southern Economic Journal, John Wiley & Sons, vol. 69(4), pages 886-902, April.
    26. Li Chuan-Zhong & Mattsson Leif, 1995. "Discrete Choice under Preference Uncertainty: An Improved Structural Model for Contingent Valuation," Journal of Environmental Economics and Management, Elsevier, vol. 28(2), pages 256-269, March.
    27. Karen Blumenschein & Magnus Johannesson & Glenn C. Blomquist & Bengt Liljas & Richard M. O’Conor, 1998. "Experimental Results on Expressed Certainty and Hypothetical Bias in Contingent Valuation," Southern Economic Journal, John Wiley & Sons, vol. 65(1), pages 169-177, July.
    28. Roy Brouwer & Thijs Dekker & John Rolfe & Jill Windle, 2010. "Choice Certainty and Consistency in Repeated Choice Experiments," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 46(1), pages 93-109, May.
    29. Glenn Blomquist & Karen Blumenschein & Magnus Johannesson, 2009. "Eliciting Willingness to Pay without Bias using Follow-up Certainty Statements: Comparisons between Probably/Definitely and a 10-point Certainty Scale," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 43(4), pages 473-502, August.
    30. Jonas Schmidt & Tammo H. A. Bijmolt, 2020. "Accurately measuring willingness to pay for consumer goods: a meta-analysis of the hypothetical bias," Journal of the Academy of Marketing Science, Springer, vol. 48(3), pages 499-518, May.
    31. John List & Craig Gallet, 2001. "What Experimental Protocol Influence Disparities Between Actual and Hypothetical Stated Values?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 20(3), pages 241-254, November.
    32. Gregory Poe & Jeremy Clark & Daniel Rondeau & William Schulze, 2002. "Provision Point Mechanisms and Field Validity Tests of Contingent Valuation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 23(1), pages 105-131, September.
    33. Champ, Patricia A. & Moore, Rebecca & Bishop, Richard C., 2009. "A Comparison of Approaches to Mitigate Hypothetical Bias," Agricultural and Resource Economics Review, Northeastern Agricultural and Resource Economics Association, vol. 38(2), pages 1-15, October.
    34. Blomquist, Glenn C. & Dickie, Mark & O'Conor, Richard M., 2011. "Willingness to pay for improving fatality risks and asthma symptoms: Values for children and adults of all ages," Resource and Energy Economics, Elsevier, vol. 33(2), pages 410-425, May.
    35. Penn, Jerrod & Hu, Wuyang, 2019. "Cheap talk efficacy under potential and actual Hypothetical Bias: A meta-analysis," Journal of Environmental Economics and Management, Elsevier, vol. 96(C), pages 22-35.
    36. Shaikh, Sabina L. & Sun, Lili & Cornelis van Kooten, G., 2007. "Treating respondent uncertainty in contingent valuation: A comparison of empirical treatments," Ecological Economics, Elsevier, vol. 62(1), pages 115-125, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Paul Hindley & O. Ashton Morgan, 2023. "The Role of Respondent Certainty and Attribute Non-Attendance on the Willingness to Pay for the Attributes of Recyclable Aluminum Bottled Water," Working Papers 23-06, Department of Economics, Appalachian State University.
    2. Jerrod M. Penn & Daniel R. Petrolia & J. Matthew Fannin, 2023. "Hypothetical bias mitigation in representative and convenience samples," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 45(2), pages 721-743, June.
    3. Vasudha Chopra & Christian A. Vossler, 2024. "Are we doing more harm than good? Hypothetical bias reduction techniques in potentially consequential survey settings," Working Papers 2024-03, University of Tennessee, Department of Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Haghani, Milad & Bliemer, Michiel C.J. & Rose, John M. & Oppewal, Harmen & Lancsar, Emily, 2021. "Hypothetical bias in stated choice experiments: Part II. Conceptualisation of external validity, sources and explanations of bias and effectiveness of mitigation methods," Journal of choice modelling, Elsevier, vol. 41(C).
    2. Ana Bobinac, 2019. "Mitigating hypothetical bias in willingness to pay studies: post-estimation uncertainty and anchoring on irrelevant information," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(1), pages 75-82, February.
    3. Voltaire, Louinord & Pirrone, Claudio & Bailly, Denis, 2013. "Dealing with preference uncertainty in contingent willingness to pay for a nature protection program: A new approach," Ecological Economics, Elsevier, vol. 88(C), pages 76-85.
    4. Sergio Colombo & Wiktor Budziński & Mikołaj Czajkowski & Klaus Glenk, 2022. "The relative performance of ex‐ante and ex‐post measures to mitigate hypothetical and strategic bias in a stated preference study," Journal of Agricultural Economics, Wiley Blackwell, vol. 73(3), pages 845-873, September.
    5. Atozou, Baoubadi & Tamini, Lota D. & Bergeronm, Stephane & Doyon, Maurice, 2020. "Factors Explaining the Hypothetical Bias: How to Improve Models for Meta-Analyses," Journal of Agricultural and Resource Economics, Western Agricultural Economics Association, vol. 45(2), March.
    6. Karen Blumenschein & GlennC. Blomquist & Magnus Johannesson & Nancy Horn & Patricia Freeman, 2008. "Eliciting Willingness to Pay Without Bias: Evidence from a Field Experiment," Economic Journal, Royal Economic Society, vol. 118(525), pages 114-137, January.
    7. Jiang, Qi & Penn, Jerrod & Hu, Wuyang, 2022. "Real payment priming to reduce potential hypothetical bias," Journal of choice modelling, Elsevier, vol. 45(C).
    8. Milad Haghani & Michiel C. J. Bliemer & John M. Rose & Harmen Oppewal & Emily Lancsar, 2021. "Hypothetical bias in stated choice experiments: Part II. Macro-scale analysis of literature and effectiveness of bias mitigation methods," Papers 2102.02945, arXiv.org.
    9. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    10. Sund, Björn, 2009. "Certainty calibration in contingent valuation - exploring the within-difference between dichotomous choice and open-ended answers as a certainty measure," Working Papers 2009:1, Örebro University, School of Business.
    11. Nikita Lyssenko & Roberto Mart󹑺-Espiñeira, 2012. "Respondent uncertainty in contingent valuation: the case of whale conservation in Newfoundland and Labrador," Applied Economics, Taylor & Francis Journals, vol. 44(15), pages 1911-1930, May.
    12. Richard C. Ready & Patricia A. Champ & Jennifer L. Lawton, 2010. "Using Respondent Uncertainty to Mitigate Hypothetical Bias in a Stated Choice Experiment," Land Economics, University of Wisconsin Press, vol. 86(2), pages 363-381.
    13. Sergio Colombo & Wiktor Budziński & Mikołaj Czajkowski & Klaus Glenk, 2020. "Ex-ante and ex-post measures to mitigate hypothetical bias. Are they alternative or complementary tools to increase the reliability and validity of DCE estimates?," Working Papers 2020-20, Faculty of Economic Sciences, University of Warsaw.
    14. Fifer, Simon & Rose, John M., 2016. "Can you ever be certain? Reducing hypothetical bias in stated choice experiments via respondent reported choice certaintyAuthor-Name: Beck, Matthew J," Transportation Research Part B: Methodological, Elsevier, vol. 89(C), pages 149-167.
    15. Rebecca Moore & Richard C. Bishop & Bill Provencher & Patricia A. Champ, 2010. "Accounting for Respondent Uncertainty to Improve Willingness‐to‐Pay Estimates," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 58(3), pages 381-401, September.
    16. Thunström, Linda & Nordström, Jonas & Shogren, Jason F., 2015. "Certainty and overconfidence in future preferences for food," Journal of Economic Psychology, Elsevier, vol. 51(C), pages 101-113.
    17. Uggeldahl, Kennet & Jacobsen, Catrine & Lundhede, Thomas Hedemark & Olsen, Søren Bøye, 2016. "Choice certainty in Discrete Choice Experiments: Will eye tracking provide useful measures?," Journal of choice modelling, Elsevier, vol. 20(C), pages 35-48.
    18. Moore, Rebecca & Bishop, Richard C. & Provencher, Bill & Champ, Patricia A., 2009. "Accounting for Respondent Uncertainty to Improve Willingness-to-Pay Estimates," Staff Papers 92233, University of Wisconsin-Madison, Department of Agricultural and Applied Economics.
    19. Jinkwon Lee & Uk Hwang, 2016. "Hypothetical Bias in Risk Preferences as a Driver of Hypothetical Bias in Willingness to Pay: Experimental Evidence," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 65(4), pages 789-811, December.
    20. Penn, Jerrod & Hu, Wuyang & Ye, Tao, 2024. "Efficacy of hypothetical bias mitigation techniques: A cross-country comparison," Journal of Environmental Economics and Management, Elsevier, vol. 125(C).

    More about this item

    Keywords

    Certainty follow-up; Hypothetical bias; Meta-analysis;
    All these keywords.

    JEL classification:

    • Q51 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Environmental Economics - - - Valuation of Environmental Effects
    • Q10 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Agriculture - - - General
    • D90 - Microeconomics - - Micro-Based Behavioral Economics - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:enreec:v:84:y:2023:i:4:d:10.1007_s10640-022-00742-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.