IDEAS home Printed from https://ideas.repec.org/a/eee/pubeco/v234y2024ics004727272400032x.html
   My bibliography  Save this article

Understanding and increasing policymakers’ sensitivity to program impact

Author

Listed:
  • Toma, Mattie
  • Bell, Elizabeth

Abstract

Policymakers routinely make high-stakes funding decisions. Assessing the value of a program is difficult and may be affected by bounded rationality. We conducted experiments involving U.S. policymakers and the general public, in which participants were given the opportunity to assess the value of various policy programs. Our findings demonstrate that decision aids enhance the responsiveness of respondents to the impact of the programs. We designed and tested two portable decision aids—one that compares programs side-by-side and another that aggregates multiple features of impact into a single metric. The two decision aids increase the elasticity of assessments of program value with respect to impact by 0.20 on a base of 0.33 among policymakers and by 0.21 on a base of 0.21 among the general public. We provide evidence that the cognitive difficulty of translating impact-relevant information into policy decisions helps explain our findings.

Suggested Citation

  • Toma, Mattie & Bell, Elizabeth, 2024. "Understanding and increasing policymakers’ sensitivity to program impact," Journal of Public Economics, Elsevier, vol. 234(C).
    • Handle: RePEc:eee:pubeco:v:234:y:2024:i:c:s004727272400032x
    DOI: 10.1016/j.jpubeco.2024.105096
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S004727272400032X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jpubeco.2024.105096?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jonas Hjort & Diana Moreira & Gautam Rao & Juan Francisco Santini, 2021. "How Research Affects Policy: Experimental Evidence from 2,150 Brazilian Municipalities," American Economic Review, American Economic Association, vol. 111(5), pages 1442-1480, May.
    2. Justin Holz & Rafael Jiménez-Durán & Eduardo Laguna-Müggenburg, 2024. "Estimating the Distaste for Price Gouging with Incentivized Consumer Reports," American Economic Journal: Applied Economics, American Economic Association, vol. 16(1), pages 33-59, January.
    3. Stefano DellaVigna & Woojin Kim & Elizabeth Linos, 2024. "Bottlenecks for Evidence Adoption," Journal of Political Economy, University of Chicago Press, vol. 132(8), pages 2748-2789.
    4. Carlos Alós-Ferrer & Ernst Fehr & Nick Netzer, 2021. "Time Will Tell: Recovering Preferences When Choices Are Noisy," Journal of Political Economy, University of Chicago Press, vol. 129(6), pages 1828-1877.
    5. Stefano DellaVigna & Elizabeth Linos, 2022. "RCTs to Scale: Comprehensive Evidence From Two Nudge Units," Econometrica, Econometric Society, vol. 90(1), pages 81-116, January.
    6. Dmitry Taubinsky & Alex Rees-Jones, 2018. "Attention Variation and Welfare: Theory and Evidence from a Tax Salience Experiment," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 85(4), pages 2462-2496.
    7. Dan M. Kahan & Ellen Peters & Maggie Wittlin & Paul Slovic & Lisa Larrimore Ouellette & Donald Braman & Gregory Mandel, 2012. "The polarizing impact of science literacy and numeracy on perceived climate change risks," Nature Climate Change, Nature, vol. 2(10), pages 732-735, October.
    8. Marcelo Bergolo & Rodrigo Ceni & Guillermo Cruces & Matias Giaccobasso & Ricardo Perez-Truglia, 2023. "Tax Audits as Scarecrows: Evidence from a Large-Scale Field Experiment," American Economic Journal: Economic Policy, American Economic Association, vol. 15(1), pages 110-153, February.
    9. Alberto Cavallo & Guillermo Cruces & Ricardo Perez-Truglia, 2017. "Inflation Expectations, Learning, and Supermarket Prices: Evidence from Survey Experiments," American Economic Journal: Macroeconomics, American Economic Association, vol. 9(3), pages 1-35, July.
    10. Sheheryar Banuri & Stefan Dercon & Varun Gauri, 2019. "Biased Policy Professionals," The World Bank Economic Review, World Bank, vol. 33(2), pages 310-327.
    11. Herbert A. Simon, 1955. "A Behavioral Model of Rational Choice," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 69(1), pages 99-118.
    12. Dur, Robert & Vollaard, Ben, 2019. "Salience of law enforcement: A field experiment," Journal of Environmental Economics and Management, Elsevier, vol. 93(C), pages 208-220.
    13. Erin L. Krupka & Roberto A. Weber, 2013. "Identifying Social Norms Using Coordination Games: Why Does Dictator Game Sharing Vary?," Journal of the European Economic Association, European Economic Association, vol. 11(3), pages 495-524, June.
    14. Omar Al-Ubaydli & John A. List & Dana L. Suskind, 2017. "What Can We Learn from Experiments? Understanding the Threats to the Scalability of Experimental Results," American Economic Review, American Economic Association, vol. 107(5), pages 282-286, May.
    15. Chris Frost & Simon G. Thompson, 2000. "Correcting for regression dilution bias: comparison of methods for a single predictor variable," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 163(2), pages 173-189.
    16. Raj Chetty & Emmanuel Saez, 2013. "Teaching the Tax Code: Earnings Responses to an Experiment with EITC Recipients," American Economic Journal: Applied Economics, American Economic Association, vol. 5(1), pages 1-31, January.
    17. Bergman, Peter & Lasky-Fink, Jessica & Rogers, Todd, 2020. "Simplification and defaults affect adoption and impact of technology, but decision makers do not realize it," Organizational Behavior and Human Decision Processes, Elsevier, vol. 158(C), pages 66-79.
    18. Kahn, Matthew & Wolak, Frank, 2013. "Using Information to Improve the Effectiveness of Nonlinear Pricing: Evidence from a Field Experiment," MPRA Paper 106089, University Library of Munich, Germany.
    19. Kahneman, Daniel & Knetsch, Jack L., 1992. "Valuing public goods: The purchase of moral satisfaction," Journal of Environmental Economics and Management, Elsevier, vol. 22(1), pages 57-70, January.
    20. Luchini, S. & Watson, V., 2014. "Are choice experiments reliable? Evidence from the lab," Economics Letters, Elsevier, vol. 124(1), pages 9-13.
    21. Iris Bohnet & Alexandra van Geen & Max Bazerman, 2016. "When Performance Trumps Gender Bias: Joint vs. Separate Evaluation," Management Science, INFORMS, vol. 62(5), pages 1225-1234, May.
    22. World Bank Group, 2015. "World Development Report 2015 [Informe sobre el desarrollo mundial 2015 : mente, sociedad y conducta - panorama general]," World Bank Publications - Books, The World Bank Group, number 20597.
    23. Small, Deborah A. & Loewenstein, George & Slovic, Paul, 2007. "Sympathy and callousness: The impact of deliberative thought on donations to identifiable and statistical victims," Organizational Behavior and Human Decision Processes, Elsevier, vol. 102(2), pages 143-153, March.
    24. David Danz & Lise Vesterlund & Alistair J. Wilson, 2022. "Belief Elicitation and Behavioral Incentive Compatibility," American Economic Review, American Economic Association, vol. 112(9), pages 2851-2883, September.
    25. Jerry Hausman, 2012. "Contingent Valuation: From Dubious to Hopeless," Journal of Economic Perspectives, American Economic Association, vol. 26(4), pages 43-56, Fall.
    26. Richard Hendra & Aaron Hill, 2019. "Rethinking Response Rates: New Evidence of Little Relationship Between Survey Response Rates and Nonresponse Bias," Evaluation Review, , vol. 43(5), pages 307-330, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Michael Thaler & Mattie Toma & Victor Yaneng Wang, 2024. "Numbers Tell, Words Sell," CESifo Working Paper Series 11600, CESifo.
    2. Guglielmo Briscese & John A. List, 2024. "Toward an Understanding of the Political Economy of Using Field Experiments in Policymaking," NBER Working Papers 33239, National Bureau of Economic Research, Inc.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ingar Haaland & Christopher Roth & Johannes Wohlfart, 2023. "Designing Information Provision Experiments," Journal of Economic Literature, American Economic Association, vol. 61(1), pages 3-40, March.
    2. Michael Thaler & Mattie Toma & Victor Yaneng Wang, 2024. "Numbers Tell, Words Sell," CESifo Working Paper Series 11600, CESifo.
    3. Libor Dušek & Nicolas Pardo & Christian Traxler, 2022. "Salience and Timely Compliance: Evidence from Speeding Tickets," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 41(2), pages 426-449, March.
    4. Antinyan, Armenak & Asatryan, Zareh, 2019. "Nudging for tax compliance: A meta-analysis," ZEW Discussion Papers 19-055, ZEW - Leibniz Centre for European Economic Research.
    5. Guglielmo Briscese & John List, 2024. "Toward an Understanding of the Political Economy of Using Field Experiments in Policymaking," Natural Field Experiments 00799, The Field Experiments Website.
    6. Aycinena, Diego & Bogliacino, Francesco & Kimbrough, Erik O., 2024. "Measuring norms: Assessing the threat of social desirability bias to the Bicchieri and Xiao elicitation method," Journal of Economic Behavior & Organization, Elsevier, vol. 222(C), pages 225-239.
    7. Bartosz Maćkowiak & Filip Matějka & Mirko Wiederholt, 2023. "Rational Inattention: A Review," Journal of Economic Literature, American Economic Association, vol. 61(1), pages 226-273, March.
    8. Edward L. Glaeser & Scott Duke Kominers & Michael Luca & Nikhil Naik, 2018. "Big Data And Big Cities: The Promises And Limitations Of Improved Measures Of Urban Life," Economic Inquiry, Western Economic Association International, vol. 56(1), pages 114-137, January.
    9. Desbureaux, Sébastien & Brimont, Laura, 2015. "Between economic loss and social identity: The multi-dimensional cost of avoiding deforestation in Eastern Madagascar," Ecological Economics, Elsevier, vol. 118(C), pages 10-20.
    10. Clemens, Michael A. & Pritchett, Lant, 2019. "The new economic case for migration restrictions: An assessment," Journal of Development Economics, Elsevier, vol. 138(C), pages 153-164.
    11. Jan-Emmanuel De Neve & Clément Imbert & Johannes Spinnewijn & Teodora Tsankova & Maarten Luts, 2021. "How to Improve Tax Compliance? Evidence from Population-Wide Experiments in Belgium," Journal of Political Economy, University of Chicago Press, vol. 129(5), pages 1425-1463.
    12. Andrew Dustan & Stanislao Maldonado & Juan Manuel Hernandez-Agramonte, 2018. "Motivating bureaucrats with non-monetary incentives when state capacity is weak: Evidence from large-scale field experiments in Peru," Working Papers 136, Peruvian Economic Association.
    13. Ryen, Linda & Svensson, Mikael, 2014. "The Willingness to Pay for a QALY: a Review of the Empirical Literature," Karlstad University Working Papers in Economics 12, Karlstad University, Department of Economics.
    14. Szabó, Zoltán, 2011. "Reducing protest responses by deliberative monetary valuation: Improving the validity of biodiversity valuation," Ecological Economics, Elsevier, vol. 72(C), pages 37-44.
    15. Desvousges, William H. & Gard, Nicholas & Michael, Holly J. & Chance, Anne D., 2018. "Habitat and Resource Equivalency Analysis: A Critical Assessment," Ecological Economics, Elsevier, vol. 143(C), pages 74-89.
    16. Despoina Alempaki & Genyue Fu & Jingcheng Fu, 2021. "Lying and social norms: a lab-in-the-field experiment with children," Discussion Papers 2021-01, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.
    17. repec:cup:judgdm:v:8:y:2013:i:4:p:397-406 is not listed on IDEAS
    18. Hasford, Jonathan & Farmer, Adam & Waites, Stacie F., 2015. "Thinking, feeling, and giving: The effects of scope and valuation on consumer donations," International Journal of Research in Marketing, Elsevier, vol. 32(4), pages 435-438.
    19. Anders Dugstad & Kristine Grimsrud & Gorm Kipperberg & Henrik Lindhjem & Ståle Navrud, 2020. "Scope elasticity and economic significance in discrete choice experiments," Discussion Papers 942, Statistics Norway, Research Department.
    20. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    21. Rebekka Kesberg & Stefan Pfattheicher, 2019. "Democracy matters: a psychological perspective on the beneficial impact of democratic punishment systems in social dilemmas," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-13, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:pubeco:v:234:y:2024:i:c:s004727272400032x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/505578 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.