IDEAS home Printed from https://ideas.repec.org/a/bla/jageco/v74y2023i3p661-685.html
   My bibliography  Save this article

External validity of economic experiments on Agri‐environmental scheme design

Author

Listed:
  • Marie Ferré
  • Stefanie Engel
  • Elisabeth Gsottbauer

Abstract

The use of laboratory experiments to study issues in agricultural policy has grown in prominence within the fields of agricultural and environmental economics. Such experiments are often conducted with university students and framed in an abstract manner. This raises questions about whether the findings of these experiments provide reliable insights on the behaviour of actual agents in real settings. We contribute to this methodological debate by analysing the impacts of sample population and framing on behaviour in the experiment and on two policy effects: the direction and the magnitude of the policy impact. We also examine the channels through which differences in results may occur. For this, we test if behaviour is correlated with a set of covariates collected from our samples, including socio‐demographics, social and risk preferences. Our main finding is that the type of subject significantly affects the magnitude of the policy impact. The two populations differ substantially in the representation of key characteristics and preferences, which in consequence affects behaviour in the experiment. We find no significant impact of framing on behaviour.

Suggested Citation

  • Marie Ferré & Stefanie Engel & Elisabeth Gsottbauer, 2023. "External validity of economic experiments on Agri‐environmental scheme design," Journal of Agricultural Economics, Wiley Blackwell, vol. 74(3), pages 661-685, September.
  • Handle: RePEc:bla:jageco:v:74:y:2023:i:3:p:661-685
    DOI: 10.1111/1477-9552.12529
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/1477-9552.12529
    Download Restriction: no

    File URL: https://libkey.io/10.1111/1477-9552.12529?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Luc Behaghel & Karen Macours & Julie Subervie, 2019. "How can randomised controlled trials help improve the design of the common agricultural policy?," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 46(3), pages 473-493.
    2. Jeffrey Carpenter & Erika Seki, 2011. "Do Social Preferences Increase Productivity? Field Experimental Evidence From Fishermen In Toyama Bay," Economic Inquiry, Western Economic Association International, vol. 49(2), pages 612-630, April.
    3. Steven Schilizzi & Uwe Latacz-Lohmann, 2007. "Assessing the Performance of Conservation Auctions: An Experimental Study," Land Economics, University of Wisconsin Press, vol. 83(4), pages 497-515.
    4. Cason, Timothy N. & Gangadharan, Lata, 2006. "Emissions variability in tradable permit markets with imperfect enforcement and banking," Journal of Economic Behavior & Organization, Elsevier, vol. 61(2), pages 199-216, October.
    5. Sophie Thoyer & Raphaële Préget, 2019. "Enriching the CAP evaluation toolbox with experimental approaches: introduction to the special issue," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 46(3), pages 347-366.
    6. Bellemare, Charles & Kroger, Sabine, 2007. "On representative social capital," European Economic Review, Elsevier, vol. 51(1), pages 183-202, January.
    7. Parkhurst, Gregory M. & Shogren, Jason F., 2007. "Spatial incentives to coordinate contiguous habitat," Ecological Economics, Elsevier, vol. 64(2), pages 344-355, December.
    8. Warziniack, Travis & Shogren, Jason F. & Parkhurst, Gregory, 2007. "Creating contiguous forest habitat: An experimental examination on incentives and communication," Journal of Forest Economics, Elsevier, vol. 13(2-3), pages 191-207, August.
    9. Schilizzi, Steven & Latacz-Lohmann, Uwe, 2007. "Assessing the performance of conservation auctions: an experimental study," 2007 Conference (51st), February 13-16, 2007, Queenstown, New Zealand 10436, Australian Agricultural and Resource Economics Society.
    10. Simanti Banerjee & Frans P. de Vries & Nick Hanley & Daan P. van Soest, 2014. "The Impact of Information Provision on Agglomeration Bonus Performance: An Experimental Study on Local Networks," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 96(4), pages 1009-1029.
    11. Arthur Schram, 2005. "Artificiality: The tension between internal and external validity in economic experiments," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(2), pages 225-237.
    12. Brick, Kerri & Visser, Martine, 2015. "Risk preferences, technology adoption and insurance uptake: A framed experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 118(C), pages 383-396.
    13. Tanjim Hossain & John A. List, 2012. "The Behavioralist Visits the Factory: Increasing Productivity Using Simple Framing Manipulations," Management Science, INFORMS, vol. 58(12), pages 2151-2167, December.
    14. James Alm & Kim M. Bloomquist & Michael McKee, 2015. "On The External Validity Of Laboratory Tax Compliance Experiments," Economic Inquiry, Western Economic Association International, vol. 53(2), pages 1170-1186, April.
    15. Banerjee, Simanti & Cason, Timothy N. & de Vries, Frans P. & Hanley, Nick, 2017. "Transaction costs, communication and spatial coordination in Payment for Ecosystem Services Schemes," Journal of Environmental Economics and Management, Elsevier, vol. 83(C), pages 68-89.
    16. Alekseev, Aleksandr & Charness, Gary & Gneezy, Uri, 2017. "Experimental methods: When and why contextual instructions are important," Journal of Economic Behavior & Organization, Elsevier, vol. 134(C), pages 48-59.
    17. Jon Anderson & Stephen Burks & Jeffrey Carpenter & Lorenz Götte & Karsten Maurer & Daniele Nosenzo & Ruth Potter & Kim Rocha & Aldo Rustichini, 2013. "Self-selection and variations in the laboratory measurement of other-regarding preferences across subject pools: evidence from one college student and two adult samples," Experimental Economics, Springer;Economic Science Association, vol. 16(2), pages 170-189, June.
    18. Peth, Denise & Mußhoff, Oliver & Funke, Katja & Hirschauer, Norbert, 2018. "Nudging Farmers to Comply With Water Protection Rules – Experimental Evidence From Germany," Ecological Economics, Elsevier, vol. 152(C), pages 310-321.
    19. Liesbeth Colen & Sergio Gomez y Paloma & Uwe Latacz-Lohmann & Marianne Lefebvre & Raphaële Préget & Sophie Thoyer, 2016. "Economic Experiments as a Tool for Agricultural Policy Evaluation: Insights from the European CAP," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 64(4), pages 667-694, December.
    20. Cason, Timothy N & Gangadharan, Lata, 2003. "Transactions Costs in Tradable Permit Markets: An Experimental Study of Pollution Market Designs," Journal of Regulatory Economics, Springer, vol. 23(2), pages 145-165, March.
    21. Blair Cleave & Nikos Nikiforakis & Robert Slonim, 2013. "Is there selection bias in laboratory experiments? The case of social and risk preferences," Experimental Economics, Springer;Economic Science Association, vol. 16(3), pages 372-382, September.
    22. Alessandro Tavoni & Astrid Dannenberg & Giorgos Kallis & Andreas L�schel, 2011. "Inequality, communication and the avoidance of disastrous climate change," GRI Working Papers 34, Grantham Research Institute on Climate Change and the Environment.
    23. Cason, Timothy N. & Gangadharan, Lata, 2011. "Price discovery and intermediation in linked emissions trading markets: A laboratory study," Ecological Economics, Elsevier, vol. 70(7), pages 1424-1433, May.
    24. Handberg, Øyvind Nystad & Angelsen, Arild, 2019. "Pay little, get little; pay more, get a little more: A framed forest experiment in Tanzania," Ecological Economics, Elsevier, vol. 156(C), pages 454-467.
    25. repec:cup:judgdm:v:6:y:2011:i:8:p:771-781 is not listed on IDEAS
    26. Rege, Mari & Telle, Kjetil, 2004. "The impact of social approval and framing on cooperation in public good situations," Journal of Public Economics, Elsevier, vol. 88(7-8), pages 1625-1644, July.
    27. Philippe Le Coent & Raphaële Préget & Sophie Thoyer, 2014. "Why pay for nothing? An experiment on a conditional subsidy scheme in a threshold public good game," Economics Bulletin, AccessEcon, vol. 34(3), pages 1976-1989.
    28. Waichman, Israel & Ness, Christiane, 2012. "Farmers’ performance and subject pool effect in decentralized bargaining markets," Economics Letters, Elsevier, vol. 115(3), pages 366-368.
    29. Hasson, Reviva & Löfgren, Åsa & Visser, Martine, 2010. "Climate change in a public goods game: Investment decision in mitigation versus adaptation," Ecological Economics, Elsevier, vol. 70(2), pages 331-338, December.
    30. Ernst Fehr & John A. List, 2004. "The Hidden Costs and Returns of Incentives-Trust and Trustworthiness Among CEOs," Journal of the European Economic Association, MIT Press, vol. 2(5), pages 743-771, September.
    31. Jens Rommel & Daniel Hermann & Malte Müller & Oliver Mußhoff, 2019. "Contextual Framing and Monetary Incentives in Field Experiments on Risk Preferences: Evidence from German Farmers," Journal of Agricultural Economics, Wiley Blackwell, vol. 70(2), pages 408-425, June.
    32. Pevnitskaya, Svetlana & Ryvkin, Dmitry, 2013. "Environmental context and termination uncertainty in games with a dynamic public bad," Environment and Development Economics, Cambridge University Press, vol. 18(1), pages 27-49, February.
    33. Krister P. Andersson & Nathan J. Cook & Tara Grillos & Maria Claudia Lopez & Carl F. Salk & Glenn D. Wright & Esther Mwangi, 2018. "Experimental evidence on payments for forest commons conservation," Nature Sustainability, Nature, vol. 1(3), pages 128-135, March.
    34. Parkhurst, Gregory M. & Shogren, Jason F. & Bastian, Chris & Kivi, Paul & Donner, Jennifer & Smith, Rodney B. W., 2002. "Agglomeration bonus: an incentive mechanism to reunite fragmented habitat for biodiversity conservation," Ecological Economics, Elsevier, vol. 41(2), pages 305-328, May.
    35. Elliott, Catherine S. & Hayward, Donald M. & Canon, Sebastian, 1998. "Institutional framing: Some experimental evidence," Journal of Economic Behavior & Organization, Elsevier, vol. 35(4), pages 455-464, May.
    36. Ernst Fehr & Urs Fischbacher, 2003. "The nature of human altruism," Nature, Nature, vol. 425(6960), pages 785-791, October.
    37. Banerjee, Simanti & Kwasnica, Anthony M. & Shortle, James S., 2012. "Agglomeration bonus in small and large local networks: A laboratory examination of spatial coordination," Ecological Economics, Elsevier, vol. 84(C), pages 142-152.
    38. Handberg, Øyvind Nystad & Angelsen, Arild, 2015. "Experimental tests of tropical forest conservation measures," Journal of Economic Behavior & Organization, Elsevier, vol. 118(C), pages 346-359.
    39. Ropret Homar, Aja & Knežević Cvelbar, Ljubica, 2021. "The effects of framing on environmental decisions: A systematic literature review," Ecological Economics, Elsevier, vol. 183(C).
    40. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    41. Gächter, Simon & Orzen, Henrik & Renner, Elke & Starmer, Chris, 2009. "Are experimental economists prone to framing effects? A natural field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 70(3), pages 443-446, June.
    42. Wünscher, Tobias & Engel, Stefanie & Wunder, Sven, 2008. "Spatial targeting of payments for environmental services: A tool for boosting conservation benefits," Ecological Economics, Elsevier, vol. 65(4), pages 822-833, May.
    43. Kaczan, David J. & Swallow, Brent M. & Adamowicz, W.L. (Vic), 2019. "Forest conservation policy and motivational crowding: Experimental evidence from Tanzania," Ecological Economics, Elsevier, vol. 156(C), pages 444-453.
    44. Armin Falk & Stephan Meier & Christian Zehnder, 2013. "Do Lab Experiments Misrepresent Social Preferences? The Case Of Self-Selected Student Samples," Journal of the European Economic Association, European Economic Association, vol. 11(4), pages 839-852, August.
    45. John Rolfe & Jill Windle & Juliana McCosker, 2009. "Testing and Implementing the Use of Multiple Bidding Rounds in Conservation Auctions: A Case Study Application," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 57(3), pages 287-303, September.
    46. Catherine Eckel & Philip Grossman, 2000. "Volunteers and Pseudo-Volunteers: The Effect of Recruitment Method in Dictator Experiments," Experimental Economics, Springer;Economic Science Association, vol. 3(2), pages 107-120, October.
    47. Elliott, Catherine S. & Hayward, Donald M., 1998. "The expanding definition of framing and its particular impact on economic experimentation," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 27(2), pages 229-243.
    48. Bottazzi, Patrick & Wiik, Emma & Crespo, David & Jones, Julia P.G., 2018. "Payment for Environmental “Self-Service”: Exploring the Links Between Farmers' Motivation and Additionality in a Conservation Incentive Programme in the Bolivian Andes," Ecological Economics, Elsevier, vol. 150(C), pages 11-23.
    49. Voors, Maarten & Demont, Matty & Bulte, Erwin, 2016. "New Experiments in Agriculture," African Journal of Agricultural and Resource Economics, African Association of Agricultural Economists, vol. 11(1), pages 1-7, March.
    50. Steven D. Levitt & John A. List, 2007. "What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World?," Journal of Economic Perspectives, American Economic Association, vol. 21(2), pages 153-174, Spring.
    51. Brian E. Roe & David R. Just, 2009. "Internal and External Validity in Economics Research: Tradeoffs between Experiments, Field Experiments, Natural Experiments, and Field Data," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 91(5), pages 1266-1271.
    52. Ferré, Marie & Engel, Stefanie & Gsottbauer, Elisabeth, 2018. "Which Agglomeration Payment for a Sustainable Management of Organic Soils in Switzerland? – An Experiment Accounting for Farmers' Cost Heterogeneity," Ecological Economics, Elsevier, vol. 150(C), pages 24-33.
    53. Francisco Alpízar & Anna Nordén & Alexander Pfaff & Juan Robalino, 2017. "Unintended Effects of Targeting an Environmental Rebate," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 67(1), pages 181-202, May.
    54. Erik Snowberg & Leeat Yariv, 2021. "Testing the Waters: Behavior across Participant Pools," American Economic Review, American Economic Association, vol. 111(2), pages 687-719, February.
    55. Timothy N. Cason & Lata Gangadharan, 2004. "Auction Design for Voluntary Conservation Programs," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 86(5), pages 1211-1217.
    56. Rachel Croson & Uri Gneezy, 2009. "Gender Differences in Preferences," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 448-474, June.
    57. Martin Kesternich & Christiane Reif & Dirk Rübbelke, 2017. "Recent Trends in Behavioral Environmental Economics," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 67(3), pages 403-411, July.
    58. Liesbeth Colen & Sergio Gomez y Paloma & Uwe Latacz-Lohmann & Marianne Lefebvre & Raphaële Préget & Sophie Thoyer, 2016. "Economic Experiments as a Tool for Agricultural Policy Evaluation: Insights from the European CAP," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 64(4), pages 667-694, December.
    59. Charness, Gary & Gneezy, Uri, 2012. "Strong Evidence for Gender Differences in Risk Taking," Journal of Economic Behavior & Organization, Elsevier, vol. 83(1), pages 50-58.
    60. Jens Rommel & Daniel Hermann & Malte Müller & Oliver Mußhoff, 2017. "Comprehension in risk elicitation experiments," Applied Economics Letters, Taylor & Francis Journals, vol. 24(9), pages 627-634, May.
    61. Dörschner, T. & Musshoff, O., 2015. "How do incentive-based environmental policies affect environment protection initiatives of farmers? An experimental economic analysis using the example of species richness," Ecological Economics, Elsevier, vol. 114(C), pages 90-103.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Frans P. Vries & Nick Hanley, 2016. "Incentive-Based Policy Design for Pollution Control and Biodiversity Conservation: A Review," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 63(4), pages 687-702, April.
    2. Gruner, Sven & Lehberger, Mira & Hirschauer, Norbert & Mußhoff, Oliver, 2022. "How (un)informative are experiments with students for other social groups? A study of agricultural students and farmers," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 66(03), January.
    3. Kuhfuss, Laure & Préget, Raphaële & Thoyer, Sophie & de Vries, Frans P. & Hanley, Nick, 2022. "Enhancing spatial coordination in payment for ecosystem services schemes with non-pecuniary preferences," Ecological Economics, Elsevier, vol. 192(C).
    4. Stephanie Rosch & Sharon Raszap Skorbiansky & Collin Weigel & Kent D. Messer & Daniel Hellerstein, 2021. "Barriers to Using Economic Experiments in Evidence‐Based Agricultural Policymaking," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 43(2), pages 531-555, June.
    5. Gruener, Sven & Lehberger, Mira & Hirschauer, Norbert & Mußhoff, Oliver, 2021. "How (un-)informative are experiments with “standard subjects” for other social groups? – The case of agricultural students and farmers," SocArXiv psda5, Center for Open Science.
    6. Zhaoyang Liu & Simanti Banerjee & Timothy N. Cason & Nick Hanley & Qi Liu & Jintao Xu & Andreas Kontoleon, 2024. "Spatially coordinated conservation auctions: A framed field experiment focusing on farmland wildlife conservation in China," American Journal of Agricultural Economics, John Wiley & Sons, vol. 106(4), pages 1354-1379, August.
    7. Zhaoyang Liu & Jintao Xu & Xiaojun Yang & Qin Tu & Nick Hanley & Andreas Kontoleon, 2019. "Performance of Agglomeration Bonuses in Conservation Auctions: Lessons from a Framed Field Experiment," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 73(3), pages 843-869, July.
    8. Matteo M. Galizzi & Daniel Navarro-Martinez, 2019. "On the External Validity of Social Preference Games: A Systematic Lab-Field Study," Management Science, INFORMS, vol. 65(3), pages 976-1002, March.
    9. Nick Hanley & Simanti Banerjee & Gareth D. Lennox & Paul R. Armsworth, 2012. "How should we incentivize private landowners to ‘produce’ more biodiversity?," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 28(1), pages 93-113, Spring.
    10. Nyanghura, Qambemeda M. & Biber-Freudenberger, Lisa & Börner, Jan, 2024. "Incentives for biodiversity conservation under asymmetric land ownership," Ecological Economics, Elsevier, vol. 219(C).
    11. Frigau, Luca & Medda, Tiziana & Pelligra, Vittorio, 2019. "From the field to the lab. An experiment on the representativeness of standard laboratory subjects," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 78(C), pages 160-169.
    12. Nguyen, Chi & Latacz-Lohmann, Uwe, 2023. "Assessing the performance of agglomeration bonus in budget-constrained conservation auctions," 97th Annual Conference, March 27-29, 2023, Warwick University, Coventry, UK 334544, Agricultural Economics Society - AES.
    13. Shimada, Hideki, 2020. "Do monetary rewards for spatial coordination enhance participation in a forest incentive program?," Ecological Economics, Elsevier, vol. 177(C).
    14. Chi Nguyen & Uwe Latacz‐Lohmann & Nick Hanley, 2024. "Landscape‐level determinants of the performance of an agglomeration bonus in conservation auctions," Journal of Agricultural Economics, Wiley Blackwell, vol. 75(2), pages 592-616, June.
    15. Nguyen, Chi & Latacz-Lohmann, Uwe & Hanley, Nick & Schilizzi, Steven & Iftekhar, Sayed, 2022. "Spatial Coordination Incentives for landscape-scale environmental management: A systematic review," Land Use Policy, Elsevier, vol. 114(C).
    16. Goeschl, Timo & Kettner, Sara Elisa & Lohse, Johannes & Schwieren, Christiane, 2020. "How much can we learn about voluntary climate action from behavior in public goods games?," Ecological Economics, Elsevier, vol. 171(C).
    17. J. Nicolas Hernandez-Aguilera & Max Mauerman & Alexandra Herrera & Kathryn Vasilaky & Walter Baethgen & Ana Maria Loboguerrero & Rahel Diro & Yohana Tesfamariam Tekeste & Daniel Osgood, 2020. "Games and Fieldwork in Agriculture: A Systematic Review of the 21st Century in Economics and Social Science," Games, MDPI, vol. 11(4), pages 1-22, October.
    18. Villamayor-Tomas, Sergio & Sagebiel, Julian & Rommel, Jens & Olschewski, Roland, 2021. "Types of collective action problems and farmers’ willingness to accept agri-environmental schemes in Switzerland," Ecosystem Services, Elsevier, vol. 50(C).
    19. Leah H. Palm-Forster & Paul J. Ferraro & Nicholas Janusch & Christian A. Vossler & Kent D. Messer, 2019. "Behavioral and Experimental Agri-Environmental Research: Methodological Challenges, Literature Gaps, and Recommendations," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 73(3), pages 719-742, July.
    20. Rommel, Jens & Anggraini, Eva, 2018. "Spatially explicit framed field experiments on ecosystem services governance," Ecosystem Services, Elsevier, vol. 34(PB), pages 201-205.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jageco:v:74:y:2023:i:3:p:661-685. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0021-857X .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.