IDEAS home Printed from https://ideas.repec.org/a/eee/jeeman/v125y2024ics0095069624000457.html
   My bibliography  Save this article

Impact evaluation with nonrepeatable outcomes: The case of forest conservation

Author

Listed:
  • Garcia, Alberto
  • Heilmayr, Robert

Abstract

The application of quasiexperimental impact evaluation to remotely sensed measures of deforestation has yielded important evidence detailing the effectiveness of conservation policies. However, researchers have paid insufficient attention to the binary and nonrepeatable structure of most deforestation datasets. Using analytical proofs and simulations, we demonstrate that many commonly employed econometric approaches are biased when applied to binary and nonrepeatable outcomes. The significance, magnitude and even direction of estimated effects from many studies are likely incorrect, threatening to undermine the evidence base that underpins conservation policy adoption and design. To address these concerns, we provide guidance and new strategies for the design of panel econometric models that yield more reliable estimates of the impacts of forest conservation policies.

Suggested Citation

  • Garcia, Alberto & Heilmayr, Robert, 2024. "Impact evaluation with nonrepeatable outcomes: The case of forest conservation," Journal of Environmental Economics and Management, Elsevier, vol. 125(C).
  • Handle: RePEc:eee:jeeman:v:125:y:2024:i:c:s0095069624000457
    DOI: 10.1016/j.jeem.2024.102971
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0095069624000457
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jeem.2024.102971?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Helmut Farbmacher & Harald Tauchmann, 2023. "Linear fixed-effects estimation with nonrepeated outcomes," Econometric Reviews, Taylor & Francis Journals, vol. 42(8), pages 635-654, September.
    2. Fernández-Val, Iván & Weidner, Martin, 2016. "Individual and time effects in nonlinear panel models with large N, T," Journal of Econometrics, Elsevier, vol. 192(1), pages 291-312.
    3. Clément de Chaisemartin & Xavier D'Haultfœuille, 2020. "Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects," American Economic Review, American Economic Association, vol. 110(9), pages 2964-2996, September.
    4. Amin, A. & Choumert-Nkolo, J. & Combes, J.-L. & Combes Motel, P. & Kéré, E.N. & Ongono-Olinga, J.-G. & Schwartz, S., 2019. "Neighborhood effects in the Brazilian Amazônia: Protected areas and deforestation," Journal of Environmental Economics and Management, Elsevier, vol. 93(C), pages 272-288.
    5. Eli Puterman & Jordan Weiss & Benjamin A. Hives & Alison Gemmill & Deborah Karasek & Wendy Berry Mendes & David H. Rehkopf, 2020. "Predicting mortality from 57 economic, behavioral, social, and psychological factors," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 117(28), pages 16273-16282, July.
    6. Jonathan Proctor & Tamma Carleton & Sandy Sum, 2023. "Parameter Recovery Using Remotely Sensed Variables," NBER Working Papers 30861, National Bureau of Economic Research, Inc.
    7. Jan Börner & Dario Schulz & Sven Wunder & Alexander Pfaff, 2020. "The Effectiveness of Forest Conservation Policies and Programs," Annual Review of Resource Economics, Annual Reviews, vol. 12(1), pages 45-64, October.
    8. Suzi Kerr & Shuguang Liu & Alexander S. P. Pfaff & R. Flint Hughes, 2003. "Carbon Dynamics and Land-Use Choices: Building a Regional-Scale Multidisciplinary Model," Working Papers 03_06, Motu Economic and Public Policy Research.
    9. Imai, Kosuke & Kim, In Song, 2021. "On the Use of Two-Way Fixed Effects Regression Models for Causal Inference with Panel Data," Political Analysis, Cambridge University Press, vol. 29(3), pages 405-415, July.
    10. Amanda Y. Agan & Michael D. Makowsky, 2023. "The Minimum Wage, EITC, and Criminal Recidivism," Journal of Human Resources, University of Wisconsin Press, vol. 58(5), pages 1712-1751.
    11. Diego Herrera & Alexander Pfaff & Juan Robalino, 2019. "Impacts of protected areas vary with the level of government: Comparing avoided deforestation across agencies in the Brazilian Amazon," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 116(30), pages 14916-14925, July.
    12. Bryan Bollinger & Kenneth Gillingham & A. Justin Kirkpatrick & Steven Sexton, 2022. "Visibility and Peer Influence in Durable Good Adoption," Marketing Science, INFORMS, vol. 41(3), pages 453-476, May.
    13. Hans Bloemen & Stefan Hochguertel & Jochem Zweerink, 2017. "The causal effect of retirement on mortality: Evidence from targeted incentives to retire early," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 204-218, December.
    14. Blackman, Allen, 2015. "Strict versus mixed-use protected areas: Guatemala's Maya Biosphere Reserve," Ecological Economics, Elsevier, vol. 112(C), pages 14-24.
    15. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, April.
    16. Rana, Pushpendra & Sills, Erin O., 2024. "Inviting oversight: Effects of forest certification on deforestation in the Brazilian Amazon," World Development, Elsevier, vol. 173(C).
    17. Pfaff, Alexander S. P. & Sanchez-Azofeifa, G. Arturo, 2004. "Deforestation pressure and biological reserve planning: a conceptual approach and an illustrative application for Costa Rica," Resource and Energy Economics, Elsevier, vol. 26(2), pages 237-254, June.
    18. Araujo, Claudio & Bonjean, Catherine Araujo & Combes, Jean-Louis & Combes Motel, Pascale & Reis, Eustaquio J., 2009. "Property rights and deforestation in the Brazilian Amazon," Ecological Economics, Elsevier, vol. 68(8-9), pages 2461-2468, June.
    19. Giovanni Mastrobuoni & Paolo Pinotti, 2015. "Legal Status and the Criminal Activity of Immigrants," American Economic Journal: Applied Economics, American Economic Association, vol. 7(2), pages 175-206, April.
    20. Rodrigo A. Arriagada, & Paul J. Ferraro & Erin O. Sills & Subhrendu K. Pattanayak & Silvia Cordero-Sancho, 2012. "Do Payments for Environmental Services Affect Forest Cover? A Farm-Level Evaluation from Costa Rica," Land Economics, University of Wisconsin Press, vol. 88(2), pages 382-399.
    21. Nicolas Koch & Erasmus K H J zu Ermgassen & Johanna Wehkamp & Francisco J B Oliveira Filho & Gregor Schwerhoff, 2019. "Agricultural Productivity and Forest Conservation: Evidence from the Brazilian Amazon," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 101(3), pages 919-940.
    22. Ruggiero, Patricia G.C. & Metzger, Jean Paul & Reverberi Tambosi, Leandro & Nichols, Elizabeth, 2019. "Payment for ecosystem services programs in the Brazilian Atlantic Forest: Effective but not enough," Land Use Policy, Elsevier, vol. 82(C), pages 283-291.
    23. Christian Baehr & Ariel BenYishay & Bradley Parks, 2021. "Linking Local Infrastructure Development and Deforestation: Evidence from Satellite and Administrative Data," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 8(2), pages 375-409.
    24. Baylis, Katherine R. & Honey‐Roses, Jordi & Ramirez, M. Isabel, 2012. "Conserving Forests: Mandates, Management or Money?," 2012 Annual Meeting, August 12-14, 2012, Seattle, Washington 124983, Agricultural and Applied Economics Association.
    25. Dan Bogart, 2018. "Party Connections, Interest Groups and the Slow Diffusion of Infrastructure: Evidence from Britain's First Transport Revolution," Economic Journal, Royal Economic Society, vol. 128(609), pages 541-575, March.
    26. Nolte, Christoph & Gobbi, Beatriz & le Polain de Waroux, Yann & Piquer-Rodríguez, María & Butsic, Van & Lambin, Eric F., 2017. "Decentralized Land Use Zoning Reduces Large-scale Deforestation in a Major Agricultural Frontier," Ecological Economics, Elsevier, vol. 136(C), pages 30-40.
    27. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    28. Blackman, Allen, 2013. "Evaluating forest conservation policies in developing countries using remote sensing data: An introduction and practical guide," Forest Policy and Economics, Elsevier, vol. 34(C), pages 1-16.
    29. Blackman, Allen & Goff, Leonard & Rivera Planter, Marisol, 2018. "Does eco-certification stem tropical deforestation? Forest Stewardship Council certification in Mexico," Journal of Environmental Economics and Management, Elsevier, vol. 89(C), pages 306-333.
    30. Karyn Tabor & Kelly W Jones & Jennifer Hewson & Andriambolantsoa Rasolohery & Andoniaina Rambeloson & Tokihenintsoa Andrianjohaninarivo & Celia A Harvey, 2017. "Evaluating the effectiveness of conservation and development investments in reducing deforestation and fires in Ankeniheny-Zahemena Corridor, Madagascar," PLOS ONE, Public Library of Science, vol. 12(12), pages 1-23, December.
    31. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    32. Kelly J. Wendland & Matthias Baumann & David J. Lewis & Anika Sieber & Volker C. Radeloff, 2015. "Protected Area Effectiveness in European Russia: A Postmatching Panel Data Analysis," Land Economics, University of Wisconsin Press, vol. 91(1), pages 149-168.
    33. Daniela A. Miteva & Subhrendu K. Pattanayak & Paul J. Ferraro, 2012. "Evaluation of biodiversity policy instruments: what works and what doesn’t?," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 28(1), pages 69-92, Spring.
    34. Sims, Katharine R.E. & Alix-Garcia, Jennifer M., 2017. "Parks versus PES: Evaluating direct and incentive-based land conservation in Mexico," Journal of Environmental Economics and Management, Elsevier, vol. 86(C), pages 8-28.
    35. Gibson, John & Olivia, Susan & Boe-Gibson, Geua & Li, Chao, 2021. "Which night lights data should we use in economics, and where?," Journal of Development Economics, Elsevier, vol. 149(C).
    36. Jonathan Roth, 2022. "Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends," American Economic Review: Insights, American Economic Association, vol. 4(3), pages 305-322, September.
    37. Anthony Joe Turkson & Francis Ayiah-Mensah & Vivian Nimoh & Niansheng Tang, 2021. "Handling Censoring and Censored Data in Survival Analysis: A Standalone Systematic Literature Review," International Journal of Mathematics and Mathematical Sciences, Hindawi, vol. 2021, pages 1-16, September.
    38. Ho, Daniel E. & Imai, Kosuke & King, Gary & Stuart, Elizabeth A., 2007. "Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference," Political Analysis, Cambridge University Press, vol. 15(3), pages 199-236, July.
    39. Andre Fernandes Tomon Avelino & Kathy Baylis & Jordi Honey-Rosés, 2016. "Goldilocks and the Raster Grid: Selecting Scale when Evaluating Conservation Programs," PLOS ONE, Public Library of Science, vol. 11(12), pages 1-24, December.
    40. Anderson, Christa M. & Asner, Gregory P. & Llactayo, William & Lambin, Eric F., 2018. "Overlapping land allocations reduce deforestation in Peru," Land Use Policy, Elsevier, vol. 79(C), pages 174-178.
    41. Rico-Straffon, Jimena & Wang, Zhenhua & Panlasigui, Stephanie & Loucks, Colby J. & Swenson, Jennifer & Pfaff, Alexander, 2023. "Forest concessions and eco-certifications in the Peruvian Amazon: Deforestation impacts of logging rights and logging restrictions," Journal of Environmental Economics and Management, Elsevier, vol. 118(C).
    42. BenYishay, Ariel & Heuser, Silke & Runfola, Daniel & Trichler, Rachel, 2017. "Indigenous land rights and deforestation: Evidence from the Brazilian Amazon," Journal of Environmental Economics and Management, Elsevier, vol. 86(C), pages 29-47.
    43. John Gardner, 2022. "Two-stage differences in differences," Papers 2207.05943, arXiv.org.
    44. Dave Donaldson & Adam Storeygard, 2016. "The View from Above: Applications of Satellite Data in Economics," Journal of Economic Perspectives, American Economic Association, vol. 30(4), pages 171-198, Fall.
    45. Meha Jain, 2020. "The Benefits and Pitfalls of Using Satellite Data for Causal Inference," Review of Environmental Economics and Policy, University of Chicago Press, vol. 14(1), pages 157-169.
    46. Lancaster, Tony, 2000. "The incidental parameter problem since 1948," Journal of Econometrics, Elsevier, vol. 95(2), pages 391-413, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Reynaert, Mathias & Souza-Rodrigues, Eduardo & van Benthem, Arthur A., 2024. "The environmental impacts of protected area policy," Regional Science and Urban Economics, Elsevier, vol. 107(C).
    2. Cisneros, Elías & Börner, Jan & Pagiola, Stefano & Wunder, Sven, 2022. "Impacts of conservation incentives in protected areas: The case of Bolsa Floresta, Brazil," Journal of Environmental Economics and Management, Elsevier, vol. 111(C).
    3. Rana, Pushpendra & Sills, Erin O., 2024. "Inviting oversight: Effects of forest certification on deforestation in the Brazilian Amazon," World Development, Elsevier, vol. 173(C).
    4. Isabelle Chort & Berk Öktem, 2024. "Agricultural shocks, coping policies and deforestation: Evidence from the coffee leaf rust epidemic in Mexico," American Journal of Agricultural Economics, John Wiley & Sons, vol. 106(3), pages 1020-1057, May.
    5. Katie Devenish & Sébastien Desbureaux & Simon Willcock & Julia P. G. Jones, 2022. "On track to achieve no net loss of forest at Madagascar’s biggest mine," Nature Sustainability, Nature, vol. 5(6), pages 498-508, June.
    6. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Jun 2024.
    7. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.
    8. Gregory Faletto, 2023. "Fused Extended Two-Way Fixed Effects for Difference-in-Differences With Staggered Adoptions," Papers 2312.05985, arXiv.org, revised Oct 2024.
    9. Goodman-Bacon, Andrew, 2021. "Difference-in-differences with variation in treatment timing," Journal of Econometrics, Elsevier, vol. 225(2), pages 254-277.
    10. Chervier, Colas & Ximenes, Arimatéa C. & Mihigo, Blaise-Pascal Ntirumenyerwa & Doumenge, Charles, 2024. "Impact of industrial logging concession on deforestation and forest degradation in the DRC," World Development, Elsevier, vol. 173(C).
    11. Callaway, Brantly & Li, Tong, 2023. "Policy evaluation during a pandemic," Journal of Econometrics, Elsevier, vol. 236(1).
    12. Rico-Straffon, Jimena & Wang, Zhenhua & Panlasigui, Stephanie & Loucks, Colby J. & Swenson, Jennifer & Pfaff, Alexander, 2023. "Forest concessions and eco-certifications in the Peruvian Amazon: Deforestation impacts of logging rights and logging restrictions," Journal of Environmental Economics and Management, Elsevier, vol. 118(C).
    13. West, Thales A.P., 2024. "Formal designation of Brazilian indigenous lands linked to small but consistent reductions in deforestation," Ecological Economics, Elsevier, vol. 218(C).
    14. Sims, Katharine R.E. & Alix-Garcia, Jennifer M., 2017. "Parks versus PES: Evaluating direct and incentive-based land conservation in Mexico," Journal of Environmental Economics and Management, Elsevier, vol. 86(C), pages 8-28.
    15. Blackman, Allen & Goff, Leonard & Rivera Planter, Marisol, 2018. "Does eco-certification stem tropical deforestation? Forest Stewardship Council certification in Mexico," Journal of Environmental Economics and Management, Elsevier, vol. 89(C), pages 306-333.
    16. Crescioli, Tommaso, 2024. "Reinforcing each other: How the combination of European and domestic reforms increased competition in liberalized industries," European Journal of Political Economy, Elsevier, vol. 83(C).
    17. Posso, Christian & Saravia, Estefanía & Uribe, Pablo, 2023. "Acing the test: Educational effects of the SaberEs test preparation program in Colombia," Economics of Education Review, Elsevier, vol. 97(C).
    18. Mark Kattenberg & Bas Scheer & Jurre Thiel, 2023. "Causal forests with fixed effects for treatment effect heterogeneity in difference-in-differences," CPB Discussion Paper 452, CPB Netherlands Bureau for Economic Policy Analysis.
    19. Pablo Cuenca & Juan Robalino & Rodrigo Arriagada & Cristian Echeverría, 2018. "Are government incentives effective for avoided deforestation in the tropical Andean forest?," PLOS ONE, Public Library of Science, vol. 13(9), pages 1-14, September.
    20. Jeffrey M Wooldridge, 2023. "Simple approaches to nonlinear difference-in-differences with panel data," The Econometrics Journal, Royal Economic Society, vol. 26(3), pages 31-66.

    More about this item

    Keywords

    Conservation; Deforestation; Impact evaluation; Remote sensing;
    All these keywords.

    JEL classification:

    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • Q23 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Renewable Resources and Conservation - - - Forestry
    • Q24 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Renewable Resources and Conservation - - - Land
    • Q57 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Environmental Economics - - - Ecological Economics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jeeman:v:125:y:2024:i:c:s0095069624000457. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/622870 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.