IDEAS home Printed from https://ideas.repec.org/p/hal/pseptp/halshs-04155302.html
   My bibliography  Save this paper

The Rigor Revolution: New Standards of Evidence for Impact Assessment of International Agricultural Research

Author

Listed:
  • James Stevenson

    (IFPRI - International Food Policy Research Institute [India] - IFPRI - International Food Policy Research Institute [Washington] - CGIAR - Consultative Group on International Agricultural Research [CGIAR], CGIAR Standing Panel on Impact Assessment)

  • Karen Macours

    (PSE - Paris School of Economics - UP1 - Université Paris 1 Panthéon-Sorbonne - ENS-PSL - École normale supérieure - Paris - PSL - Université Paris Sciences et Lettres - EHESS - École des hautes études en sciences sociales - ENPC - École des Ponts ParisTech - CNRS - Centre National de la Recherche Scientifique - INRAE - Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement, PJSE - Paris Jourdan Sciences Economiques - UP1 - Université Paris 1 Panthéon-Sorbonne - ENS-PSL - École normale supérieure - Paris - PSL - Université Paris Sciences et Lettres - EHESS - École des hautes études en sciences sociales - ENPC - École des Ponts ParisTech - CNRS - Centre National de la Recherche Scientifique - INRAE - Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement)

  • Douglas Gollin

    (University of Oxford)

Abstract

We take stock of the major changes in methodology for studying the impacts of international agricultural research, focusing on the period 2006–2020. Impact assessment of agricultural research has a long and recognized tradition. Until the mid-2000s, such assessments were dominated by a model of demand for and supply of agricultural products in partial equilibrium. The basic ideas for this approach were sketched out by Griliches more than half a century ago. We describe the implications of heightened standards of evidence for good practice in three domains of research design: causal inference, valid measurement, and statistical representativeness. We document advances in each of these domains and review recent evidence that demonstrates the lessons that can be learned from adopting these practices, emphasizing the importance of evidence at-scale, the need to consider portfolios of innovations at a national level, and the challenges of accounting for innovations that are promoted as bundles.

Suggested Citation

  • James Stevenson & Karen Macours & Douglas Gollin, 2023. "The Rigor Revolution: New Standards of Evidence for Impact Assessment of International Agricultural Research," PSE-Ecole d'économie de Paris (Postprint) halshs-04155302, HAL.
  • Handle: RePEc:hal:pseptp:halshs-04155302
    DOI: 10.1146/annurev-resource-101722-082519
    as

    Download full text from publisher

    To our knowledge, this item is not available for download. To find whether it is available, there are three options:
    1. Check below whether another version of this item is available online.
    2. Check on the provider's web page whether it is in fact available.
    3. Perform a search for a similarly titled item that would be available.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Howard White, 2009. "Theory-based impact evaluation: principles and practice," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 1(3), pages 271-284.
    2. Douglas Gollin & Christopher Udry, 2021. "Heterogeneity, Measurement Error, and Misallocation: Evidence from African Agriculture," Journal of Political Economy, University of Chicago Press, vol. 129(1), pages 1-80.
    3. Lori Beaman & Ariel BenYishay & Jeremy Magruder & Ahmed Mushfiq Mobarak, 2021. "Can Network Theory-Based Targeting Increase Technology Adoption?," American Economic Review, American Economic Association, vol. 111(6), pages 1918-1943, June.
    4. Rachid Laaja & Karen Macours, 2021. "Measuring Skills in Developing Countries," Journal of Human Resources, University of Wisconsin Press, vol. 56(4), pages 1254-1295.
    5. Rachid Laajaj & Karen Macours & Cargele Masso & Moses Thuita & Bernard Vanlauwe, 2020. "Reconciling yield gains in agronomic trials with returns under African smallholder conditions," PSE-Ecole d'économie de Paris (Postprint) halshs-02973685, HAL.
    6. Xudong Rao & Terrance M. Hurley & Philip G. Pardey, 2020. "Recalibrating the reported returns to agricultural R&D: what if we all heeded Griliches?," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 64(3), pages 977-1001, July.
    7. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    8. Beaman, Lori & Dillon, Andrew, 2018. "Diffusion of agricultural information within social networks: Evidence on gender inequalities from Mali," Journal of Development Economics, Elsevier, vol. 133(C), pages 147-161.
    9. Benjamin A. Olken, 2015. "Promises and Perils of Pre-analysis Plans," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 61-80, Summer.
    10. Beegle, Kathleen & Carletto, Calogero & Himelein, Kristen, 2012. "Reliability of recall in agricultural data," Journal of Development Economics, Elsevier, vol. 98(1), pages 34-41.
    11. Quisumbing, Agnes R. & Ahmed, Akhter & Gilligan, Daniel O. & Hoddinott, John & Kumar, Neha & Leroy, Jef L. & Menon, Purnima & Olney, Deanna K. & Roy, Shalini & Ruel, Marie, 2020. "Randomized controlled trials of multi-sectoral programs: Lessons from development research," World Development, Elsevier, vol. 127(C).
    12. Tanguy Bernard & Sylvie Lambert & Karen Macours & Margaux Vinez, 2023. "Impact of small farmers' access to improved seeds and deforestation in DR Congo," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    13. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    14. Carletto,Calogero & Dillon,Andrew S. & Zezza,Alberto, 2021. "Agricultural Data Collection to Minimize Measurement Error and Maximize Coverage," Policy Research Working Paper Series 9745, The World Bank.
    15. Christopher B. Barrett & Tim G. Benton & Karen A. Cooper & Jessica Fanzo & Rikin Gandhi & Mario Herrero & Steven James & Mark Kahn & Daniel Mason-D’Croz & Alexander Mathys & Rebecca J. Nelson & Jianbo, 2020. "Bundling innovations to transform agri-food systems," Nature Sustainability, Nature, vol. 3(12), pages 974-976, December.
    16. Jeffrey J. Reimer, 2007. "Assessing Global Computable General Equilibrium Model Validity Using Agricultural Price Volatility," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 89(2), pages 383-397.
    17. Hurley, Terrance M. & Pardey, Philip G. & Rao, Xudong & Andrade, Robert S., 2016. "Returns to Food and Agricultural R&D Investments Worldwide, 1958-2015," Briefs 249356, University of Minnesota, International Science and Technology Practice and Policy.
    18. Christopher B. Barrett & Michael R. Carter, 2010. "The Power and Pitfalls of Experiments in Development Economics: Some Non-random Reflections," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 32(4), pages 515-548.
    19. repec:oup:apecpp:v:40:y:2018:i:3:p:421-444. is not listed on IDEAS
    20. Aragón, Fernando M. & Restuccia, Diego & Rud, Juan Pablo, 2024. "Assessing misallocation in agriculture: Plots versus farms," Food Policy, Elsevier, vol. 128(C).
    21. Kibrom A. Abay & Tesfamicheal Wossen & Gashaw T. Abate & James R. Stevenson & Hope Michelson & Christopher B. Barrett, 2023. "Inferential and Behavioral Implications of Measurement Error in Agricultural Data," Annual Review of Resource Economics, Annual Reviews, vol. 15(1), pages 63-83, October.
    22. Isaiah Andrews & Maximilian Kasy, 2019. "Identification of and Correction for Publication Bias," American Economic Review, American Economic Association, vol. 109(8), pages 2766-2794, August.
    23. Alessandro De Pinto & Keith Wiebe & Pablo Pacheco, 2017. "Help bigger palm oil yields to save land," Nature, Nature, vol. 544(7651), pages 416-416, April.
    24. Joachim De Weerdt & John Gibson & Kathleen Beegle, 2020. "What Can We Learn from Experimenting with Survey Methods?," Annual Review of Resource Economics, Annual Reviews, vol. 12(1), pages 431-447, October.
    25. Maredia, Mywish K. & Reyes, Byron A. & Manu-Aduening, Joseph & Dankyi, Awere & Hamazakaza, Petan & Muimui, Kennedy & Rabbi, Ismail & Kulakow, Peter & Parkes, Elizabeth & Abdoulaye, Tahirou & Katungi, , 2016. "Testing Alternative Methods of Varietal Identification Using DNA Fingerprinting: Results of Pilot Studies in Ghana and Zambia," Food Security International Development Working Papers 246950, Michigan State University, Department of Agricultural, Food, and Resource Economics.
    26. David A Raitzer & Timothy G Kelley, 2008. "Assessing the contribution of impact assessment to donor decisions for international agricultural research," Research Evaluation, Oxford University Press, vol. 17(3), pages 187-199, September.
    27. Cheryl Doss & Caitlin Kieran & Talip Kilic, 2020. "Measuring Ownership, Control, and Use of Assets," Feminist Economics, Taylor & Francis Journals, vol. 26(3), pages 144-168, July.
    28. Ruth Stewart & Laurenz Langer & Natalie Rebelo Da Silva & Evans Muchiri & Hazel Zaranyika & Yvonne Erasmus & Nicola Randall & Shannon Rafferty & Marcel Korth & Nolizwe Madinga & Thea de Wet, 2015. "The Effects of Training, Innovation and New Technology on African Smallholder Farmers' Economic Outcomes and Food Security: A Systematic Review," Campbell Systematic Reviews, John Wiley & Sons, vol. 11(1), pages 1-224.
    29. Mark R Rosenzweig & Christopher Udry, 2020. "External Validity in a Stochastic World: Evidence from Low-Income Countries," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 87(1), pages 343-381.
    30. Magnan, Nicholas & Spielman, David J. & Lybbert, Travis J. & Gulati, Kajal, 2015. "Leveling with friends: Social networks and Indian farmers' demand for a technology with heterogeneous benefits," Journal of Development Economics, Elsevier, vol. 116(C), pages 223-251.
    31. Susan W. Parker & Petra E. Todd, 2017. "Conditional Cash Transfers: The Case of Progresa/Oportunidades," Journal of Economic Literature, American Economic Association, vol. 55(3), pages 866-915, September.
    32. Karen Macours, 2019. "Farmers’ Demand and the Traits and Diffusion of Agricultural Innovations in Developing Countries," Annual Review of Resource Economics, Annual Reviews, vol. 11(1), pages 483-499, October.
    33. Lilyan E. Fulginiti, 2010. "Estimating Griliches' k-Shifts," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 92(1), pages 86-101.
    34. Dalrymple, Dana G., 1978. "Development and Spread of High-Yielding Varieties of Wheat and Rice in the Less Developed Nations (Sixth Edition)," Foreign Agricultural Economic Report (FAER) 330481, United States Department of Agriculture, Economic Research Service.
    35. Lant Pritchett, 2002. "It pays to be ignorant: A simple political economy of rigorous program evaluation," Journal of Economic Policy Reform, Taylor & Francis Journals, vol. 5(4), pages 251-269.
    36. repec:hal:pseptp:halshs-03956411 is not listed on IDEAS
    37. Jenny C. Aker & Kelsey Jack, 2021. "Harvesting the Rain: The Adoption of Environmental Technologies in the Sahel," NBER Working Papers 29518, National Bureau of Economic Research, Inc.
    38. Erwin Corong & Thomas Hertel & Robert McDougall & Marinos Tsigas & Dominique van der Mensbrugghe, 2017. "The Standard GTAP Model, version 7," Journal of Global Economic Analysis, Center for Global Trade Analysis, Department of Agricultural Economics, Purdue University, vol. 2(1), pages 1-119, June.
    39. Sarah Baird & J. Aislinn Bohren & Craig McIntosh & Berk Özler, 2018. "Optimal Design of Experiments in the Presence of Interference," The Review of Economics and Statistics, MIT Press, vol. 100(5), pages 844-860, December.
    40. Kyle Emerick & Manzoor H. Dar, 2021. "Farmer Field Days and Demonstrator Selection for Increasing Technology Adoption," The Review of Economics and Statistics, MIT Press, vol. 103(4), pages 680-693, October.
    41. repec:hal:journl:halshs-03956411 is not listed on IDEAS
    42. Erwin Bulte & Gonne Beekman & Salvatore Di Falco & Joseph Hella & Pan Lei, 2014. "Behavioral Responses and the Impact of New Agricultural Technologies: Evidence from a Double-blind Field Experiment in Tanzania," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 96(3), pages 813-830.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Alemu, Solomon & Kosmowski, Frederic & Stevenson, James R. & Mallia, Paola & Taye, Lemi & Macours, Karen, 2024. "Studying inclusive innovation with the right data: An empirical illustration from Ethiopia," Agricultural Systems, Elsevier, vol. 219(C).
    2. Fuglie, Keith O. & Echeverria, Ruben G., 2024. "The economic impact of CGIAR-related crop technologies on agricultural productivity in developing countries, 1961–2020," World Development, Elsevier, vol. 176(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wossen, Tesfamicheal & Abay, Kibrom A. & Abdoulaye, Tahirou, 2022. "Misperceiving and misreporting input quality: Implications for input use and productivity," Journal of Development Economics, Elsevier, vol. 157(C).
    2. Masselus, Lise & Fiala, Nathan, 2024. "Whom to ask? Testing respondent effects in household surveys," Journal of Development Economics, Elsevier, vol. 168(C).
    3. Kazushi Takahashi & Rie Muraoka & Keijiro Otsuka, 2020. "Technology adoption, impact, and extension in developing countries’ agriculture: A review of the recent literature," Agricultural Economics, International Association of Agricultural Economists, vol. 51(1), pages 31-45, January.
    4. Fiala, Nathan & Masselus, Lise, 2022. "Whom to ask? Testing respondent effects in household surveys," Ruhr Economic Papers 935, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    5. Kabeer, Naila, 2020. "‘Misbehaving’ RCTs: The confounding problem of human agency," World Development, Elsevier, vol. 127(C).
    6. Maximilian Kasy & Jann Spiess, 2022. "Optimal Pre-Analysis Plans: Statistical Decisions Subject to Implementability," Papers 2208.09638, arXiv.org, revised Jul 2024.
    7. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," Working Papers hal-03455978, HAL.
    8. Maurizio Canavari & Andreas C. Drichoutis & Jayson L. Lusk & Rodolfo M. Nayga, Jr., 2018. "How to run an experimental auction: A review of recent advances," Working Papers 2018-5, Agricultural University of Athens, Department Of Agricultural Economics.
    9. repec:lic:licosd:40718 is not listed on IDEAS
    10. Michelson, Hope & Gourlay, Sydney & Lybbert, Travis & Wollburg, Philip, 2023. "Review: Purchased agricultural input quality and small farms," Food Policy, Elsevier, vol. 116(C).
    11. Jérémie Gignoux & Karen Macours & Daniel Stein & Kelsey Wright, 2023. "Input subsidies, credit constraints, and expectations of future transfers: Evidence from Haiti," American Journal of Agricultural Economics, John Wiley & Sons, vol. 105(3), pages 809-835, May.
    12. Anna Dreber & Magnus Johannesson & Yifan Yang, 2024. "Selective reporting of placebo tests in top economics journals," Economic Inquiry, Western Economic Association International, vol. 62(3), pages 921-932, July.
    13. Paola Mallia, 2022. "You reap what (you think) you sow? Evidence on farmers’behavioral adjustments in the case of correct crop varietal identification," PSE Working Papers hal-03597332, HAL.
    14. Michael Carter & Rachid Laajaj & Dean Yang, 2021. "Subsidies and the African Green Revolution: Direct Effects and Social Network Spillovers of Randomized Input Subsidies in Mozambique," American Economic Journal: Applied Economics, American Economic Association, vol. 13(2), pages 206-229, April.
    15. Dominika Ehrenbergerova & Josef Bajzik & Tomas Havranek, 2023. "When Does Monetary Policy Sway House Prices? A Meta-Analysis," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(2), pages 538-573, June.
    16. Annemie Maertens & Hope Michelson & Vesall Nourani, 2021. "How Do Farmers Learn from Extension Services? Evidence from Malawi," American Journal of Agricultural Economics, John Wiley & Sons, vol. 103(2), pages 569-595, March.
    17. Christopher B. Barrett & Asad Islam & Abdul Mohammad Malek & Debayan Pakrashi & Ummul Ruthbah, 2022. "Experimental Evidence on Adoption and Impact of the System of Rice Intensification," American Journal of Agricultural Economics, John Wiley & Sons, vol. 104(1), pages 4-32, January.
    18. Abel Brodeur, Nikolai M. Cook, Jonathan S. Hartley, Anthony Heyes, 2022. "Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?," LCERPA Working Papers am0132, Laurier Centre for Economic Research and Policy Analysis.
    19. Lee, Guenwoo & Suzuki, Aya & Vu, Hoang Nam, 2018. "Comparison of Targeting Methods for the Diffusion of Farming Practices: Evidence from Shrimp Producers in Viet Nam," 2018 Annual Meeting, August 5-7, Washington, D.C. 274043, Agricultural and Applied Economics Association.
    20. Ankel-Peters, Jörg & Schmidt, Christoph M., 2023. "Rural electrification, the credibility revolution, and the limits of evidence-based policy," Ruhr Economic Papers 1051, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    21. Arslan, Cansın & Wollni, Meike & Oduol, Judith & Hughes, Karl, 2022. "Who communicates the information matters for technology adoption," World Development, Elsevier, vol. 158(C).

    More about this item

    Keywords

    agricultural research for development; impact assessment; measurement; causality; scale;
    All these keywords.

    JEL classification:

    • O12 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Microeconomic Analyses of Economic Development
    • O13 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Agriculture; Natural Resources; Environment; Other Primary Products
    • O32 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Management of Technological Innovation and R&D
    • O33 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Technological Change: Choices and Consequences; Diffusion Processes
    • Q16 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Agriculture - - - R&D; Agricultural Technology; Biofuels; Agricultural Extension Services

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:pseptp:halshs-04155302. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Caroline Bauer (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.