IDEAS home Printed from https://ideas.repec.org/p/hal/journl/halshs-03623394.html
   My bibliography  Save this paper

How impact evaluation methods influence the outcomes of development projects? Evidence from a meta-analysis on decentralized solar nano projects

Author

Listed:
  • Fatoumata Nankoto Cissé

    (CES - Centre d'économie de la Sorbonne - UP1 - Université Paris 1 Panthéon-Sorbonne - CNRS - Centre National de la Recherche Scientifique, I&P - Investisseurs et Partenaires)

Abstract

This study analyzes the effect of impact evaluation methodologies on the positive and negative outcomes of decentralized solar nano projects in developing countries. Data originate from the Collaborative Smart Mapping of Mini-grid Actions (CoSMMA) developed by the Foundation for Studies and Research on International Development (FERDI). This study is based on a total of 727 tested effects from 10 decentralized solar nano projects which have been measured by experimental and quasi-experimental approaches. Using a multinomial-logit regression shown that randomized and non-randomized evaluation methods have a similar probability of generating a proven favorable outcome on the sustainable development of decentralized solar nano projects. By estimating a complementary log-log model, projects are most often evaluated as successful when effects on education are tested. In addition, a discrepancy of impacts is found between randomized control trials and difference-indifference strategies in proven-unfavorable outcomes of projects. This analysis also highlights the convergence of impacts between randomization and matching techniques on projects implemented in Africa. Findings from this paper provide strong evidence for development practitioners to choose the appropriate impact assessment method.

Suggested Citation

  • Fatoumata Nankoto Cissé, 2022. "How impact evaluation methods influence the outcomes of development projects? Evidence from a meta-analysis on decentralized solar nano projects," Post-Print halshs-03623394, HAL.
    • Handle: RePEc:hal:journl:halshs-03623394
    Note: View the original document on HAL open archive server: https://shs.hal.science/halshs-03623394
    as

    Download full text from publisher

    File URL: https://shs.hal.science/halshs-03623394/document
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    2. Angus Deaton, 2009. "Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development," Working Papers 1128, Princeton University, Woodrow Wilson School of Public and International Affairs, Center for Health and Wellbeing..
    3. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    4. T. D. Stanley & Stephen B. Jarrell, 2005. "Meta‐Regression Analysis: A Quantitative Method of Literature Surveys," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 299-308, July.
    5. Steven Glazerman & Dan M. Levy & David Myers, 2003. "Nonexperimental Versus Experimental Estimates of Earnings Impacts," The ANNALS of the American Academy of Political and Social Science, , vol. 589(1), pages 63-93, September.
    6. Angus Deaton, 2020. "Randomization in the Tropics Revisited: a Theme and Eleven Variations," NBER Working Papers 27600, National Bureau of Economic Research, Inc.
    7. Chauvet, Lisa & Ehrhart, Hélène, 2018. "Aid and growth: evidence from firm-level data," Journal of Development Economics, Elsevier, vol. 135(C), pages 461-477.
    8. Abhijit Vinayak Banerjee, 2020. "Field Experiments and the Practice of Economics," American Economic Review, American Economic Association, vol. 110(7), pages 1937-1951, July.
    9. Arráiz, Irani & Calero, Carla, 2015. "From Candles to Light: The Impact of Rural Electrification," IDB Publications (Working Papers) 6917, Inter-American Development Bank.
    10. Michael Bamberger & Vijayendra Rao & Michael Woolcock, 2009. "Using Mixed Methods in Monitoring and Evaluation: Experiences from International Development’," Global Development Institute Working Paper Series 10709, GDI, The University of Manchester.
    11. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    12. Jean-Claude Berthélemy & Arnaud Millien, 2018. "Impact of Decentralized Electrification Projects on Sustainable Development: A Meta-Analysis," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) hal-01922517, HAL.
    13. Hristos Doucouliagos & Martin Paldam, 2009. "The Aid Effectiveness Literature: The Sad Results Of 40 Years Of Research," Journal of Economic Surveys, Wiley Blackwell, vol. 23(3), pages 433-461, July.
    14. Jean-Claude Berthélemy, 2019. "Challenges of decentralized electrification for economic development: lessons from experience," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) hal-02394467, HAL.
    15. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2020. "Controversies around RCT in Development. Epistemology, Ethics, and Politics," Working Papers DT/2020/15, DIAL (Développement, Institutions et Mondialisation).
    16. Martin Ravallion, 2009. "Evaluation in the Practice of Development," The World Bank Research Observer, World Bank, vol. 24(1), pages 29-53, March.
    17. Francis Pang & Mike Drummond & Fujian Song, 1999. "The use of meta-analysis in economic evaluation," Working Papers 173chedp, Centre for Health Economics, University of York.
    18. Martin Ravallion, 2018. "Should the Randomistas (Continue to) Rule?," Working Papers 492, Center for Global Development, revised 17 Jan 2019.
    19. Rodrik, Dani, 2008. "The New Development Economics: We Shall Experiment, but How Shall We Learn?," Working Paper Series rwp08-055, Harvard University, John F. Kennedy School of Government.
    20. Labandeira, Xavier & Labeaga, José M. & Linares, Pedro & López-Otero, Xiral, 2020. "The impacts of energy efficiency policies: Meta-analysis," Energy Policy, Elsevier, vol. 147(C).
    21. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009 is not listed on IDEAS
    22. Gertler, Paul J & Simone Boyce, 2003. "An Experiment in Incentive-Based Welfare: The Impact of PROGRESA on Health in Mexico," Royal Economic Society Annual Conference 2003 85, Royal Economic Society.
    23. Irani Arráiz & Carla Calero, 2015. "From Candles to Light: The Impact of Rural Electrification," IDB Publications (Working Papers) 89136, Inter-American Development Bank.
    24. repec:mpr:mprres:3694 is not listed on IDEAS
    25. Florent BEDECARRATS & Isabelle GUERIN & François ROUBAUD, 2017. "L'étalon-or des évaluations randomisées : économie politique des expérimentations aléatoires dans le domaine du développement," Working Paper 753120cd-506f-4c5f-80ed-7, Agence française de développement.
    26. Bédécarrats F (ed & Isabelle (ed.) Guérin & François (ed.) Roubaud, 2020. "Randomized control trials in the field of development : a critical perspective," Post-Print hal-03409465, HAL.
    27. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009.pdf is not listed on IDEAS
    28. Legovini, Arianna & Di Maro, Vincenzo & Piza, Caio, 2015. "Impact evaluation helps deliver development projects," Policy Research Working Paper Series 7157, The World Bank.
    29. Buddelmeyer, Hielke & Skoufias, Emmanuel, 2003. "An Evaluation of the Performance of Regression Discontinuity Design on PROGRESA," IZA Discussion Papers 827, Institute of Labor Economics (IZA).
    30. Corduneanu-Huci, Cristina & Dorsch, Michael T. & Maarek, Paul, 2021. "The politics of experimentation: Political competition and randomized controlled trials," Journal of Comparative Economics, Elsevier, vol. 49(1), pages 1-21.
    31. McKenzie, David, 2012. "Beyond baseline and follow-up: The case for more T in experiments," Journal of Development Economics, Elsevier, vol. 99(2), pages 210-221.
    32. Drew B. Cameron & Anjini Mishra & Annette N. Brown, 2016. "The growth of impact evaluation for international development: how much have we learned?," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 8(1), pages 1-21, March.
    33. repec:pri:rpdevs:instruments_of_development.pdf is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fatoumata Nankoto Cissé, 2022. "How impact evaluation methods influence the outcomes of development projects? Evidence from a meta-analysis on decentralized solar nano projects," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-03623394, HAL.
    2. Fatoumata Nankoto Cissé, 2022. "How impact evaluation methods influence the outcomes of development projects? Evidence from a meta-analysis on decentralized solar nano projects," Documents de travail du Centre d'Economie de la Sorbonne 22008, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
    3. Vivian C. Wong & Peter M. Steiner & Kylie L. Anglin, 2018. "What Can Be Learned From Empirical Evaluations of Nonexperimental Methods?," Evaluation Review, , vol. 42(2), pages 147-175, April.
    4. Tarek Azzam & Michael Bates & David Fairris, 2019. "Do Learning Communities Increase First Year College Retention? Testing Sample Selection and External Validity of Randomized Control Trials," Working Papers 202002, University of California at Riverside, Department of Economics.
    5. Cadot, Olivier & Fernandes, Ana M. & Gourdon, Julien & Mattoo, Aaditya, 2015. "Are the benefits of export support durable? Evidence from Tunisia," Journal of International Economics, Elsevier, vol. 97(2), pages 310-324.
    6. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2015. "The gold standard for randomized evaluations: from discussion of method to political economy," Working Papers DT/2015/01, DIAL (Développement, Institutions et Mondialisation).
    7. Katherine Baicker & Theodore Svoronos, 2019. "Testing the Validity of the Single Interrupted Time Series Design," NBER Working Papers 26080, National Bureau of Economic Research, Inc.
    8. Florent Bedecarrats & Isabelle Guérin & François Roubaud, 2017. "L'étalon-or des évaluations randomisées : du discours de la méthode à l'économie politique," Working Papers ird-01445209, HAL.
    9. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    10. Todd, Petra E., 2012. "Effectiveness of interventions aimed at improving women's employability and quality of work : a critical review," Policy Research Working Paper Series 6189, The World Bank.
    11. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    12. Daniel Zaga, 2014. "The Impact of Three Mexican Nutritional Programs: The Case of Dif-Puebla," CFD Working Papers 09-2015, Centre for Finance and Development, The Graduate Institute.
    13. Corduneanu-Huci, Cristina & Dorsch, Michael T. & Maarek, Paul, 2021. "The politics of experimentation: Political competition and randomized controlled trials," Journal of Comparative Economics, Elsevier, vol. 49(1), pages 1-21.
    14. Gustavo Canavire-Bacarreza & Luis Castro Peñarrieta & Darwin Ugarte Ontiveros, 2021. "Outliers in Semi-Parametric Estimation of Treatment Effects," Econometrics, MDPI, vol. 9(2), pages 1-32, April.
    15. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2019. "All that Glitters is not Gold. The Political Economy of Randomized Evaluations in Development," Development and Change, International Institute of Social Studies, vol. 50(3), pages 735-762, May.
    16. Mattoo, Aaditya & Cadot, Olivier & Gourdon, Julien & Fernandes, Ana Margarida, 2011. "Impact Evaluation of Trade Interventions: Paving the Way," CEPR Discussion Papers 8638, C.E.P.R. Discussion Papers.
    17. Yonatan Eyal, 2020. "Self-Assessment Variables as a Source of Information in the Evaluation of Intervention Programs: A Theoretical and Methodological Framework," SAGE Open, , vol. 10(1), pages 21582440198, January.
    18. Boris Salazar-Trujillo & Daniel Otero Robles, 2019. "La revolución empírica en economía," Apuntes del Cenes, Universidad Pedagógica y Tecnológica de Colombia, vol. 38(68), pages 15-48, July.
    19. Niño-Zarazúa, Miguel, 2012. "Quantitative analysis in social sciences: An brief introduction for non-economists," MPRA Paper 39216, University Library of Munich, Germany.
    20. Ralitza Dimova, 2019. "A Debate that Fatigues…: To Randomise or Not to Randomise; What’s the Real Question?," The European Journal of Development Research, Palgrave Macmillan;European Association of Development Research and Training Institutes (EADI), vol. 31(2), pages 163-168, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:halshs-03623394. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.