IDEAS home Printed from https://ideas.repec.org/p/cid/wpfacu/249.html
   My bibliography  Save this paper

It’s All About MeE: Using Structured Experiential Learning (‘e’) to Crawl the Design Space

Author

Listed:
  • Lant Pritchett

    (Center for International Development at Harvard University)

  • Salimah Samji

    (Center for International Development at Harvard University)

  • Jeffrey Hammer

Abstract

There is an inherent tension between implementing organizations—which have specific objectives and narrow missions and mandates—and executive organizations—which provide resources to multiple implementing organizations. Ministries of finance/planning/budgeting allocate across ministries and projects/programmes within ministries, development organizations allocate across sectors (and countries), foundations or philanthropies allocate across programmes/grantees. Implementing organizations typically try to do the best they can with the funds they have and attract more resources, while executive organizations have to decide what and who to fund. Monitoring and Evaluation (M&E) has always been an element of the accountability of implementing organizations to their funders. There has been a recent trend towards much greater rigor in evaluations to isolate causal impacts of projects and programmes and more ‘evidence base’ approaches to accountability and budget allocations. Here we extend the basic idea of rigorous impact evaluation—the use of a valid counter-factual to make judgments about causality—to emphasize that the techniques of impact evaluation can be directly useful to implementing organizations (as opposed to impact evaluation being seen by implementing organizations as only an external threat to their funding). We introduce structured experiential learning (which we add to M&E to get MeE) which allows implementing agencies to actively and rigorously search across alternative project designs using the monitoring data that provides real time performance information with direct feedback into the decision loops of project design and implementation. Our argument is that within-project variations in design can serve as their own counter-factual and this dramatically reduces the incremental cost of evaluation and increases the direct usefulness of evaluation to implementing agencies. The right combination of M, e, and E provides the right space for innovation and organizational capability building while at the same time providing accountability and an evidence base for funding agencies.

Suggested Citation

  • Lant Pritchett & Salimah Samji & Jeffrey Hammer, 2012. "It’s All About MeE: Using Structured Experiential Learning (‘e’) to Crawl the Design Space," CID Working Papers 249, Center for International Development at Harvard University.
  • Handle: RePEc:cid:wpfacu:249
    as

    Download full text from publisher

    File URL: https://www.hks.harvard.edu/sites/default/files/centers/cid/files/publications/faculty-working-papers/249.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Hammer, Jeffrey S, 1997. "Economic Analysis for Health Projects," The World Bank Research Observer, World Bank, vol. 12(1), pages 47-71, February.
    2. Baird, Sarah & McIntosh, Craig & Ozler, Berk, 2009. "Designing cost-effective cash transfer programs to boost schooling among young women in Sub-Saharan Africa," Policy Research Working Paper Series 5090, The World Bank.
    3. Ravallion Martin, 2015. "On the Implications of Essential Heterogeneity for Estimating Causal Impacts Using Social Experiments," Journal of Econometric Methods, De Gruyter, vol. 4(1), pages 145-151, January.
    4. Angus Deaton, 2009. "Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development," Working Papers 1128, Princeton University, Woodrow Wilson School of Public and International Affairs, Center for Health and Wellbeing..
    5. Elizabeth M. King & Jere R. Behrman, 2009. "Timing and Duration of Exposure in Evaluations of Social Programs," The World Bank Research Observer, World Bank, vol. 24(1), pages 55-82, February.
    6. Andrews, Matt & Pritchett, Lant & Woolcock, Michael, 2013. "Escaping Capability Traps Through Problem Driven Iterative Adaptation (PDIA)," World Development, Elsevier, vol. 51(C), pages 234-244.
    7. White, Howard, 2006. "Impact evaluation: the experience of the Independent Evaluation Group of the World Bank," MPRA Paper 1111, University Library of Munich, Germany.
    8. Bertrand, Marianne & Karlan, Dean S. & Mullainathan, Sendhil & Shafir, Eldar & Zinman, Jonathan, 2005. "What's Psychology Worth? A Field Experiment in the Consumer Credit Market," Center Discussion Papers 28441, Yale University, Economic Growth Center.
    9. Lant Pritchett & Michael Woolcock & Matt Andrews, 2013. "Looking Like a State: Techniques of Persistent Failure in State Capability for Implementation," Journal of Development Studies, Taylor & Francis Journals, vol. 49(1), pages 1-18, January.
    10. Abhijit Banerjee & Raghabendra Chattopadhyay & Esther Duflo & Daniel Keniston & Nina Singh, 2021. "Improving Police Performance in Rajasthan, India: Experimental Evidence on Incentives, Managerial Autonomy, and Training," American Economic Journal: Economic Policy, American Economic Association, vol. 13(1), pages 36-66, February.
    11. Henry Mintzberg & James A. Waters, 1985. "Of strategies, deliberate and emergent," Strategic Management Journal, Wiley Blackwell, vol. 6(3), pages 257-272, July.
    12. Acemoglu, Daron & Johnson, Simon & Robinson, James A., 2005. "Institutions as a Fundamental Cause of Long-Run Growth," Handbook of Economic Growth, in: Philippe Aghion & Steven Durlauf (ed.), Handbook of Economic Growth, edition 1, volume 1, chapter 6, pages 385-472, Elsevier.
    13. Pritchett, Lant & Woolcock, Michael, 2004. "Solutions When the Solution is the Problem: Arraying the Disarray in Development," World Development, Elsevier, vol. 32(2), pages 191-212, February.
    14. Denizer, Cevdet & Kaufmann, Daniel & Kraay, Aart, 2013. "Good countries or good projects? Macro and micro correlates of World Bank project performance," Journal of Development Economics, Elsevier, vol. 105(C), pages 288-302.
    15. Dani Rodrik, 2007. "Introductiion to One Economics, Many Recipes: Globalization, Institutions, and Economic Growth," Introductory Chapters, in: One Economics, Many Recipes: Globalization, Institutions, and Economic Growth, Princeton University Press.
    16. Dreze, Jean & Stern, Nicholas, 1987. "The theory of cost-benefit analysis," Handbook of Public Economics, in: A. J. Auerbach & M. Feldstein (ed.), Handbook of Public Economics, edition 1, volume 2, chapter 14, pages 909-989, Elsevier.
    17. Das, Jishnu & Hammer, Jeffrey, 2007. "Money for nothing: The dire straits of medical practice in Delhi, India," Journal of Development Economics, Elsevier, vol. 83(1), pages 1-36, May.
    18. repec:unu:wpaper:wp2012-64 is not listed on IDEAS
    19. Ricardo Hausmann, 2008. "The Other Hand: High Bandwidth Development Policy," CID Working Papers 179, Center for International Development at Harvard University.
    20. Joshua Angrist & Eric Bettinger & Erik Bloom & Elizabeth King & Michael Kremer, 2002. "Vouchers for Private Schooling in Colombia: Evidence from a Randomized Natural Experiment," American Economic Review, American Economic Association, vol. 92(5), pages 1535-1558, December.
    21. Alicia H. Munnell, 1987. "Lessons from the income maintenance experiments: an overview," New England Economic Review, Federal Reserve Bank of Boston, issue May, pages 32-45.
    22. Deon Filmer & Jeffrey S. Hammer & Lant H. Pritchett, 2002. "Weak Links in the Chain II: A Prescription for Health Policy in Poor Countries," The World Bank Research Observer, World Bank, vol. 17(1), pages 47-66.
    23. Miguel Szekely, 2011. "Toward Results-Based Social Policy Design and Implementation - Working Paper 249," Working Papers 249, Center for Global Development.
    24. Banerjee, Abhijit & Duflo, Esther & , & Keniston, Daniel & Singh, Nina, 2012. "Can Institutions Be Reformed from Within? Evidence from a Randomized Experiment with the Rajasthan Police," CEPR Discussion Papers 8869, C.E.P.R. Discussion Papers.
    25. Devarajan, Shantayanan & Squire, Lyn & Suthiwart-Narueput, Sethaput, 1997. "Beyond Rate of Return: Reorienting Project Appraisal," The World Bank Research Observer, World Bank, vol. 12(1), pages 35-46, February.
    26. Rodrik, Dani, 2008. "The New Development Economics: We Shall Experiment, but How Shall We Learn?," Working Paper Series rwp08-055, Harvard University, John F. Kennedy School of Government.
    27. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009 is not listed on IDEAS
    28. Spears, Dean, 2013. "Policy Lessons from the Implementation of India’s Total Sanitation Campaign," India Policy Forum, National Council of Applied Economic Research, vol. 9(1), pages 63-104.
    29. Sanjeev Khagram & Craig Thomas & Catrina Lucero & Subarna Mathes, 2009. "Evidence for development effectiveness," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 1(3), pages 247-270.
    30. Marianne Bertrand & Dean Karlan & Sendhil Mullainathan & Eldar Shafir & Jonathan Zinman, 2010. "What's Advertising Content Worth? Evidence from a Consumer Credit Marketing Field Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 263-306.
    31. Andrews, Matt & Pritchett, Lant & Woolcock, Michael, 2013. "Escaping Capability Traps Through Problem Driven Iterative Adaptation (PDIA)," World Development, Elsevier, vol. 51(C), pages 234-244.
    32. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009.pdf is not listed on IDEAS
    33. Jessica Cohen & Pascaline Dupas, 2010. "Free Distribution or Cost-Sharing? Evidence from a Randomized Malaria Prevention Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 1-45.
    34. Benjamin A. Olken, 2007. "Monitoring Corruption: Evidence from a Field Experiment in Indonesia," Journal of Political Economy, University of Chicago Press, vol. 115(2), pages 200-249.
    35. Felipe Barrera-Osorio & Deon Filmer, 2016. "Incentivizing Schooling for Learning: Evidence on the Impact of Alternative Targeting Approaches," Journal of Human Resources, University of Wisconsin Press, vol. 51(2), pages 461-499.
    36. Lant Pritchett, 2002. "It pays to be ignorant: A simple political economy of rigorous program evaluation," Journal of Economic Policy Reform, Taylor & Francis Journals, vol. 5(4), pages 251-269.
    37. repec:unu:wpaper:wp2012-63 is not listed on IDEAS
    38. Filmer, Deon & Hammer, Jeffrey S & Pritchett, Lant H, 2000. "Weak Links in the Chain: A Diagnosis of Health Policy in Poor Countries," The World Bank Research Observer, World Bank, vol. 15(2), pages 199-224, August.
    39. Alaka Holla & Michael Kremer, 2009. "Lessons from Randomized Evaluations in Education and Health," Working Papers 158, Center for Global Development.
    40. Joshua Angrist & Ivan Fernandez-Val, 2010. "ExtrapoLATE-ing: External Validity and Overidentification in the LATE Framework," NBER Working Papers 16566, National Bureau of Economic Research, Inc.
    41. Ostrom, Elinor, 1996. "Crossing the great divide: Coproduction, synergy, and development," World Development, Elsevier, vol. 24(6), pages 1073-1087, June.
    42. Robert Jensen, 2010. "The (Perceived) Returns to Education and the Demand for Schooling," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(2), pages 515-548.
    43. Lant Pritchett, Michael Woolcock, Matt Andrews, 2010. "Capability Traps? The Mechanisms of Persistent Implementation Failure - Working Paper 234," Working Papers 234, Center for Global Development.
    44. repec:pri:rpdevs:instruments_of_development.pdf is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Farah Hani & Miguel Angel Santos, 2021. "Diagnosing Human Capital as a Binding Constraint to Growth: Tests, Symptoms and Prescriptions," Growth Lab Working Papers 168, Harvard's Growth Lab.
    2. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2019. "All that Glitters is not Gold. The Political Economy of Randomized Evaluations in Development," Development and Change, International Institute of Social Studies, vol. 50(3), pages 735-762, May.
    3. Arkedis, Jean & Creighton, Jessica & Dixit, Akshay & Fung, Archon & Kosack, Stephen & Levy, Dan & Tolmie, Courtney, 2021. "Can transparency and accountability programs improve health? Experimental evidence from Indonesia and Tanzania," World Development, Elsevier, vol. 142(C).
    4. Hammer, Jeffrey & Spears, Dean, 2013. "Village sanitation and children's human capital : evidence from a randomized experiment by the Maharashtra government," Policy Research Working Paper Series 6580, The World Bank.
    5. Dan Levy, 2019. "Can Transparency and Accountability Programs Improve Health? Experimental Evidence from Indonesia and Tanzania," CID Working Papers 352, Center for International Development at Harvard University.
    6. Woolcock, Michael, 2013. "Using Case Studies to Explore the External Validity of 'Complex' Development Interventions," Working Paper Series rwp13-048, Harvard University, John F. Kennedy School of Government.
    7. Florent BEDECARRATS & Isabelle GUERIN & François ROUBAUD, 2017. "L'étalon-or des évaluations randomisées : économie politique des expérimentations aléatoires dans le domaine du développement," Working Paper 753120cd-506f-4c5f-80ed-7, Agence française de développement.
    8. Janssen, Matthijs J., 2019. "What bangs for your buck? Assessing the design and impact of Dutch transformative policy," Technological Forecasting and Social Change, Elsevier, vol. 138(C), pages 78-94.
    9. Woolcock, Michael, 2013. "Using Case Studies to Explore the External Validity of 'Complex' Development Interventions," Working Paper Series rwp13-048, Harvard University, John F. Kennedy School of Government.
    10. repec:pri:cheawb:tscjeff2013%20paper is not listed on IDEAS
    11. Holvoet, Nathalie & Van Esbroeck, Dirk & Inberg, Liesbeth & Popelier, Lisa & Peeters, Bob & Verhofstadt, Ellen, 2018. "To evaluate or not: Evaluability study of 40 interventions of Belgian development cooperation," Evaluation and Program Planning, Elsevier, vol. 67(C), pages 189-199.
    12. Holvoet, Nathalie & Casten, Wanda & Demissie, Eshetu Woldeyohannes & Dewachter, Sara & Gamboa, Marian Kaye C. & Adhanom, Tewelde Gebremariam & Ibrahim, Abdurahman Hamza & Makundi, Hezron & Manguni, Gr, 2023. "Theory-based evaluation of the impact of Master’s programmes in development studies: Insights from a mixed-methods and multicultural alumni action research project," Evaluation and Program Planning, Elsevier, vol. 97(C).
    13. Eduardo Levy Yeyati, 2019. "What Works for Active Labor Market Policies?," CID Working Papers 358, Center for International Development at Harvard University.
    14. Clive Bell & Lyn Squire, 2017. "Providing Policy Makers with Timely Advice: The Timeliness-Rigor Trade-off," The World Bank Economic Review, World Bank, vol. 31(2), pages 553-569.
    15. Matthijs Janssen, 2016. "What bangs for your bucks? Assessing the design and impact of transformative policy," Innovation Studies Utrecht (ISU) working paper series 16-05, Utrecht University, Department of Innovation Studies, revised Dec 2016.
    16. Arkedis, Jean & Creighton, Jessica & Dixit, Akshay & Fung, Archon & Kosack, Stephen & Levy, Dan & Tolmie, Courtney, 2019. "Can Transparency and Accountability Programs Improve Health? Experimental Evidence from Indonesia and Tanzania," Working Paper Series rwp19-020, Harvard University, John F. Kennedy School of Government.
    17. Nadel, Sara & Pritchett, Lant, 2016. "Searching for the Devil in the Details: Learning about Development Program Design," Working Paper Series rwp16-041, Harvard University, John F. Kennedy School of Government.
    18. Cameron, Lisa & Olivia, Susan & Shah, Manisha, 2019. "Scaling up sanitation: Evidence from an RCT in Indonesia," Journal of Development Economics, Elsevier, vol. 138(C), pages 1-16.
    19. Jorge Cornick & Alberto Trejos, 2016. "Building Public Capabilities for Productive Development Policies: Costa Rican Case Studies," IDB Publications (Working Papers) 96956, Inter-American Development Bank.
    20. Cornick, Jorge & Trejos, Alberto, 2016. "Building Public Capabilities for Productive Development Policies: Costa Rican Case Studies," IDB Publications (Working Papers) 8017, Inter-American Development Bank.
    21. Michael Clemens, Gabriel Demombynes, 2013. "The New Transparency in Development Economics: Lessons from the Millennium Villages Controversy," Working Papers 342, Center for Global Development.
    22. Ilia Murtazashvili & Jennifer Murtazashvili, 2019. "The political economy of legal titling," The Review of Austrian Economics, Springer;Society for the Development of Austrian Economics, vol. 32(3), pages 251-268, September.
    23. Lant Pritchett, Justin Sandefur, 2013. "Context Matters for Size: Why External Validity Claims and Development Practice Don't Mix-Working Paper 336," Working Papers 336, Center for Global Development.
    24. Sara Nadel and Lant Pritchett, 2016. "Searching for the Devil in the Details: Learning about Development Program Design," Working Papers 434, Center for Global Development.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lant Pritchett & Salimah Samji & Jeffrey Hammer, 2012. "It’s All About MeE: Using Structured Experiential Learning (‘e’) to Crawl the Design Space," CID Working Papers 249, Center for International Development at Harvard University.
    2. repec:pri:rpdevs:hammer_its_all_about_me is not listed on IDEAS
    3. Andrews, Matt & Pritchett, Lant & Woolcock, Michael, 2017. "Building State Capability: Evidence, Analysis, Action," OUP Catalogue, Oxford University Press, number 9780198747482.
    4. William Easterly, 2009. "Can the West Save Africa?," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 373-447, June.
    5. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
    6. Sara Nadel and Lant Pritchett, 2016. "Searching for the Devil in the Details: Learning about Development Program Design," Working Papers 434, Center for Global Development.
    7. Matt Andrews & Lant Pritchett & Michael Woolcock, 2015. "The Challenge of Building (Real) State Capability," CID Working Papers 306, Center for International Development at Harvard University.
    8. B Kelsey Jack, "undated". "Market Inefficiencies and the Adoption of Agricultural Technologies in Developing Countries," CID Working Papers 50, Center for International Development at Harvard University.
    9. Lant Pritchett & Michael Woolcock & Matt Andrews, 2013. "Looking Like a State: Techniques of Persistent Failure in State Capability for Implementation," Journal of Development Studies, Taylor & Francis Journals, vol. 49(1), pages 1-18, January.
    10. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    11. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    12. Lant Pritchett & Michael Woolcock & Matt Andrews, 2013. "Looking Like a State: Techniques of Persistent Failure in State Capability for Implementation," Journal of Development Studies, Taylor & Francis Journals, vol. 49(1), pages 1-18, January.
    13. repec:unu:wpaper:wp2012-63 is not listed on IDEAS
    14. Larson, Greg & Ajak, Peter Biar & Pritchett, Lant, 2013. "South Sudan's Capability Trap: Building a State with Disruptive Innovation," Working Paper Series rwp13-041, Harvard University, John F. Kennedy School of Government.
    15. Karthik Muralidharan & Paul Niehaus, 2017. "Experimentation at Scale," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 103-124, Fall.
    16. Karthik Muralidharan & Mauricio Romero & Kaspar Wüthrich, 2019. "Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments," NBER Working Papers 26562, National Bureau of Economic Research, Inc.
    17. Greg Larson & Peter Biar Ajak & Lant Pritchett, 2013. "South Sudan’s Capability Trap: Building a State with Disruptive Innovation," CID Working Papers 268, Center for International Development at Harvard University.
    18. Andrews, Matt & Pritchett, Lant & Woolcock, Michael, 2015. "Doing Problem Driven Work," Working Paper Series 15-073, Harvard University, John F. Kennedy School of Government.
    19. Damgaard, Mette Trier & Nielsen, Helena Skyt, 2018. "Nudging in education," Economics of Education Review, Elsevier, vol. 64(C), pages 313-342.
    20. Andrews, Matt & Pritchett, Lant & Woolcock, Michael, 2013. "Escaping Capability Traps Through Problem Driven Iterative Adaptation (PDIA)," World Development, Elsevier, vol. 51(C), pages 234-244.
    21. Frauke de Weijer, 2013. "A Capable State in Afghanistan: a Building Without a Foundation?," WIDER Working Paper Series wp-2013-063, World Institute for Development Economic Research (UNU-WIDER).
    22. repec:unu:wpaper:wp2012-64 is not listed on IDEAS
    23. Roland G. Fryer, Jr, 2010. "Financial Incentives and Student Achievement: Evidence from Randomized Trials," NBER Working Papers 15898, National Bureau of Economic Research, Inc.

    More about this item

    Keywords

    Evaluation; Monitoring; Learning; Experimentation; Implementation; Feedback Loops;
    All these keywords.

    JEL classification:

    • D61 - Microeconomics - - Welfare Economics - - - Allocative Efficiency; Cost-Benefit Analysis
    • D78 - Microeconomics - - Analysis of Collective Decision-Making - - - Positive Analysis of Policy Formulation and Implementation
    • H11 - Public Economics - - Structure and Scope of Government - - - Structure and Scope of Government
    • C40 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cid:wpfacu:249. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chuck McKenney (email available below). General contact details of provider: https://edirc.repec.org/data/ciharus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.