IDEAS home Printed from https://ideas.repec.org/a/taf/jdevef/v4y2012i3p388-408.html
   My bibliography  Save this article

Systematic reviews: from ‘bare bones’ reviews to policy relevance

Author

Listed:
  • Birte Snilstveit

Abstract

Theory-based systematic reviews, which summarise evidence on what works, when and why, strive for greater policy relevance. Reviews that answer these questions adopt a mixed methods approach and draw on a range of study types. Answering the ‘what works’ and ‘what doesn't’ questions means drawing on effectiveness studies, conducted to standards of high-quality impact evaluation. But in formulating answers to the ‘when’ and ‘why’ questions requires a broader range of evidence from both quantitative and qualitative research. Based on a review of the methodological literature in this field and the experience of 3ie's systematic reviews program, this article provides an outline of how a theory-based approach to systematic reviews, including appropriate quantitative and qualitative evidence, can be operationalised. We propose an approach based on three principles: (1) develop a program theory for the intervention; (2) adopt a mixed methods approach and include a broader range of evidence; and (3) maintain the rigour and transparency that characterise systematic reviews. The approach translates into two broad options. Effectiveness plus reviews focus on providing a detailed causal chain analysis by drawing on a program theory and additional data collection on context and intervention implementation. And effectiveness plus with a parallel review component, which is designed to answer specific research questions related to effectiveness, adopts separate inclusion criteria, reflecting the type of studies appropriate for answering those questions.

Suggested Citation

  • Birte Snilstveit, 2012. "Systematic reviews: from ‘bare bones’ reviews to policy relevance," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 4(3), pages 388-408, September.
    • Handle: RePEc:taf:jdevef:v:4:y:2012:i:3:p:388-408
    DOI: 10.1080/19439342.2012.709875
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/19439342.2012.709875
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/19439342.2012.709875?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Howard White, 2009. "Theory-based impact evaluation: principles and practice," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 1(3), pages 271-284.
    2. White, Howard, 2009. "Theory-Based Impact Evaluation," 3ie Publications 2009-3, International Initiative for Impact Evaluation (3ie).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. repec:wly:camsys:v:10:y:2014:i:1:p:1-46:a is not listed on IDEAS
    2. Natalie Rebelo Da Silva & Hazel Zaranyika & Laurenz Langer & Nicola Randall & Evans Muchiri & Ruth Stewart, 2017. "Making the Most of What We Already Know," Evaluation Review, , vol. 41(2), pages 155-172, April.
    3. Constanza Gonzalez Parrao & Marta Moratti & Shannon Shisler & Birte Snilstveit & John Eyers, 2021. "PROTOCOL: Aquaculture for improving productivity, income, nutrition and women's empowerment in low‐ and middle‐income countries: A systematic review and meta‐analysis," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(3), September.
    4. Sara Stevano & Suneetha Kadiyala & Deborah Johnston & Hazel Malapit & Elizabeth Hull & Sofia Kalamatianou, 2019. "Time-Use Analytics: An Improved Way of Understanding Gendered Agriculture-Nutrition Pathways," Feminist Economics, Taylor & Francis Journals, vol. 25(3), pages 1-22, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Johnson, Nancy L. & Atherstone, Christine & Grace, Delia, 2015. "The potential of farm-level technologies and practices to contribute to reducing consumer exposure to aflatoxins: A theory of change analysis," IFPRI discussion papers 1452, International Food Policy Research Institute (IFPRI).
    2. Samuel G Schumacher & Hojoon Sohn & Zhi Zhen Qin & Genevieve Gore & J Lucian Davis & Claudia M Denkinger & Madhukar Pai, 2016. "Impact of Molecular Diagnostics for Tuberculosis on Patient-Important Outcomes: A Systematic Review of Study Methodologies," PLOS ONE, Public Library of Science, vol. 11(3), pages 1-21, March.
    3. Sara Rafael Almeida & Joana Sousa Lourenco & Francois J. Dessart & Emanuele Ciriolo, 2017. "Insights from behavioural sciences to prevent and combat violence against women. Literature review," JRC Research Reports JRC103975, Joint Research Centre.
    4. Quentin Ssossé & Johanna Wagner & Carina Hopper, 2021. "Assessing the Impact of ESD: Methods, Challenges, Results," Sustainability, MDPI, vol. 13(5), pages 1-26, March.
    5. Joana Silva Afonso, 2020. "Impact evaluation, social performance assessment and standardisation: reflections from microfinance evaluations in Pakistan and Zimbabwe," Working Papers in Economics & Finance 2020-14, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.
    6. Gwenolé Le Velly & Céline Dutilly, 2016. "Evaluating Payments for Environmental Services: Methodological Challenges," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-20, February.
    7. Calina-Ana Butiu, 2017. "Evidence based practice in academic dropout policy. The pro-integra model," Journal of Community Positive Practices, Catalactica NGO, issue 1, pages 3-12.
    8. Pablo Vidueira & José M. Díaz-Puente & María Rivera, 2014. "Socioeconomic Impact Assessment in Ex Ante Evaluations," Evaluation Review, , vol. 38(4), pages 309-335, August.
    9. Yonatan Eyal, 2020. "Self-Assessment Variables as a Source of Information in the Evaluation of Intervention Programs: A Theoretical and Methodological Framework," SAGE Open, , vol. 10(1), pages 21582440198, January.
    10. Gala D�az Langou & Vanesa Weyrauch, 2013. "Sound expectations: from impact evaluations to policy change," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 5(3), pages 269-304, September.
    11. Hughes, Karl & Morgan, Seth & Baylis, Katherine & Oduol, Judith & Smith-Dumont, Emilie & Vågen, Tor-Gunnar & Kegode, Hilda, 2020. "Assessing the downstream socioeconomic impacts of agroforestry in Kenya," World Development, Elsevier, vol. 128(C).
    12. Isnawati Hidayah & Imam Mukhlis, 2019. "Dana Desa on clean water and sanitation access in Indonesia: Does Cash-for-work (PKT) matter?," Proceedings of Economics and Finance Conferences 9512011, International Institute of Social and Economic Sciences.
    13. John Gaventa & Rosemary McGee, 2013. "The Impact of Transparency and Accountability Initiatives," Development Policy Review, Overseas Development Institute, vol. 31, pages 3-28, July.
    14. Varghese, Rekha & Roy, Manan, 2019. "Coresidence with mother-in-law and maternal anemia in rural India," Social Science & Medicine, Elsevier, vol. 226(C), pages 37-46.
    15. Hansen, Marc, 2013. "Aid's Incomplete Trial: reconsidering the aid-growth paradigm in the macro aid effectiveness literature," IEE Working Papers 200, Ruhr University Bochum, Institute of Development Research and Development Policy (IEE).
    16. Gaarder, Marie M. & Bartsch, Ulrich, 2014. "The second wave of independence : shopping for solutions," Policy Research Working Paper Series 7069, The World Bank.
    17. Garbero, A., 2016. "IFAD RESEARCH SERIES 7 - Measuring IFAD’s impact: background paper to the IFAD9 Impact Assessment Initiative," IFAD Research Series 280045, International Fund for Agricultural Development (IFAD).
    18. Howard White, 2013. "An introduction to the use of randomised control trials to evaluate development interventions," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 5(1), pages 30-49, March.
    19. Rasheed, Sulaiman V. & Hall, Andy & Reddy, T.S. Vamsidhar, 2011. "Missing the target: Lessons from enabling innovation in South Asia," MERIT Working Papers 2011-050, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    20. Dawson, Neil & Martin, Adrian & Sikor, Thomas, 2016. "Green Revolution in Sub-Saharan Africa: Implications of Imposed Innovation for the Wellbeing of Rural Smallholders," World Development, Elsevier, vol. 78(C), pages 204-218.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jdevef:v:4:y:2012:i:3:p:388-408. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/RJDE20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.