IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/47993.html
   My bibliography  Save this paper

Systematic Reviews: Questions, Methods and Usage

Author

Listed:
  • Hansen, Henrik
  • Trifkovic, Neda

Abstract

While their application in the context of development aid is quite new, systematic reviews have been used and debated in other fields for various decades. Although evidence gathering and synthesis in the context of development aid inarguably faces challenges distinct from other fields, it is expected that experiences about systematic review methodology from other contexts can provide useful inputs to the field of development aid evaluation. Systematic reviews operate in a complex multidisciplinary environment, which requires acknowledging the influence of institutions and social interaction. The Evaluation Study suggests that the scarcity of comparable evidence about the effects of development interventions necessitates that authors of systematic reviews change their strategies when assessing the strength of evidence or synthesizing data. The focus on asking the ‘right’ questions in international development reviews is important precisely because no review process is immune to bias. The study emphasizes that since systematic reviews in international development may be vulnerable to a range of biases. Systematic reviews should not aim, at all cost, at pursuing the classical approach suitable for traditional, ‘easy-to-measure’ situations.

Suggested Citation

  • Hansen, Henrik & Trifkovic, Neda, 2013. "Systematic Reviews: Questions, Methods and Usage," MPRA Paper 47993, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:47993
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/47993/1/MPRA_paper_47993.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Birte Snilstveit & Sandy Oliver & Martina Vojtkova, 2012. "Narrative approaches to systematic review and synthesis of evidence for international development policy and practice," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 4(3), pages 409-429, September.
    2. Hugh Waddington & Howard White & Birte Snilstveit & Jorge Garcia Hombrados & Martina Vojtkova & Philip Davies & Ami Bhavsar & John Eyers & Tracey Perez Koehlmoos & Mark Petticrew & Jeffrey C. Valentin, 2012. "How to do a good systematic review of effects in international development: a tool kit," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 4(3), pages 359-387, September.
    3. David Moher & Jennifer Tetzlaff & Andrea C Tricco & Margaret Sampson & Douglas G Altman, 2007. "Epidemiology and Reporting Characteristics of Systematic Reviews," PLOS Medicine, Public Library of Science, vol. 4(3), pages 1-9, March.
    4. Greenhalgh, Trisha & Robert, Glenn & Macfarlane, Fraser & Bate, Paul & Kyriakidou, Olympia & Peacock, Richard, 2005. "Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review," Social Science & Medicine, Elsevier, vol. 61(2), pages 417-430, July.
    5. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    6. George Teeling Smith;Nick Wells, 1979. "Medicines for the year 2000," Monograph 000301, Office of Health Economics.
    7. Campbell, Rona & Pound, Pandora & Pope, Catherine & Britten, Nicky & Pill, Roisin & Morgan, Myfanwy & Donovan, Jenny, 2003. "Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care," Social Science & Medicine, Elsevier, vol. 56(4), pages 671-684, February.
    8. Maren Duvendack & Jorge Garcia Hombrados & Richard Palmer-Jones & Hugh Waddington, 2012. "Assessing ‘what works’ in international development: meta-analysis for sophisticated dummies," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 4(3), pages 456-471, September.
    9. Ruth Stewart & Carina van Rooyen & Thea de Wet, 2012. "Purity or pragmatism? Reflecting on the use of systematic review methodology in development," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 4(3), pages 430-444, September.
    10. Hilda Bastian & Paul Glasziou & Iain Chalmers, 2010. "Seventy-Five Trials and Eleven Systematic Reviews a Day: How Will We Ever Keep Up?," PLOS Medicine, Public Library of Science, vol. 7(9), pages 1-6, September.
    11. Howard White & Hugh Waddington, 2012. "Why do we care about evidence synthesis? An introduction to the special issue on systematic reviews," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 4(3), pages 351-358, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Laurenz Langer & Ruth Stewart, 2014. "What have we learned from the application of systematic review methodology in international development? - a thematic overview," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(3), pages 236-248, September.
    2. Natalie Rebelo Da Silva & Hazel Zaranyika & Laurenz Langer & Nicola Randall & Evans Muchiri & Ruth Stewart, 2017. "Making the Most of What We Already Know," Evaluation Review, , vol. 41(2), pages 155-172, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Túlio A. Cravo & Caio Piza, 2019. "The impact of business-support services on firm performance: a meta-analysis," Small Business Economics, Springer, vol. 53(3), pages 753-770, October.
    2. Laura Sheble, 2017. "Macro‐level diffusion of a methodological knowledge innovation: Research synthesis methods, 1972–2011," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(12), pages 2693-2708, December.
    3. repec:wly:camsys:v:10:y:2014:i:1:p:1-46:b is not listed on IDEAS
    4. Lijuan Du & Li Xu & Yanping Li & Changshun Liu & Zhenhua Li & Jefferson S. Wong & Bo Lei, 2019. "China’s Agricultural Irrigation and Water Conservancy Projects: A Policy Synthesis and Discussion of Emerging Issues," Sustainability, MDPI, vol. 11(24), pages 1-20, December.
    5. Johnston, Deborah & Stevano, Sara & Malapit, Hazel J. & Hull, Elizabeth & Kadiyala, Suneetha, 2018. "Review: Time Use as an Explanation for the Agri-Nutrition Disconnect: Evidence from Rural Areas in Low and Middle-Income Countries," Food Policy, Elsevier, vol. 76(C), pages 8-18.
    6. Maren Duvendack, 2022. "Payment‐by‐results for health interventions in low‐ and middle‐income countries: A critical review," Development Policy Review, Overseas Development Institute, vol. 40(1), January.
    7. Davidson, Angus Alexander & Young, Michael Denis & Leake, John Espie & O’Connor, Patrick, 2022. "Aid and forgetting the enemy: A systematic review of the unintended consequences of international development in fragile and conflict-affected situations," Evaluation and Program Planning, Elsevier, vol. 92(C).
    8. Lucy Turner & James Galipeau & Chantelle Garritty & Eric Manheimer & L Susan Wieland & Fatemeh Yazdi & David Moher, 2013. "An Evaluation of Epidemiological and Reporting Characteristics of Complementary and Alternative Medicine (CAM) Systematic Reviews (SRs)," PLOS ONE, Public Library of Science, vol. 8(1), pages 1-11, January.
    9. Graham McDowell & Eleanor Stephenson & James Ford, 2014. "Adaptation to climate change in glaciated mountain regions," Climatic Change, Springer, vol. 126(1), pages 77-91, September.
    10. Jessie Pullar & Luke Allen & Nick Townsend & Julianne Williams & Charlie Foster & Nia Roberts & Mike Rayner & Bente Mikkelsen & Francesco Branca & Kremlin Wickramasinghe, 2018. "The impact of poverty reduction and development interventions on non-communicable diseases and their behavioural risk factors in low and lower-middle income countries: A systematic review," PLOS ONE, Public Library of Science, vol. 13(2), pages 1-27, February.
    11. Porjai Pattanittum & Malinee Laopaiboon & David Moher & Pisake Lumbiganon & Chetta Ngamjarus, 2012. "A Comparison of Statistical Methods for Identifying Out-of-Date Systematic Reviews," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-6, November.
    12. Emmy De Buck & Karin Hannes & Hans Van Remoortel & Thashlin Govender & Axel Vande Veegaete & Alfred Musekiwa & Vittoria Lutje & Margaret Cargo & Hans‐Joachim Mosler & Philippe Vandekerckhove & Taryn Y, 2016. "PROTOCOL: Approaches to Promote Handwashing and Sanitation Behaviour Change in Low‐ and Middle Income Countries: A Mixed Method Systematic Review," Campbell Systematic Reviews, John Wiley & Sons, vol. 12(1), pages 1-46.
    13. Johnston, Deborah & Stevano, Sara & Malapit, Hazel J. & Hull, Elizabeth & Kadiyala, Suneetha, 2015. "Agriculture, gendered time use, and nutritional outcomes: A systematic review," IFPRI discussion papers 1456, International Food Policy Research Institute (IFPRI).
    14. Vicki-Ann Ware & Kim Dunphy, 2019. "Methodological Practices in Research on Arts-Based Programs in International Development: A Systematic Review," The European Journal of Development Research, Palgrave Macmillan;European Association of Development Research and Training Institutes (EADI), vol. 31(3), pages 480-503, July.
    15. Julius Sim & Anne Marit Mengshoel, 2023. "Metasynthesis: issues of empirical and theoretical context," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(4), pages 3339-3361, August.
    16. Etienne Lwamba & Will Ridlehoover & Meital Kupfer & Shannon Shisler & Ada Sonnenfeld & Laurenz Langer & John Eyers & Sean Grant & Bidisha Barooah, 2021. "PROTOCOL: Strengthening women's empowerment and gender equality in fragile contexts towards peaceful and inclusive societies: A systematic review and meta‐analysis," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(3), September.
    17. Eduardo Fernández-Arias & Ricardo Hausmann & Ugo Panizza, 2020. "Smart Development Banks," Journal of Industry, Competition and Trade, Springer, vol. 20(2), pages 395-420, June.
    18. Ashish Arora & Michelle Gittelman & Sarah Kaplan & John Lynch & Will Mitchell & Nicolaj Siggelkow & Aaron K. Chatterji & Michael Findley & Nathan M. Jensen & Stephan Meier & Daniel Nielson, 2016. "Field experiments in strategy research," Strategic Management Journal, Wiley Blackwell, vol. 37(1), pages 116-132, January.
    19. Patrick Guillaumont, 2011. "Aid effectiveness for poverty reduction:macroeconomic overview and emerging issues," CERDI Working papers halshs-00554285, HAL.
    20. Neal R. Haddaway & Matthew J. Page & Chris C. Pritchard & Luke A. McGuinness, 2022. "PRISMA2020: An R package and Shiny app for producing PRISMA 2020‐compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(2), June.
    21. Carattini, Stefano & Gillingham, Kenneth & Meng, Xiangyu & Yoeli, Erez, 2024. "Peer-to-peer solar and social rewards: Evidence from a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 219(C), pages 340-370.

    More about this item

    Keywords

    Systetmatic Reviews; International Development;

    JEL classification:

    • C0 - Mathematical and Quantitative Methods - - General
    • O20 - Economic Development, Innovation, Technological Change, and Growth - - Development Planning and Policy - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:47993. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.