IDEAS home Printed from https://ideas.repec.org/a/taf/jdevef/v10y2018i4p400-421.html
   My bibliography  Save this article

Mixing and matching: using qualitative methods to improve quantitative impact evaluations (IEs) and systematic reviews (SRs) of development outcomes

Author

Listed:
  • Emmanuel Jimenez
  • Hugh Waddington
  • Neeta Goel
  • Audrey Prost
  • Andrew Pullin
  • Howard White
  • Shaon Lahiri
  • Anmol Narain

Abstract

Recent evaluations have begun to use qualitative data in a manner that helps improve the quality and relevance of studies through the inferences that are drawn from them, and their applicability to policy makers and programme implementers. This paper reviews this work and identifies good practices to integrate qualitative methods into quantitative impact evaluations (IEs) and systematic reviews (SRs). Using recent literature on the characteristics of such practices, we developed two tools to assess the methodological rigour and mixed methods integration of 40 IEs and 7 SRs, drawing upon previous approaches. Our findings are that successful mixed methods quantitative impact evaluations: (1) provide a clear rationale for integration of methods; (2) deploy multidisciplinary teams; (3) provide adequate documentation; and (4) acknowledge limitations to the generalisability of qualitative and quantitative findings. Successful integration tended to improve mixed methods impact evaluations by collecting better data to inform the study design and findings, which helped contextualise quantitative findings. Our main observation on the integration of mixed methods in the systematic reviews is that mixed methods systematic reviews bringing together literatures that answer different questions can go beyond the ‘sum of their parts’ to provide holistic answers about development effectiveness. The findings of this study inform several recommendations to improve the conduct and reporting of mixed methods impact evaluations and systematic reviews.

Suggested Citation

  • Emmanuel Jimenez & Hugh Waddington & Neeta Goel & Audrey Prost & Andrew Pullin & Howard White & Shaon Lahiri & Anmol Narain, 2018. "Mixing and matching: using qualitative methods to improve quantitative impact evaluations (IEs) and systematic reviews (SRs) of development outcomes," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 10(4), pages 400-421, October.
    • Handle: RePEc:taf:jdevef:v:10:y:2018:i:4:p:400-421
    DOI: 10.1080/19439342.2018.1534875
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/19439342.2018.1534875
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/19439342.2018.1534875?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Anita Rizvi & Vivian Welch & Marcia Gibson & Patrick R. Labelle & Christina Pollard & George A. Wells & Elizabeth Kristjansson, 2022. "PROTOCOL: Effects of guaranteed basic income interventions on poverty‐related outcomes in high‐income countries: A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(4), December.
    2. Hugh Sharma Waddington & Sandy Cairncross, 2021. "PROTOCOL: Water, sanitation and hygiene for reducing childhood mortality in low‐ and middle‐income countries," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(1), March.
    3. Erik Christopher Simmons & Matthew Sanders, 2022. "Building sustainable communities for sustainable development: An evidence‐based behavior change intervention to reduce plastic waste and destructive fishing in Southeast Asia," Sustainable Development, John Wiley & Sons, Ltd., vol. 30(5), pages 1018-1029, October.
    4. Áine Aventin & Martin Robinson & Jennifer Hanratty & Eimear Ruane‐McAteer & Mark Tomlinson & Mike Clarke & Friday Okonofua & Chris Bonell & Maria Lohan, 2021. "PROTOCOL: Involving men and boys in family planning: A systematic review of the effective components and characteristics of complex interventions in low‐ and middle‐income countries," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(1), March.
    5. Marie Gaarder, 2019. "A Commentary to ‘Bridging to Action Requires Mixed Methods, Not Only Randomised Control Trials’," The European Journal of Development Research, Palgrave Macmillan;European Association of Development Research and Training Institutes (EADI), vol. 31(2), pages 169-173, April.
    6. Monica Jain & Mark Engelbert & Marie Gaarder & Avantika Bagai & John Eyers, 2020. "PROTOCOL: Use of community participation interventions to improve child immunisation in low‐ and middle‐income countries: A systematic review and meta‐analysis," Campbell Systematic Reviews, John Wiley & Sons, vol. 16(4), December.
    7. Áine Aventin & Martin Robinson & Jennifer Hanratty & Ciara Keenan & Jayne Hamilton & Eimear Ruane McAteer & Mark Tomlinson & Mike Clarke & Friday Okonofua & Chris Bonell & Maria Lohan, 2023. "Involving men and boys in family planning: A systematic review of the effective components and characteristics of complex interventions in low‐ and middle‐income countries," Campbell Systematic Reviews, John Wiley & Sons, vol. 19(1), March.
    8. Anita Rizvi & Madeleine Kearns & Michael Dignam & Alison Coates & Melissa K. Sharp & Olivia Magwood & Patrick R. Labelle & Nour Elmestekawy & Sydney Rossiter & Ali A. A. Al‐Zubaidi & Omar Dewidar & Le, 2024. "Effects of guaranteed basic income interventions on poverty‐related outcomes in high‐income countries: A systematic review and meta‐analysis," Campbell Systematic Reviews, John Wiley & Sons, vol. 20(2), June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jdevef:v:10:y:2018:i:4:p:400-421. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/RJDE20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.