IDEAS home Printed from https://ideas.repec.org/a/zbw/espost/268010.html
   My bibliography  Save this article

Case study meta‐analysis in the social sciences. Insights on data quality and reliability from a large‐N case survey

Author

Listed:
  • Jager, Nicolas W.
  • Newig, Jens
  • Challies, Edward
  • Kochskämper, Elisa
  • von Wehrden, Henrik

Abstract

Meta-analytical methods face particular challenges in research fields such as social and political research, where studies often rest primarily on qualitative and case study research. In such contexts, where research findings are less standardized and amenable to structured synthesis, the case survey method has been proposed as a means of data generation and analysis. The method offers a meta-analytical tool to synthesize larger numbers of qualitative case studies, yielding data amenable to large-N analysis. However, resulting data is prone to specific threats to validity, including biases due to publication type, rater behaviour, and variable characteristics, which researchers need to be aware of. While these biases are well known in theory, and typically explored for primary research, their prevalence in case survey meta-analyses remains relatively unexplored. We draw on a case survey of 305 published qualitative case studies of public environmental decision-making, and systematically analyze these biases in the resultant data. Our findings indicate that case surveys can deliver high-quality and reliable results. However, we also find that these biases do indeed occur, albeit to a small degree or under specific conditions of complexity. We identify a number of design choices to mitigate biases that may threaten validity in case survey meta-analysis. Our findings are of importance to those using the case survey method – and to those who might apply insights derived by this method to inform policy and practice.

Suggested Citation

  • Jager, Nicolas W. & Newig, Jens & Challies, Edward & Kochskämper, Elisa & von Wehrden, Henrik, 2022. "Case study meta‐analysis in the social sciences. Insights on data quality and reliability from a large‐N case survey," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 13(1), pages 12-27.
  • Handle: RePEc:zbw:espost:268010
    DOI: 10.1002/jrsm.1514
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/268010/1/Research%20Synthesis%20Methods%20-%202021%20-%20Jager%20-%20Case%20study%20meta%e2%80%90analysis%20in%20the%20social%20sciences%20Insights%20on%20data%20quality%20and.pdf
    Download Restriction: no

    File URL: https://libkey.io/10.1002/jrsm.1514?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:espost:268010. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/zbwkide.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.