IDEAS home Printed from https://ideas.repec.org/p/osf/socarx/5bck9.html
   My bibliography  Save this paper

Replication in Social Science

Author

Listed:
  • Freese, Jeremy
  • Peterson, David

Abstract

Across the medical and social sciences, new discussions about replication have been transforming research practices. Sociologists, however, have been largely absent from these discussions. The goals of this review are to introduce sociologists to these developments, synthesize insights from science studies about replication in general, and detail the specific issues regarding replication that occur in sociology. The first half of the article argues that a sociologically sophisticated understanding of replication must take into account both the ways that replication rules and conventions evolved within an epistemic culture and how those cultures are shaped by specific research challenges. The second half outlines the four main dimensions of replicability in quantitative sociology- verifiability, robustness, repeatability, and generalizability- and discusses the specific ambiguities of interpretation that can arise in each. We conclude by advocating some commonsense changes to promote replication while acknowledging the epistemic diversity of our field.

Suggested Citation

  • Freese, Jeremy & Peterson, David, 2017. "Replication in Social Science," SocArXiv 5bck9, Center for Open Science.
    • Handle: RePEc:osf:socarx:5bck9
    DOI: 10.31219/osf.io/5bck9
    as

    Download full text from publisher

    File URL: https://osf.io/download/592c96676c613b0265c43f6a/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/5bck9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Andrew Abbott, 2007. "Notes on Replication," Sociological Methods & Research, , vol. 36(2), pages 210-219, November.
    2. Michael A. Clemens, 2017. "The Meaning Of Failed Replications: A Review And Proposal," Journal of Economic Surveys, Wiley Blackwell, vol. 31(1), pages 326-342, February.
    3. Cartwright,Nancy, 1999. "The Dappled World," Cambridge Books, Cambridge University Press, number 9780521643368, September.
    4. Cartwright,Nancy, 1999. "The Dappled World," Cambridge Books, Cambridge University Press, number 9780521644112, September.
    5. Jesse Chandler & et. al, 2016. "Response to Comment on "Estimating the Reproducibility of Psychological Science"," Mathematica Policy Research Reports cff9c2f16bb544c4bcca530c0, Mathematica Policy Research.
    6. Annette N. Brown & Drew B. Cameron & Benjamin D. K. Wood, 2014. "Quality evidence for policymaking: I'll believe it when I see the replication," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(3), pages 215-235, September.
    7. Lall, Ranjit, 2016. "How Multiple Imputation Makes a Difference," Political Analysis, Cambridge University Press, vol. 24(4), pages 414-433.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Do economists replicate?," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 219-232.
    2. Willy Bolander & Nawar N. Chaker & Alec Pappas & Daniel R. Bradbury, 2021. "Operationalizing salesperson performance with secondary data: aligning practice, scholarship, and theory," Journal of the Academy of Marketing Science, Springer, vol. 49(3), pages 462-481, May.
    3. Antonia Krefeld-Schwalb & Benjamin Scheibehenne, 2023. "Tighter nets for smaller fishes? Mapping the development of statistical practices in consumer research between 2008 and 2020," Marketing Letters, Springer, vol. 34(3), pages 351-365, September.
    4. Martin Kreidl & Zuzana Žilinčíková, 2023. "Adult children’s union type and contact with mothers: A replication," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 48(23), pages 641-680.
    5. Dreber, Anna & Johannesson, Magnus, 2023. "A framework for evaluating reproducibility and replicability in economics," I4R Discussion Paper Series 38, The Institute for Replication (I4R).
    6. Luis Alfonso Dau & Grazia D. Santangelo & Arjen Witteloostuijn, 2022. "Replication studies in international business," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 53(2), pages 215-230, March.
    7. Thompson, Phillip S. & Klotz, Anthony C., 2022. "Led by curiosity and responding with voice: The influence of leader displays of curiosity and leader gender on follower reactions of psychological safety and voice," Organizational Behavior and Human Decision Processes, Elsevier, vol. 172(C).
    8. Fišar, Miloš & Greiner, Ben & Huber, Christoph & Katok, Elena & Ozkes, Ali & Management Science Reproducibility Collaboration, 2023. "Reproducibility in Management Science," Department for Strategy and Innovation Working Paper Series 03/2023, WU Vienna University of Economics and Business.
    9. Fanelli, Daniele, 2022. "The "Tau" of Science - How to Measure, Study, and Integrate Quantitative and Qualitative Knowledge," MetaArXiv 67sak, Center for Open Science.
    10. Boris Forthmann & Mark A. Runco, 2020. "An Empirical Test of the Inter-Relationships between Various Bibliometric Creative Scholarship Indicators," Publications, MDPI, vol. 8(2), pages 1-16, June.
    11. Fanelli, Daniele, 2020. "Metascientific reproducibility patterns revealed by informatic measure of knowledge," MetaArXiv 5vnhj, Center for Open Science.
    12. Michel Herzig, 2020. "Mediating Factors of Family Structure and Early Home-leaving: A Replication and Extension of van den Berg, Kalmijn, and Leopold (2018)," European Journal of Population, Springer;European Association for Population Studies, vol. 36(4), pages 643-674, September.
    13. Ian Lundberg & Arvind Narayanan & Karen Levy & Matthew Salganik, 2018. "Privacy, ethics, and data access: A case study of the Fragile Families Challenge," Working Papers wp18-09-ff, Princeton University, School of Public and International Affairs, Center for Research on Child Wellbeing..
    14. B. Parker Ellen & Christian Kiewitz & Patrick Raymund James M. Garcia & Wayne A. Hochwarter, 2019. "Dealing with the Full-of-Self-Boss: Interactive Effects of Supervisor Narcissism and Subordinate Resource Management Ability on Work Outcomes," Journal of Business Ethics, Springer, vol. 157(3), pages 847-864, July.
    15. Daniel T. L. Shek & Diya Dou & Xiaoqin Zhu & Xiang Li & Lindan Tan, 2022. "Materialism, Egocentrism and Delinquent Behavior in Chinese Adolescents in Mainland China: A Short-Term Longitudinal Study," IJERPH, MDPI, vol. 19(8), pages 1-15, April.
    16. Yuyan Jiang & Xueli Liu, 2023. "A construction and empirical research of the journal disruption index based on open citation data," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(7), pages 3935-3958, July.
    17. Mats Alvesson & Jörgen Sandberg, 2020. "The Problematizing Review: A Counterpoint to Elsbach and Van Knippenberg’s Argument for Integrative Reviews," Journal of Management Studies, Wiley Blackwell, vol. 57(6), pages 1290-1304, September.
    18. Chin, Jason & Zeiler, Kathryn, 2021. "Replicability in Empirical Legal Research," LawArXiv 2b5k4, Center for Open Science.
    19. Lundberg, Ian & Brand, Jennie E. & Jeon, Nanum, 2022. "Researcher reasoning meets computational capacity: Machine learning for social science," SocArXiv s5zc8, Center for Open Science.
    20. Andrew G.H. Thompson & Oliver Escobar & Jennifer J. Roberts & Stephen Elstub & Niccole M. Pamphilis, 2021. "The Importance of Context and the Effect of Information and Deliberation on Opinion Change Regarding Environmental Issues in Citizens’ Juries," Sustainability, MDPI, vol. 13(17), pages 1-21, September.
    21. Jack I. Richter & Pankaj C. Patel, 2022. "Impact of the COVID-19 pandemic on the hours lost by self-employed racial minorities: evidence from Brazil," Small Business Economics, Springer, vol. 58(2), pages 769-805, February.
    22. Franz Neuberger & Tobias Rüttenauer & Martin Bujard, 2022. "Where does public childcare boost female labor force participation? Exploring geographical heterogeneity across Germany 2007–2017," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 46(24), pages 693-722.
    23. Stephan Puehringer, 2023. "Wie viel Wettbewerb wollen wir (uns leisten)? Zur Verwettbewerblichung der Universitaeten in Oesterreich und darueber hinaus," ICAE Working Papers 149, Johannes Kepler University, Institute for Comprehensive Analysis of the Economy.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Donovan, Kevin P., 2018. "The rise of the randomistas: on the experimental turn in international aid," SocArXiv xygzb, Center for Open Science.
    2. Giandomenica Becchio, 2020. "The Two Blades of Occam's Razor in Economics: Logical and Heuristic," Economic Thought, World Economics Association, vol. 9(1), pages 1-17, July.
    3. Julian Reiss, 2001. "Natural economic quantities and their measurement," Journal of Economic Methodology, Taylor & Francis Journals, vol. 8(2), pages 287-311.
    4. Aumann, Craig A., 2007. "A methodology for developing simulation models of complex systems," Ecological Modelling, Elsevier, vol. 202(3), pages 385-396.
    5. Florian Ellsaesser & Eric W. K. Tsang & Jochen Runde, 2014. "Models of causal inference: Imperfect but applicable is better than perfect but inapplicable," Strategic Management Journal, Wiley Blackwell, vol. 35(10), pages 1541-1551, October.
    6. Stephen Pratten, 2007. "Realism, closed systems and abstraction," Journal of Economic Methodology, Taylor & Francis Journals, vol. 14(4), pages 473-497.
    7. Kinouchi, Renato, 2018. "Philosophical issues related to risks and values," LSE Research Online Documents on Economics 90470, London School of Economics and Political Science, LSE Library.
    8. Maren Duvendack & Richard Palmer-Jones & W. Robert Reed, 2017. "What Is Meant by "Replication" and Why Does It Encounter Resistance in Economics?," American Economic Review, American Economic Association, vol. 107(5), pages 46-51, May.
    9. Marcel Boumans & Mary Morgan, 2002. "Ceteris paribus conditions: materiality and the application of economic theories," Journal of Economic Methodology, Taylor & Francis Journals, vol. 8(1), pages 11-26.
    10. Peter C. B. Phillips, 2003. "Laws and Limits of Econometrics," Economic Journal, Royal Economic Society, vol. 113(486), pages 26-52, March.
    11. Midgley, Gerald, 2008. "Response to paper "Systems thinking" by D. Cabrera et al.:: The unification of systems thinking: Is there gold at the end of the rainbow?," Evaluation and Program Planning, Elsevier, vol. 31(3), pages 317-321, August.
    12. Benjamin D K Wood & Rui Müller & Annette N Brown, 2018. "Push button replication: Is impact evaluation evidence for international development verifiable?," PLOS ONE, Public Library of Science, vol. 13(12), pages 1-15, December.
    13. Nicolas Brisset, 2018. "Models as speech acts: the telling case of financial models," Journal of Economic Methodology, Taylor & Francis Journals, vol. 25(1), pages 21-41, January.
    14. Toby Ord & Rafaela Hillerbrand & Anders Sandberg, 2010. "Probing the improbable: methodological challenges for risks with low probabilities and high stakes," Journal of Risk Research, Taylor & Francis Journals, vol. 13(2), pages 191-205, March.
    15. Enzo Lenine, 2020. "Modelling Coalitions: From Concept Formation to Tailoring Empirical Explanations," Games, MDPI, vol. 11(4), pages 1-12, November.
    16. Simon Hall & Nilufa Ali & Nick Chater & Mike Oaksford, 2016. "Discounting and Augmentation in Causal Conditional Reasoning: Causal Models or Shallow Encoding?," PLOS ONE, Public Library of Science, vol. 11(12), pages 1-23, December.
    17. Paul Shaffer, 2018. "Causal pluralism and mixed methods in the analysis of poverty dynamics," WIDER Working Paper Series 115, World Institute for Development Economic Research (UNU-WIDER).
    18. Catherine Laurent & Jacques Baudry & Marielle Berriet-Solliec & Marc Kirsch & Daniel Perraud & Bruno Tinel & Aurélie Trouvé & Nicky Allsopp & Patrick Bonnafous & Françoise Burel & Maria José Carneiro , 2009. "Pourquoi s'intéresser à la notion d' « evidence-based policy » ?," Revue Tiers-Monde, Armand Colin, vol. 0(4), pages 853-873.
    19. Javier Guillermo Gómez P., 2008. ""El crecimiento económico y la supervivencia": el caso de las matemáticas y la economía," Borradores de Economia 498, Banco de la Republica de Colombia.
    20. Sang Yi, 2002. "The nature of model-based understanding in condensed matter physics," Mind & Society: Cognitive Studies in Economics and Social Sciences, Springer;Fondazione Rosselli, vol. 3(1), pages 81-91, March.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:socarx:5bck9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://arabixiv.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.