IDEAS home Printed from https://ideas.repec.org/a/spr/testjl/v33y2024i3d10.1007_s11749-023-00916-4.html
   My bibliography  Save this article

Bayesian sample size determination for detecting heterogeneity in multi-site replication studies

Author

Listed:
  • Konstantinos Bourazas

    (University of Cyprus, Nicosia)

  • Guido Consonni

    (Università Cattolica del Sacro Cuore)

  • Laura Deldossi

    (Università Cattolica del Sacro Cuore)

Abstract

An ongoing “replication crisis” calls into question scientific discoveries across a variety of disciplines ranging from life to social sciences. Replication studies aim to investigate the validity of findings in published research, and try to assess whether the latter are statistically consistent with those in the replications. While the majority of replication projects are based on a single experiment, multiple independent replications of the same experiment conducted simultaneously at different sites are becoming more frequent. In connection with these types of projects, we deal with testing heterogeneity among sites; specifically, we focus on sample size determination suitable to deliver compelling evidence once the experimental data are gathered.

Suggested Citation

  • Konstantinos Bourazas & Guido Consonni & Laura Deldossi, 2024. "Bayesian sample size determination for detecting heterogeneity in multi-site replication studies," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 33(3), pages 697-716, September.
    • Handle: RePEc:spr:testjl:v:33:y:2024:i:3:d:10.1007_s11749-023-00916-4
    DOI: 10.1007/s11749-023-00916-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11749-023-00916-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11749-023-00916-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Leonhard Held, 2020. "A new standard for the analysis and design of replication studies," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(2), pages 431-448, February.
    2. Leonard E. Burman & W. Robert Reed & James Alm, 2011. "A Call for Replication Studies," Public Finance Review, , vol. 39(1), pages 190-190, January.
    3. Anthony O’Hagan & John W. Stevens, 2001. "Bayesian Assessment of Sample Size for Clinical Trials of Cost-Effectiveness," Medical Decision Making, , vol. 21(3), pages 219-230, May.
    4. Kewei Hou & Chen Xue & Lu Zhang, 2020. "Replicating Anomalies," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2019-2133.
    5. Samuel Pawel & Leonhard Held, 2022. "The sceptical Bayes factor for the assessment of replication success," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(3), pages 879-911, July.
    6. Ronald L. Wasserstein & Nicole A. Lazar, 2016. "The ASA's Statement on p -Values: Context, Process, and Purpose," The American Statistician, Taylor & Francis Journals, vol. 70(2), pages 129-133, May.
    7. Samuel Pawel & Leonhard Held, 2020. "Probabilistic forecasting of replication studies," PLOS ONE, Public Library of Science, vol. 15(4), pages 1-23, April.
    8. Richard Simon, 1999. "Bayesian Design and Analysis of Active Control Clinical Trials," Biometrics, The International Biometric Society, vol. 55(2), pages 484-487, June.
    9. Pierpaolo Brutti & Fulvio Santis & Stefania Gubbiotti, 2014. "Bayesian-frequentist sample size determination: a game of two priors," METRON, Springer;Sapienza Università di Roma, vol. 72(2), pages 133-151, August.
    10. Psarakis, Stelios & Panaretos, John, 1990. "The Folded t Distribution," MPRA Paper 6257, University Library of Munich, Germany.
    11. Alexander Etz & Joachim Vandekerckhove, 2016. "A Bayesian Perspective on the Reproducibility Project: Psychology," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-12, February.
    12. Larry V. Hedges & Jacob M. Schauer, 2019. "More Than One Replication Study Is Needed for Unambiguous Tests of Replication," Journal of Educational and Behavioral Statistics, , vol. 44(5), pages 543-570, October.
    13. Christopher Harms, 2019. "A Bayes Factor for Replications of ANOVA Results," The American Statistician, Taylor & Francis Journals, vol. 73(4), pages 327-339, October.
    14. Hensel, Przemysław G., 2021. "Reproducibility and replicability crisis: How management compares to psychology and economics – A systematic review of literature," European Management Journal, Elsevier, vol. 39(5), pages 577-594.
    15. Valen E. Johnson & David Rossell, 2010. "On the use of non‐local prior densities in Bayesian hypothesis tests," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(2), pages 143-170, March.
    16. Larry V. Hedges & Jacob M. Schauer, 2021. "The design of replication studies," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(3), pages 868-886, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Samuel Pawel & Leonhard Held, 2022. "The sceptical Bayes factor for the assessment of replication success," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(3), pages 879-911, July.
    2. Charlotte Micheloud & Fadoua Balabdaoui & Leonhard Held, 2023. "Assessing replicability with the sceptical p$$ p $$‐value: Type‐I error control and sample size planning," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 77(4), pages 573-591, November.
    3. Michaelides, Michael, 2021. "Large sample size bias in empirical finance," Finance Research Letters, Elsevier, vol. 41(C).
    4. Kelter, Riko, 2022. "Power analysis and type I and type II error rates of Bayesian nonparametric two-sample tests for location-shifts based on the Bayes factor under Cauchy priors," Computational Statistics & Data Analysis, Elsevier, vol. 165(C).
    5. Fulvio De Santis & Stefania Gubbiotti, 2021. "On the predictive performance of a non-optimal action in hypothesis testing," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(2), pages 689-709, June.
    6. Freuli, Francesca & Held, Leonhard & Heyard, Rachel, 2022. "Replication Success under Questionable Research Practices - A Simulation Study," I4R Discussion Paper Series 2, The Institute for Replication (I4R).
    7. Andrew Y. Chen & Tom Zimmermann, 2022. "Publication Bias in Asset Pricing Research," Papers 2209.13623, arXiv.org, revised Sep 2023.
    8. Lawrence L. Kupper & Sandra L. Martin, 2022. "Replication study design: confidence intervals and commentary," Statistical Papers, Springer, vol. 63(5), pages 1577-1583, October.
    9. Kim, Jae H. & Shamsuddin, Abul, 2023. "Stock market anomalies: An extreme bounds analysis," International Review of Financial Analysis, Elsevier, vol. 90(C).
    10. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    11. Eugenio Melilli & Piero Veronese, 2024. "Confidence distributions and hypothesis testing," Statistical Papers, Springer, vol. 65(6), pages 3789-3820, August.
    12. Freuli, Francesca & Held, Leonhard & Heyard, Rachel, 2022. "Replication success under questionable research practices – a simulation study," MetaArXiv s4b65, Center for Open Science.
    13. D. Vélez & M. E. Pérez & L. R. Pericchi, 2022. "Increasing the replicability for linear models via adaptive significance levels," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 31(3), pages 771-789, September.
    14. Larry V. Hedges & Jacob M. Schauer, 2021. "The design of replication studies," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(3), pages 868-886, July.
    15. Shi, Huai-Long & Zhou, Wei-Xing, 2022. "Factor volatility spillover and its implications on factor premia," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 80(C).
    16. Klaus Grobys & James W. Kolari & Jere Rutanen, 2022. "Factor momentum, option-implied volatility scaling, and investor sentiment," Journal of Asset Management, Palgrave Macmillan, vol. 23(2), pages 138-155, March.
    17. Onishchenko, Olena & Zhao, Jing & Kongahawatte, Sampath & Kuruppuarachchi, Duminda, 2024. "Investor heterogeneity and anchoring-induced momentum," Journal of Behavioral and Experimental Finance, Elsevier, vol. 42(C).
    18. Fetene B. Tekle & Dereje W. Gudicha & Jeroen K. Vermunt, 2016. "Power analysis for the bootstrap likelihood ratio test for the number of classes in latent class models," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 10(2), pages 209-224, June.
    19. Cakici, Nusret & Zaremba, Adam, 2022. "Salience theory and the cross-section of stock returns: International and further evidence," Journal of Financial Economics, Elsevier, vol. 146(2), pages 689-725.
    20. Jyotirmoy Sarkar, 2018. "Will P†Value Triumph over Abuses and Attacks?," Biostatistics and Biometrics Open Access Journal, Juniper Publishers Inc., vol. 7(4), pages 66-71, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:testjl:v:33:y:2024:i:3:d:10.1007_s11749-023-00916-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.