IDEAS home Printed from https://ideas.repec.org/a/bla/jorssc/v69y2020i3p697-708.html
   My bibliography  Save this article

The harmonic mean χ2‐test to substantiate scientific findings

Author

Listed:
  • Leonhard Held

Abstract

Statistical methodology plays a crucial role in drug regulation. Decisions by the US Food and Drug Administration or European Medicines Agency are typically made based on multiple primary studies testing the same medical product, where the two‐trials rule is the standard requirement, despite shortcomings. A new approach is proposed for this task based on the harmonic mean of the squared study‐specific test statistics. Appropriate scaling ensures that, for any number of independent studies, the null distribution is a χ2‐distribution with 1 degree of freedom. This gives rise to a new method for combining one‐sided p‐values and calculating confidence intervals for the overall treatment effect. Further properties are discussed and a comparison with the two‐trials rule is made, as well as with alternative research synthesis methods. An attractive feature of the new approach is that a claim of success requires each study to be convincing on its own to a certain degree depending on the overall level of significance and the number of studies. The new approach is motivated by and applied to data from five clinical trials investigating the effect of carvedilol for the treatment of patients with moderate to severe heart failure.

Suggested Citation

  • Leonhard Held, 2020. "The harmonic mean χ2‐test to substantiate scientific findings," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 69(3), pages 697-708, June.
    • Handle: RePEc:bla:jorssc:v:69:y:2020:i:3:p:697-708
    DOI: 10.1111/rssc.12410
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssc.12410
    Download Restriction: no
    File URL: https://libkey.io/10.1111/rssc.12410?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Leonhard Held, 2020. "A new standard for the analysis and design of replication studies," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(2), pages 431-448, February.
    2. N A Heard & P Rubin-Delanchy, 2018. "Choosing between methods of combining $p$-values," Biometrika, Biometrika Trust, vol. 105(1), pages 239-246.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Song, Zhi & Mukherjee, Amitava & Zhang, Jiujun, 2021. "Some robust approaches based on copula for monitoring bivariate processes and component-wise assessment," European Journal of Operational Research, Elsevier, vol. 289(1), pages 177-196.
    2. Charlotte Micheloud & Fadoua Balabdaoui & Leonhard Held, 2023. "Assessing replicability with the sceptical p$$ p $$‐value: Type‐I error control and sample size planning," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 77(4), pages 573-591, November.
    3. Paulo C. Rodrigues & Vanda M. Lourenço, 2020. "Comments on: Hierarchical Inference for genome-wide association studies: a view on methodology with software by Paulo C. Rodrigues and Vanda M. Lourenço," Computational Statistics, Springer, vol. 35(1), pages 57-58, March.
    4. Juan Antonio Villatoro-García & Jordi Martorell-Marugán & Daniel Toro-Domínguez & Yolanda Román-Montoya & Pedro Femia & Pedro Carmona-Sáez, 2022. "DExMA: An R Package for Performing Gene Expression Meta-Analysis with Missing Genes," Mathematics, MDPI, vol. 10(18), pages 1-15, September.
    5. Samuel Pawel & Leonhard Held, 2022. "The sceptical Bayes factor for the assessment of replication success," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(3), pages 879-911, July.
    6. Andra-Octavia Roman & Pedro Jimenez-Sandoval & Sebastian Augustin & Caroline Broyart & Ludwig A. Hothorn & Julia Santiago, 2022. "HSL1 and BAM1/2 impact epidermal cell development by sensing distinct signaling peptides," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    7. Konstantinos Bourazas & Guido Consonni & Laura Deldossi, 2024. "Bayesian sample size determination for detecting heterogeneity in multi-site replication studies," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 33(3), pages 697-716, September.
    8. Jai Won Choi & Balgobin Nandram & Boseung Choi, 2022. "Combining Correlated P-values From Primary Data Analyses," International Journal of Statistics and Probability, Canadian Center of Science and Education, vol. 11(6), pages 1-12, November.
    9. Freuli, Francesca & Held, Leonhard & Heyard, Rachel, 2022. "Replication Success under Questionable Research Practices - A Simulation Study," I4R Discussion Paper Series 2, The Institute for Replication (I4R).
    10. Xin Yuan & Yanran Ma & Ruitian Gao & Shuya Cui & Yifan Wang & Botao Fa & Shiyang Ma & Ting Wei & Shuangge Ma & Zhangsheng Yu, 2024. "HEARTSVG: a fast and accurate method for identifying spatially variable genes in large-scale spatial transcriptomics," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    11. Muradchanian, Jasmine & Hoekstra, Rink & Kiers, Henk & van Ravenzwaaij, Don, 2020. "How Best to Quantify Replication Success? A Simulation Study on the Comparison of Replication Success Metrics," MetaArXiv wvdjf, Center for Open Science.
    12. Lawrence L. Kupper & Sandra L. Martin, 2022. "Replication study design: confidence intervals and commentary," Statistical Papers, Springer, vol. 63(5), pages 1577-1583, October.
    13. Freuli, Francesca & Held, Leonhard & Heyard, Rachel, 2022. "Replication success under questionable research practices – a simulation study," MetaArXiv s4b65, Center for Open Science.
    14. Yu, Xiufan & Yao, Jiawei & Xue, Lingzhou, 2024. "Power enhancement for testing multi-factor asset pricing models via Fisher’s method," Journal of Econometrics, Elsevier, vol. 239(2).
    15. Wimmer, Thomas & Geyer-Klingeberg, Jerome & Hütter, Marie & Schmid, Florian & Rathgeber, Andreas, 2021. "The impact of speculation on commodity prices: A Meta-Granger analysis," Journal of Commodity Markets, Elsevier, vol. 22(C).
    16. Patrick B. Langthaler & Riccardo Ceccato & Luigi Salmaso & Rosa Arboretti & Arne C. Bathke, 2023. "Permutation testing for thick data when the number of variables is much greater than the sample size: recent developments and some recommendations," Computational Statistics, Springer, vol. 38(1), pages 101-132, March.
    17. Heyard, Rachel & Pawel, Samuel & Frese, Joris & Voelkl, Bernhard & Würbel, Hanno & McCann, Sarah & Held, Leonhard & Wever, Kimberley E. PhD & Hartmann, Helena & Townsin, Louise, 2024. "A scoping review on metrics to quantify reproducibility: a multitude of questions leads to a multitude of metrics," MetaArXiv apdxk, Center for Open Science.
    18. Xiong, Peihan & Hu, Taizhong, 2022. "On Samuel’s p-value model and the Simes test under dependence," Statistics & Probability Letters, Elsevier, vol. 187(C).
    19. Gary L. Rosner & Peter Müller, 2020. "Discussion on “Predictively consistent prior effective sample sizes,” by Beat Neuenschwander, Sebastian Weber, Heinz Schmidli, and Anthony O'Hagan," Biometrics, The International Biometric Society, vol. 76(2), pages 599-601, June.
    20. Zimmermann, Paul, 2021. "The role of the leverage effect in the price discovery process of credit markets," Journal of Economic Dynamics and Control, Elsevier, vol. 122(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssc:v:69:y:2020:i:3:p:697-708. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.