IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v9y2021i9p958-d543024.html
   My bibliography  Save this article

Statistical Significance Revisited

Author

Listed:
  • Maike Tormählen

    (Department of Information Management, Neu-Ulm University, Wileystr. 1, 89231 Neu-Ulm, Germany)

  • Galiya Klinkova

    (Department of Business and Economics, Neu-Ulm University, Wileystr. 1, 89231 Neu-Ulm, Germany)

  • Michael Grabinski

    (Department of Business and Economics, Neu-Ulm University, Wileystr. 1, 89231 Neu-Ulm, Germany)

Abstract

Statistical significance measures the reliability of a result obtained from a random experiment. We investigate the number of repetitions needed for a statistical result to have a certain significance. In the first step, we consider binomially distributed variables in the example of medication testing with fixed placebo efficacy, asking how many experiments are needed in order to achieve a significance of 95%. In the next step, we take the probability distribution of the placebo efficacy into account, which to the best of our knowledge has not been done so far. Depending on the specifics, we show that in order to obtain identical significance, it may be necessary to perform twice as many experiments than in a setting where the placebo distribution is neglected. We proceed by considering more general probability distributions and close with comments on some erroneous assumptions on probability distributions which lead, for instance, to a trivial explanation of the fat tail.

Suggested Citation

  • Maike Tormählen & Galiya Klinkova & Michael Grabinski, 2021. "Statistical Significance Revisited," Mathematics, MDPI, vol. 9(9), pages 1-16, April.
    • Handle: RePEc:gam:jmathe:v:9:y:2021:i:9:p:958-:d:543024
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/9/9/958/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/9/9/958/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jae H. Kim, 2020. "Decision-Theoretic Hypothesis Testing: A Primer With R Package OptSig," The American Statistician, Taylor & Francis Journals, vol. 74(4), pages 370-379, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Maike Tormahlen & Galiya Klinkova & Michael Grabinski, 2021. "Statistical significance revisited," Papers 2104.00262, arXiv.org, revised Jun 2024.
    2. Jeremy Arkes, 2020. "Teaching Graduate (and Undergraduate) Econometrics: Some Sensible Shifts to Improve Efficiency, Effectiveness, and Usefulness," Econometrics, MDPI, vol. 8(3), pages 1-23, September.
    3. Jae H. Kim, 2022. "Moving to a world beyond p-value," Review of Managerial Science, Springer, vol. 16(8), pages 2467-2493, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:9:y:2021:i:9:p:958-:d:543024. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.