IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2104.00262.html
   My bibliography  Save this paper

Statistical significance revisited

Author

Listed:
  • Maike Tormahlen
  • Galiya Klinkova
  • Michael Grabinski

Abstract

Statistical significance measures the reliability of a result obtained from a random experiment. We investigate the number of repetitions needed for a statistical result to have a certain significance. In the first step, we consider binomially distributed variables in the example of medication testing with fixed placebo efficacy, asking how many experiments are needed in order to achieve a significance of 95 %. In the next step, we take the probability distribution of the placebo efficacy into account, which to the best of our knowledge has not been done so far. Depending on the specifics, we show that in order to obtain identical significance, it may be necessary to perform twice as many experiments than in a setting where the placebo distribution is neglected. We proceed by considering more general probability distributions and close with comments on some erroneous assumptions on probability distributions which lead, for instance, to a trivial explanation of the fat tail.

Suggested Citation

  • Maike Tormahlen & Galiya Klinkova & Michael Grabinski, 2021. "Statistical significance revisited," Papers 2104.00262, arXiv.org, revised Jun 2024.
    • Handle: RePEc:arx:papers:2104.00262
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2104.00262
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jae H. Kim, 2020. "Decision-Theoretic Hypothesis Testing: A Primer With R Package OptSig," The American Statistician, Taylor & Francis Journals, vol. 74(4), pages 370-379, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Maike Tormählen & Galiya Klinkova & Michael Grabinski, 2021. "Statistical Significance Revisited," Mathematics, MDPI, vol. 9(9), pages 1-16, April.
    2. Jeremy Arkes, 2020. "Teaching Graduate (and Undergraduate) Econometrics: Some Sensible Shifts to Improve Efficiency, Effectiveness, and Usefulness," Econometrics, MDPI, vol. 8(3), pages 1-23, September.
    3. Jae H. Kim, 2022. "Moving to a world beyond p-value," Review of Managerial Science, Springer, vol. 16(8), pages 2467-2493, November.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2104.00262. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.