IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/avkgj.html
   My bibliography  Save this paper

Posterior Probabilities of Effect Sizes and Heterogeneity in Meta-Analysis: An Intuitive Approach of Dealing with Publication Bias

Author

Listed:
  • Augusteijn, Hilde Elisabeth Maria

    (Tilburg University)

  • van Aert, Robbie Cornelis Maria
  • van Assen, Marcel A. L. M.

Abstract

Publication bias remains to be a great challenge when conducting a meta-analysis. It may result in overestimated effect sizes, increased frequency of false positives, and over- or underestimation of the effect size heterogeneity parameter. A new method is introduced, Bayesian Meta-Analytic Snapshot (BMAS), which evaluates both effect size and its heterogeneity and corrects for potential publication bias. It evaluates the probability of the true effect size being zero, small, medium or large, and the probability of true heterogeneity being zero, small, medium or large. This approach, which provides an intuitive evaluation of uncertainty in the evaluation of effect size and heterogeneity, is illustrated with a real-data example, a simulation study, and a Shiny web application of BMAS.

Suggested Citation

  • Augusteijn, Hilde Elisabeth Maria & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2021. "Posterior Probabilities of Effect Sizes and Heterogeneity in Meta-Analysis: An Intuitive Approach of Dealing with Publication Bias," OSF Preprints avkgj, Center for Open Science.
    • Handle: RePEc:osf:osfxxx:avkgj
    DOI: 10.31219/osf.io/avkgj
    as

    Download full text from publisher

    File URL: https://osf.io/download/6053162990bb4e006614f8cb/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/avkgj?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Viechtbauer, Wolfgang, 2010. "Conducting Meta-Analyses in R with the metafor Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 36(i03).
    2. Sue Duval & Richard Tweedie, 2000. "Trim and Fill: A Simple Funnel-Plot–Based Method of Testing and Adjusting for Publication Bias in Meta-Analysis," Biometrics, The International Biometric Society, vol. 56(2), pages 455-463, June.
    3. Lutz Bornmann & Rüdiger Mutz, 2015. "Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(11), pages 2215-2222, November.
    4. Robbie C M van Aert & Jelte M Wicherts & Marcel A L M van Assen, 2019. "Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis," PLOS ONE, Public Library of Science, vol. 14(4), pages 1-32, April.
    5. Daniele Fanelli, 2012. "Negative results are disappearing from most disciplines and countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 891-904, March.
    6. Dan Jackson, 2007. "Assessing the Implications of Publication Bias for Two Popular Estimates of between-Study Variance in Meta-Analysis," Biometrics, The International Biometric Society, vol. 63(1), pages 187-193, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Robbie C M van Aert & Jelte M Wicherts & Marcel A L M van Assen, 2019. "Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis," PLOS ONE, Public Library of Science, vol. 14(4), pages 1-32, April.
    2. Bart Verkuil & Serpil Atasayi & Marc L Molendijk, 2015. "Workplace Bullying and Mental Health: A Meta-Analysis on Cross-Sectional and Longitudinal Data," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-16, August.
    3. Piers Steel & Sjoerd Beugelsdijk & Herman Aguinis, 2021. "The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 52(1), pages 23-44, February.
    4. Molo, Fabio & Pawel, Samuel & Fraga González, Gorka, 2024. "A Robustness Reproduction of "A Systematic Review and Meta-Analysis of 90 Cohort Studies of Social Isolation, Loneliness and Mortality"," I4R Discussion Paper Series 169, The Institute for Replication (I4R).
    5. Georgiou, George K. & Guo, Kan & Naveenkumar, Nithya & Vieira, Ana Paula Alves & Das, J.P., 2020. "PASS theory of intelligence and academic achievement: A meta-analytic review," Intelligence, Elsevier, vol. 79(C).
    6. Geller, Susann & Wilhelm, Oliver & Wacker, Jan & Hamm, Alfons & Hildebrandt, Andrea, 2017. "Associations of the COMT Val158Met polymorphism with working memory and intelligence – A review and meta-analysis," Intelligence, Elsevier, vol. 65(C), pages 75-92.
    7. Christopher Winchester & Kelsey E. Medeiros, 2023. "In Bounds but Out of the Box: A Meta-Analysis Clarifying the Effect of Ethicality on Creativity," Journal of Business Ethics, Springer, vol. 183(3), pages 713-743, March.
    8. Lifeng Lin & Haitao Chu, 2018. "Rejoinder to “quantifying publication bias in meta‐analysis”," Biometrics, The International Biometric Society, vol. 74(3), pages 801-802, September.
    9. Johannes Hönekopp & Audrey Helen Linden, 2022. "Heterogeneity estimates in a biased world," PLOS ONE, Public Library of Science, vol. 17(2), pages 1-21, February.
    10. Nian-Feng Wan & Liwan Fu & Matteo Dainese & Yue-Qing Hu & Lars Pødenphant Kiær & Forest Isbell & Christoph Scherber, 2022. "Plant genetic diversity affects multiple trophic levels and trophic interactions," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    11. Franco Curadelli & Marcelo Alberto & Ernesto Martín Uliarte & Mariana Combina & Iván Funes-Pinter, 2023. "Meta-Analysis of Yields of Crops Fertilized with Compost Tea and Anaerobic Digestate," Sustainability, MDPI, vol. 15(2), pages 1-23, January.
    12. Kim, Yeolib & Kim, Seung Hyun & Peterson, Robert A. & Choi, Jeonghye, 2023. "Privacy concern and its consequences: A meta-analysis," Technological Forecasting and Social Change, Elsevier, vol. 196(C).
    13. Smith, Stephanie R. & Kroon, Jeroen & Schwarzer, Ralf & Hamilton, Kyra, 2020. "Parental social-cognitive correlates of preschoolers’ oral hygiene behavior: A systematic review and meta-analysis," Social Science & Medicine, Elsevier, vol. 264(C).
    14. K. Praveen Parboteeah & Matthias Weiss & Martin Hoegl, 2024. "Ethical Climates Across National Contexts: A Meta-Analytical Investigation," Journal of Business Ethics, Springer, vol. 189(3), pages 573-590, January.
    15. Mohith M. Varma & Shengzi Zeng & Laura Singh & Emily A. Holmes & Jingyun Huang & Man Hey Chiu & Xiaoqing Hu, 2024. "A systematic review and meta-analysis of experimental methods for modulating intrusive memories following lab-analogue trauma exposure in non-clinical populations," Nature Human Behaviour, Nature, vol. 8(10), pages 1968-1987, October.
    16. Woodley of Menie, Michael A. & Peñaherrera-Aguirre, Mateo & Sarraf, Matthew A., 2022. "Signs of a Flynn effect in rodents? Secular differentiation of the manifold of general cognitive ability in laboratory mice (Mus musculus) and Norwegian rats (Rattus norvegicus) over a century—Results," Intelligence, Elsevier, vol. 95(C).
    17. Wenke, Kathrin & Zapkau, Florian B. & Schwens, Christian, 2021. "Too small to do it all? A meta-analysis on the relative relationships of exploration, exploitation, and ambidexterity with SME performance," Journal of Business Research, Elsevier, vol. 132(C), pages 653-665.
    18. Stav Fainshmidt & Amir Pezeshkan & M. Lance Frazier & Anil Nair & Edward Markowski, 2016. "Dynamic Capabilities and Organizational Performance: A Meta-Analytic Evaluation and Extension," Journal of Management Studies, Wiley Blackwell, vol. 53(8), pages 1348-1380, December.
    19. Jaskiewicz, Peter & Block, Joern & Wagner, Dominik & Carney, Michael & Hansen, Christopher, 2021. "How do cross-country differences in institutional trust and trust in family explain the mixed performance effects of family management? A meta-analysis," Journal of World Business, Elsevier, vol. 56(5).
    20. Anouk Decuypere & Wilmar Schaufeli, 2021. "Exploring the Leadership–Engagement Nexus: A Moderated Meta-Analysis and Review of Explaining Mechanisms," IJERPH, MDPI, vol. 18(16), pages 1-34, August.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:avkgj. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.