What Can We Learn From 1000 Meta-Analyses Across 10 Different Disciplines?

This study describes and analyzes 1000 meta-analyses across ten different disciplines spanning medicine, science, and the social sciences. It highlights significant variations in methodology across disciplines and offers targeted recommendations for enhancing research synthesis. Our analysis reveals discipline-specific differences in the number of studies and estimates per study, types of effect sizes used, and the prevalence of unpublished studies included in meta-analyses. It also examines the extent of effect heterogeneity and the employment of meta-regression to explain this heterogeneity across different fields. Our findings underscore the underutilization of robust meta-analytic methods like three-level models and CR2 clustered standard errors, which are crucial for addressing dependencies among multiple estimates within studies. Finally, we discuss the implications of publication bias and the prevalence of various tests and corrections across disciplines. Our recommendations aim to learn from and apply best practices across all disciplines. This work serves as a resource for researchers conducting their first meta-analyses, as a benchmark for researchers designing simulation experiments, and as a reference for applied meta-analysts aiming to improve their methodological practices.