The Power of Tests for Detecting $p$-Hacking
Author
Abstract
Suggested Citation
Download full text from publisher
References listed on IDEAS
- Jérôme Adda & Christian Decker & Marco Ottaviani, 2020.
"P-hacking in clinical trials and how incentives shape the distribution of results across phases,"
Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 117(24), pages 13386-13392, June.
- Ottaviani, Marco & Adda, Jérôme & Decker, Christian, 2019. "P-hacking in clinical trials and how incentives shape the distribution of results across phases," CEPR Discussion Papers 13836, C.E.P.R. Discussion Papers.
- J'er^ome Adda & Christian Decker & Marco Ottaviani, 2019. "P-hacking in clinical trials and how incentives shape the distribution of results across phases," Papers 1907.00185, arXiv.org, revised Mar 2020.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016.
"Star Wars: The Empirics Strike Back,"
American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2012. "Star wars: The empirics strike back," Working Papers halshs-00710122, HAL.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2015. "Star Wars: The Empirics Strike Back," PSE Working Papers halshs-01158500, HAL.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," PSE-Ecole d'économie de Paris (Postprint) hal-01447851, HAL.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2015. "Star Wars: The Empirics Strike Back," Working Papers halshs-01158500, HAL.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2015. "Star Wars: The Empirics Strike Back," AMSE Working Papers 1523, Aix-Marseille School of Economics, France, revised May 2015.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2012. "Star wars: The empirics strike back," PSE Working Papers halshs-00710122, HAL.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2012. "Star wars: The empirics strike back," SciencePo Working papers Main halshs-00710122, HAL.
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2015. "Star Wars: The Empirics Strike Back," Working Papers 1505E, University of Ottawa, Department of Economics.
- Brodeur, Abel & Lé, Mathias & Sangnier, Marc & Zylberberg, Yanos, 2013. "Star Wars: The Empirics Strike Back," IZA Discussion Papers 7268, Institute of Labor Economics (IZA).
- Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," Post-Print hal-01447851, HAL.
- McCloskey, Adam & Michaillat, Pascal, 2022.
"Incentive-Compatible Critical Values,"
CEPR Discussion Papers
16942, C.E.P.R. Discussion Papers.
- Adam McCloskey & Pascal Michaillat, 2022. "Incentive-Compatible Critical Values," NBER Working Papers 29702, National Bureau of Economic Research, Inc.
- Garret Christensen & Edward Miguel, 2018.
"Transparency, Reproducibility, and the Credibility of Economics Research,"
Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
- Garret S. Christensen & Edward Miguel, 2016. "Transparency, Reproducibility, and the Credibility of Economics Research," NBER Working Papers 22989, National Bureau of Economic Research, Inc.
- Christensen, Garret & Miguel, Edward & Sturdy, Jennifer, 2017. "Transparency, Reproducibility, and the Credibility of Economics Research," MetaArXiv 9a3rw, Center for Open Science.
- Christensen, Garret & Miguel, Edward, 2017. "Transparency, Reproducibility, and the Credibility of Economics Research," Department of Economics, Working Paper Series qt52h6x1cq, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
- Matias D. Cattaneo & Michael Jansson & Xinwei Ma, 2020.
"Simple Local Polynomial Density Estimators,"
Journal of the American Statistical Association, Taylor & Francis Journals, vol. 115(531), pages 1449-1455, July.
- Matias D. Cattaneo & Michael Jansson & Xinwei Ma, 2018. "Simple Local Polynomial Density Estimators," Papers 1811.11512, arXiv.org, revised Jun 2019.
- Cattaneo, Matias D & Jansson, Michael & Ma, Xinwei, 2020. "Simple Local Polynomial Density Estimators," Department of Economics, Working Paper Series qt9vt997qn, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
- Cattaneo, Matias D & Jansson, Michael & Ma, Xinwei, 2020. "Simple Local Polynomial Density Estimators," University of California at San Diego, Economics Working Paper Series qt9vt997qn, Department of Economics, UC San Diego.
- McCrary, Justin, 2008. "Manipulation of the running variable in the regression discontinuity design: A density test," Journal of Econometrics, Elsevier, vol. 142(2), pages 698-714, February.
- Leamer, Edward E, 1983.
"Let's Take the Con Out of Econometrics,"
American Economic Review, American Economic Association, vol. 73(1), pages 31-43, March.
- Edward E. Leamer, 1982. "Let's Take the Con Out of Econometrics," UCLA Economics Working Papers 239, UCLA Department of Economics.
- Eva Vivalt, 2019. "Specification Searching and Significance Inflation Across Time, Methods and Disciplines," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 81(4), pages 797-816, August.
- Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
- Alexander Frankel & Maximilian Kasy, 2022.
"Which Findings Should Be Published?,"
American Economic Journal: Microeconomics, American Economic Association, vol. 14(1), pages 1-38, February.
- Kasy, Maximilian & Frankel, Alexander, 2018. "Which findings should be published?," MetaArXiv mbvz3, Center for Open Science.
- Isaiah Andrews & Maximilian Kasy, 2019.
"Identification of and Correction for Publication Bias,"
American Economic Review, American Economic Association, vol. 109(8), pages 2766-2794, August.
- Isaiah Andrews & Maximilian Kasy, 2017. "Identification of and correction for publication bias," Papers 1711.10527, arXiv.org.
- Kasy, Maximilian & Andrews, Isaiah, 2018. "Identification of and correction for publication bias," MetaArXiv 49yst, Center for Open Science.
- Isaiah Andrews & Maximilian Kasy, 2017. "Identification of and Correction for Publication Bias," NBER Working Papers 23298, National Bureau of Economic Research, Inc.
- Stephan B Bruns & John P A Ioannidis, 2016. "p-Curve and p-Hacking in Observational Research," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-13, February.
- Gerber, Alan & Malhotra, Neil, 2008. "Do Statistical Reporting Standards Affect What Is Published? Publication Bias in Two Leading Political Science Journals," Quarterly Journal of Political Science, now publishers, vol. 3(3), pages 313-326, October.
- Ronald L. Wasserstein & Nicole A. Lazar, 2016. "The ASA's Statement on p -Values: Context, Process, and Purpose," The American Statistician, Taylor & Francis Journals, vol. 70(2), pages 129-133, May.
- Vladimir N. Kulikov & Hendrik P. Lopuhaä, 2008. "Distribution of Global Measures of Deviation Between the Empirical Distribution Function and Its Concave Majorant," Journal of Theoretical Probability, Springer, vol. 21(2), pages 356-377, June.
- Stephan B. Bruns, 2017. "Meta-Regression Models and Observational Research," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 79(5), pages 637-653, October.
- Guido W. Imbens, 2021. "Statistical Significance, p-Values, and the Reporting of Uncertainty," Journal of Economic Perspectives, American Economic Association, vol. 35(3), pages 157-174, Summer.
- Megan L Head & Luke Holman & Rob Lanfear & Andrew T Kahn & Michael D Jennions, 2015. "The Extent and Consequences of P-Hacking in Science," PLOS Biology, Public Library of Science, vol. 13(3), pages 1-15, March.
- Bruns, Stephan B. & Asanov, Igor & Bode, Rasmus & Dunger, Melanie & Funk, Christoph & Hassan, Sherif M. & Hauschildt, Julia & Heinisch, Dominik & Kempa, Karol & König, Johannes & Lips, Johannes & Verb, 2019. "Reporting errors and biases in published empirical findings: Evidence from innovation research," Research Policy, Elsevier, vol. 48(9), pages 1-1.
- Maximilian Kasy, 2021. "Of Forking Paths and Tied Hands: Selective Publication of Findings, and What Economists Should Do about It," Journal of Economic Perspectives, American Economic Association, vol. 35(3), pages 175-192, Summer.
- Kinal, Terrence W, 1980. "The Existence of Moments of k-Class Estimators," Econometrica, Econometric Society, vol. 48(1), pages 241-249, January.
- Matias D. Cattaneo & Michael Jansson & Xinwei Ma, 2018. "Manipulation testing based on density discontinuity," Stata Journal, StataCorp LP, vol. 18(1), pages 234-261, March.
- Christopher A. Carolan & Joshua M. Tebbs, 2005. "Nonparametric tests for and against likelihood ratio ordering in the two-sample problem," Biometrika, Biometrika Trust, vol. 92(1), pages 159-171, March.
- Beare, Brendan K. & Moon, Jong-Myun, 2015. "Nonparametric Tests Of Density Ratio Ordering," Econometric Theory, Cambridge University Press, vol. 31(3), pages 471-492, June.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Graham Elliott & Nikolay Kudrin & Kaspar Wüthrich, 2022.
"Detecting p‐Hacking,"
Econometrica, Econometric Society, vol. 90(2), pages 887-906, March.
- Graham Elliott & Nikolay Kudrin & Kaspar Wuthrich, 2019. "Detecting p-hacking," Papers 1906.06711, arXiv.org, revised May 2021.
- Elliott, Graham & Kudrin, Nikolay & Wüthrich, Kaspar, 2022. "Detecting p‐Hacking," University of California at San Diego, Economics Working Paper Series qt2p04s3dr, Department of Economics, UC San Diego.
- Abel Brodeur & Scott Carrell & David Figlio & Lester Lusher, 2023.
"Unpacking P-hacking and Publication Bias,"
American Economic Review, American Economic Association, vol. 113(11), pages 2974-3002, November.
- Brodeur, Abel & Carrell, Scott & Figlio, David & Lusher, Lester, 2023. "Unpacking P-Hacking and Publication Bias," I4R Discussion Paper Series 52, The Institute for Replication (I4R).
- Abel Brodeur & Scott E. Carrell & David N. Figlio & Lester R. Lusher, 2023. "Unpacking P-Hacking and Publication Bias," NBER Working Papers 31548, National Bureau of Economic Research, Inc.
- Brodeur, Abel & Carrell, Scott E. & Figlio, David N. & Lusher, Lester, 2023. "Unpacking P-Hacking and Publication Bias," IZA Discussion Papers 16369, Institute of Labor Economics (IZA).
- Abel Brodeur & Nikolai Cook & Carina Neisser, 2024.
"p-Hacking, Data type and Data-Sharing Policy,"
The Economic Journal, Royal Economic Society, vol. 134(659), pages 985-1018.
- Brodeur, Abel & Cook, Nikolai & Neisser, Carina, 2022. "P-Hacking, Data Type and Data-Sharing Policy," IZA Discussion Papers 15586, Institute of Labor Economics (IZA).
- Abel Brodeur & Nikolai Cook & Carina Neisser, 2022. "P-Hacking, Data Type and Data-Sharing Policy," ECONtribute Discussion Papers Series 200, University of Bonn and University of Cologne, Germany.
- Brodeur, Abel & Cook, Nikolai & Hartley, Jonathan & Heyes, Anthony, 2022.
"Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?,"
MetaArXiv
uxf39, Center for Open Science.
- Brodeur, Abel & Cook, Nikolai M. & Hartley, Jonathan S. & Heyes, Anthony, 2022. "Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?," GLO Discussion Paper Series 1147, Global Labor Organization (GLO).
- Brodeur, Abel & Cook, Nikolai & Hartley, Jonathan S. & Heyes, Anthony, 2022. "Do Pre-registration and Pre-analysis Plans Reduce P-Hacking and Publication Bias?," IZA Discussion Papers 15476, Institute of Labor Economics (IZA).
- Abel Brodeur, Nikolai M. Cook, Jonathan S. Hartley, Anthony Heyes, 2022. "Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?," LCERPA Working Papers am0132, Laurier Centre for Economic Research and Policy Analysis.
- Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
- Dominika Ehrenbergerova & Josef Bajzik & Tomas Havranek, 2023.
"When Does Monetary Policy Sway House Prices? A Meta-Analysis,"
IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(2), pages 538-573, June.
- Havranek, Tomas & Kolcunova, Dominika & Bajzik, Josef, 2021. "When Does Monetary Policy Sway House Prices? A Meta-Analysis," CEPR Discussion Papers 16196, C.E.P.R. Discussion Papers.
- Josef Bajzik & Dominika Ehrenbergerova & Tomas Havranek, 2021. "When Does Monetary Policy Sway House Prices? A Meta-Analysis," Working Papers IES 2021/17, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised May 2021.
- Ehrenbergerova, Dominika & Bajzik, Josef & Havranek, Tomas, 2021. "When Does Monetary Policy Sway House Prices? A Meta-Analysis," EconStor Preprints 234126, ZBW - Leibniz Information Centre for Economics.
- Abel Brodeur, Nikolai M. Cook, Anthony Heyes, 2022.
"We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments,"
LCERPA Working Papers
am0133, Laurier Centre for Economic Research and Policy Analysis.
- Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2022. "We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments," MetaArXiv a9vhr, Center for Open Science.
- Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2022. "We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments," IZA Discussion Papers 15478, Institute of Labor Economics (IZA).
- Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2022.
"We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments,"
GLO Discussion Paper Series
1157, Global Labor Organization (GLO).
- Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2022. "We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about p-Hacking and Publication Bias in Online Experiments," I4R Discussion Paper Series 8, The Institute for Replication (I4R).
- Anna Dreber & Magnus Johannesson & Yifan Yang, 2024.
"Selective reporting of placebo tests in top economics journals,"
Economic Inquiry, Western Economic Association International, vol. 62(3), pages 921-932, July.
- Dreber, Anna & Johannesson, Magnus & Yang, Yifan, 2023. "Selective Reporting of Placebo Tests in Top Economics Journals," I4R Discussion Paper Series 31, The Institute for Replication (I4R).
- Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
- Stanley, T. D. & Doucouliagos, Chris, 2019. "Practical Significance, Meta-Analysis and the Credibility of Economics," IZA Discussion Papers 12458, Institute of Labor Economics (IZA).
- Tomas Havranek & Zuzana Irsova & Lubica Laslopova & Olesia Zeynalova, 2020.
"Skilled and Unskilled Labor Are Less Substitutable than Commonly Thought,"
Working Papers IES
2020/29, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Sep 2020.
- Havranek, Tomas & Irsova, Zuzana & Laslopova, Lubica & Zeynalova, Olesia, 2021. "Skilled and Unskilled Labor Are Less Substitutable than Commonly Thought," CEPR Discussion Papers 15724, C.E.P.R. Discussion Papers.
- Havranek, Tomas & Irsova, Zuzana & Laslopova, Lubica & Zeynalova, Olesia, 2020. "Skilled and Unskilled Labor Are Less Substitutable than Commonly Thought," EconStor Preprints 223060, ZBW - Leibniz Information Centre for Economics.
- Abel Brodeur & Nikolai M. Cook & Jonathan S. Hartley & Anthony Heyes, 2024.
"Do Preregistration and Preanalysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement,"
Journal of Political Economy Microeconomics, University of Chicago Press, vol. 2(3), pages 527-561.
- Brodeur, Abel & Cook, Nikolai M. & Hartley, Jonathan S. & Heyes, Anthony, 2023. "Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias?: Evidence from 15,992 Test Statistics and Suggestions for Improvement," GLO Discussion Paper Series 1147 [pre.], Global Labor Organization (GLO).
- Brodeur, Abel & Cook, Nikolai M. & Hartley, Jonathan S. & Heyes, Anthony, 2024. "Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement," I4R Discussion Paper Series 101, The Institute for Replication (I4R).
- Felix Chopra & Ingar Haaland & Christopher Roth & Andreas Stegmann, 2024.
"The Null Result Penalty,"
The Economic Journal, Royal Economic Society, vol. 134(657), pages 193-219.
- Felix Chopra & Ingar Haaland & Christopher Roth & Andreas Stegmann, 2023. "The Null Result Penalty," The Economic Journal, Royal Economic Society, vol. 134(657), pages 193-219.
- Felix Chopra & Ingar K. Haaland & Christopher Roth & Andreas Stegmann, 2022. "The Null Result Penalty," CESifo Working Paper Series 9776, CESifo.
- Felix Chopra & Ingar Haaland & Christopher Roth & Andreas Stegmann, 2022. "The Null Result Penalty," ECONtribute Discussion Papers Series 169, University of Bonn and University of Cologne, Germany.
- Chopra, Felix & Haaland, Ingar & Roth, Christopher & Stegmann, Andreas, 2022. "The Null Result Penalty," CEPR Discussion Papers 17331, C.E.P.R. Discussion Papers.
- Matteo Picchio & Michele Ubaldi, 2024.
"Unemployment and health: A meta‐analysis,"
Journal of Economic Surveys, Wiley Blackwell, vol. 38(4), pages 1437-1472, September.
- Picchio, Matteo & Ubaldi, Michele, 2022. "Unemployment and Health: A Meta-Analysis," GLO Discussion Paper Series 1128, Global Labor Organization (GLO).
- Matteo Picchio & Michele Ubaldi, 2022. "Unemployment And Health: A Meta-Analysis," Working Papers 467, Universita' Politecnica delle Marche (I), Dipartimento di Scienze Economiche e Sociali.
- Picchio, Matteo & Ubaldi, Michele, 2022. "Unemployment and Health: A Meta-Analysis," IZA Discussion Papers 15433, Institute of Labor Economics (IZA).
- Ali Elminejad & Tomas Havranek & Roman Horvath & Zuzana Irsova, 2023.
"Intertemporal Substitution in Labor Supply: A Meta-Analysis,"
Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 51, pages 1095-1113, December.
- Ali Elminejad & Tomas Havranek & Roman Horvath & Zuzana Irsova, 2023. "Online Appendix to "Intertemporal Substitution in Labor Supply: A Meta-Analysis"," Online Appendices 23-196, Review of Economic Dynamics.
- Ali Elminejad & Tomas Havranek & Roman Horvath & Zuzana Irsova, 2023. "Code and data files for "Intertemporal Substitution in Labor Supply: A Meta-Analysis"," Computer Codes 23-196, Review of Economic Dynamics.
- Doucouliagos, Hristos & Hinz, Thomas & Zigova, Katarina, 2022.
"Bias and careers: Evidence from the aid effectiveness literature,"
European Journal of Political Economy, Elsevier, vol. 71(C).
- Doucouliagos, Chris & Hinz, Thomas & Zigova, Katarina, 2020. "Bias and Careers: Evidence from the Aid Effectiveness Literature," IZA Discussion Papers 13287, Institute of Labor Economics (IZA).
- Bruns, Stephan B. & Asanov, Igor & Bode, Rasmus & Dunger, Melanie & Funk, Christoph & Hassan, Sherif M. & Hauschildt, Julia & Heinisch, Dominik & Kempa, Karol & König, Johannes & Lips, Johannes & Verb, 2019. "Reporting errors and biases in published empirical findings: Evidence from innovation research," Research Policy, Elsevier, vol. 48(9), pages 1-1.
- Furukawa, Chishio, 2019. "Publication Bias under Aggregation Frictions: Theory, Evidence, and a New Correction Method," EconStor Preprints 194798, ZBW - Leibniz Information Centre for Economics.
- Stefano DellaVigna & Elizabeth Linos, 2022.
"RCTs to Scale: Comprehensive Evidence From Two Nudge Units,"
Econometrica, Econometric Society, vol. 90(1), pages 81-116, January.
- Stefano DellaVigna & Elizabeth Linos, 2020. "RCTs to Scale: Comprehensive Evidence from Two Nudge Units," NBER Working Papers 27594, National Bureau of Economic Research, Inc.
More about this item
NEP fields
This paper has been announced in the following NEP Reports:- NEP-ECM-2022-07-11 (Econometrics)
Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2205.07950. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.