The Cult of Statistical Significance – What Economists Should and Should Not Do to Make their Data Talk

This article takes issue with a rather devastating critique of statistical significance testing propagated in a recent book by Ziliak/McCloskey (2008) of the same title. Ziliak/McCloskey argue that statistical significance testing is a barrier rather than a booster for empirical research in economics and should therefore be abandoned altogether. The present article argues that this is good advice in some research areas but not in others, with the aim of making practitioners aware of various fallacies connected with the concept of statistical significance, and at the same time showing where significance testing is most fruitfully employed. Taking all issues which have appeared so far of the German Economic Review and a recent epidemiological meta-analysis as examples, the present paper shows that there has indeed been a lot of misleading work where confirmatory significance testing has played a major role, and that at the same time many promising avenues, best summarized under the heading “specification tests”, have not been used.