Comments On: Understanding and Misunderstanding Randomized Controlled Trails by Cartwright and Deaton

Deaton and Cartwright (DC2017 from hereon) view the increasing popularity of randomized experiments in social sciences with some skepticism. They are concerned about the quality of the inferences in practice, and fear that researchers may not fully appreciate the pitfalls and limitations of such experiments. I am more sanguine about the recent developments in empirical practice in economics and other social sciences, and am optimistic about the ongoing research in this area, both empirical and theoretical. I see the surge in use of randomized experiments as part of what Angrist and Pischke [2010] call the credibility revolution, where, starting in the late eighties and early nineties a group of researchers associated with the labor economics group at Princeton University, including Orley Ashenfelter, David Card, Alan Krueger and Joshua Angrist, led empirical researchers to pay more attention to the identification strategies underlying empirical work. This has led to important methodological developments in causal inference, including new approaches to instrumental variables, difference-in-differences, regression discontinuity designs, and, most recently, synthetic control methods (Abadie et al. [2010]). I view the increased focus on randomized experiments in particular in development economics, led by researchers such as Michael Kremer, Abhijit Banerjee, Esther Duflo, and their many coauthors and students, as taking this development even further.1 Nothwithstanding the limitations of experimentation in answering some questions, and the difficulties in implementation, these developments have greatly improved the credibility of empirical work in economics compared to the standards prior to the mid-eighties, and I view this as a major achievement by these researchers. It would be disappointing if DC2017 takes away from this, and were to move empirical practice away from the attention paid to identification and the use of randomized experiments. In the remainder of this comment I will discuss four specific issues. Some of these elaborate on points I raised in a previous discussion of D2010, Imbens [2010].