IDEAS home Printed from https://ideas.repec.org/p/ags/rffdps/10596.html
   My bibliography  Save this paper

The Analysis of Randomized Experiments with Missing Data

Author

Listed:
  • Imbens, Guido W.
  • Pizer, William A.

Abstract

The otherwise straightforward analysis of randomized experiments is often complicated by the presence of missing data. In such situations it is necessary to make assumptions about the dependence of the selection mechanism on treatment, response, and covariates. The widely used approach of assuming that the data is missing at random conditional on treatment and other fully observed covariates is shown to be inadequate to describe data from a randomized experiment when partially observed covariates are also present. This paper presents an alternative to the missing at random model (MAR) which is both consistent with the data and preserves the appeal of MAR. In particular, the proposed family of models minimize the discrepancy with MAR while explaining observed deviations. We apply this approach to data from the Restart job training program in the United Kingdom as well as an artificial data set. Evaluation of the Restart program is not affected by the assumption of MAR; both approaches suggest that the program increased the chances of exiting unemployment by around 9% within six months. However, analysis of the artificial data demonstrates that assuming MAR can easily lead to erroneous conclusions.

Suggested Citation

  • Imbens, Guido W. & Pizer, William A., 2000. "The Analysis of Randomized Experiments with Missing Data," Discussion Papers 10596, Resources for the Future.
    • Handle: RePEc:ags:rffdps:10596
    DOI: 10.22004/ag.econ.10596
    as

    Download full text from publisher

    File URL: https://ageconsearch.umn.edu/record/10596/files/dp000019.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.22004/ag.econ.10596?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Imbens, G.W. & Rubin, D. & Sacerdote, B., 1999. "Estimating the effect of unearned income on labor supply, earnings, savings and consumption : Evidence from a survey of lottery players," Other publications TiSEM 43407245-d868-4cec-b289-c, Tilburg University, School of Economics and Management.
    2. Manski, Charles F., 1992. "Identification Problems In The Social Sciences," SSRI Workshop Series 292716, University of Wisconsin-Madison, Social Systems Research Institute.
    3. Peter Dolton & Donal O'Neill, 1996. "The Restart Effect and the Return to Full‐Time Stable Employment," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(2), pages 275-288, March.
    4. Christian Gourieroux & Alain Monfort, 1981. "On the Problem of Missing Data in Linear Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 48(4), pages 579-586.
    5. Dolton, Peter & O'Neill, Donal, 1996. "Unemployment Duration and the Restart Effect: Some Experimental Evidence," Economic Journal, Royal Economic Society, vol. 106(435), pages 387-400, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Molinari, Francesca, 2010. "Missing Treatments," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(1), pages 82-95.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Denis Conniffe & Donal O'Neill, 2011. "Efficient Probit Estimation with Partially Missing Covariates," Advances in Econometrics, in: Missing Data Methods: Cross-sectional Methods and Applications, pages 209-245, Emerald Group Publishing Limited.
    2. van den Berg, Gerard J & Lindeboom, Maarten & Dolton, Peter J, 2004. "Survey non-response and unemployment duration," Working Paper Series 2004:12, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    3. Denis Conniffe & Donal O’Neill, 2008. "An Efficient Estimator for Dealing with Missing Data on Explanatory Variables in a Probit Choice Model," Economics Department Working Paper Series n1960908.pdf, Department of Economics, National University of Ireland - Maynooth.
    4. Gerard J. Van Den Berg & Maarten Lindeboom & Peter J. Dolton, 2006. "Survey non‐response and the duration of unemployment," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(3), pages 585-604, July.
    5. Jeffrey Smith, 2000. "A Critical Survey of Empirical Methods for Evaluating Active Labor Market Policies," Swiss Journal of Economics and Statistics (SJES), Swiss Society of Economics and Statistics (SSES), vol. 136(III), pages 247-268, September.
    6. Brent Kreider & Steven C. Hill, 2009. "Partially Identifying Treatment Effects with an Application to Covering the Uninsured," Journal of Human Resources, University of Wisconsin Press, vol. 44(2).
    7. Phillips, David C., 2014. "Getting to work: Experimental evidence on job search and transportation costs," Labour Economics, Elsevier, vol. 29(C), pages 72-82.
    8. Gunnar Andersson & Karsten Hank & Andres Vikat, 2006. "Understanding parental gender preferences in advanced societies: lessons from Sweden and Finland," MPIDR Working Papers WP-2006-019, Max Planck Institute for Demographic Research, Rostock, Germany.
    9. Sascha O. Becker & Marco Caliendo, 2007. "Sensitivity analysis for average treatment effects," Stata Journal, StataCorp LP, vol. 7(1), pages 71-83, February.
    10. Helen Jensen & Justo Manrique, 1998. "Demand for food commodities by income groups in Indonesia," Applied Economics, Taylor & Francis Journals, vol. 30(4), pages 491-501.
    11. Bruno Crépon & Esther Duflo & Marc Gurgand & Roland Rathelot & Philippe Zamora, 2013. "Do Labor Market Policies have Displacement Effects? Evidence from a Clustered Randomized Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 128(2), pages 531-580.
    12. John Van Reenen, 2004. "Active Labor Market Policies and the British New Deal for the Young Unemployed in Context," NBER Chapters, in: Seeking a Premier Economy: The Economic Effects of British Economic Reforms, 1980–2000, pages 461-496, National Bureau of Economic Research, Inc.
    13. Giulio Zanella, 2004. "Discrete Choice with Social Interactions and Endogenous Memberships," Department of Economics University of Siena 442, Department of Economics, University of Siena.
    14. Alberto Abadie & Guido W. Imbens, 2002. "Simple and Bias-Corrected Matching Estimators for Average Treatment Effects," NBER Technical Working Papers 0283, National Bureau of Economic Research, Inc.
    15. Bruno Crépon & Muriel Dejemeppe & Marc Gurgand, 2005. "Counseling the unemployed: does it lower unemployment duration and recurrence?," Working Papers halshs-00590769, HAL.
    16. Bennmarker, Helge & Skans, Oskar Nordström & Vikman, Ulrika, 2013. "Workfare for the old and long-term unemployed," Labour Economics, Elsevier, vol. 25(C), pages 25-34.
    17. van Soest, Arthur & Hurd, Michael, 2008. "A Test for Anchoring and Yea-Saying in Experimental Consumption Data," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 126-136, March.
    18. Gerard J. van den Berg & Bas van der Klaauw, 2005. "Job Search Monitoring and Sanctions," ifo DICE Report, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 3(2), pages 26-29, 07.
    19. Keisuke Hirano & Guido W. Imbens & Geert Ridder & Donald B. Rubin, 2001. "Combining Panel Data Sets with Attrition and Refreshment Samples," Econometrica, Econometric Society, vol. 69(6), pages 1645-1659, November.
    20. McGuinness, Seamus & Bergin, Adele & Whelan, Adele, 2016. "An Exploration of (Area-based) Social Inclusion and Community Development Training Programmes in Ireland," Research Series, Economic and Social Research Institute (ESRI), number RS54.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ags:rffdps:10596. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: AgEcon Search (email available below). General contact details of provider: https://edirc.repec.org/data/rffffus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.