IDEAS home Printed from https://ideas.repec.org/p/uct/uconnp/2005-10.html
   My bibliography  Save this paper

Optimally Combining Censored and Uncensored Datasets

Author

Listed:
  • Paul J. Devereux

    (UCLA)

  • Gautam Tripathi

    (University of Connecticut)

Abstract

Economists and other social scientists often face situations where they have access to two datasets that they can use but one set of data suffers from censoring or truncation. If the censored sample is much bigger than the uncensored sample, it is common for researchers to use the censored sample alone and attempt to deal with the problem of partial observation in some manner. Alternatively, they simply use only the uncensored sample and ignore the censored one so as to avoid biases. It is rarely the case that researchers use both datasets together, mainly because they lack guidance about how to combine them. In this paper, we develop a simple semiparametric framework for combining the censored and uncensored datasets so that the resulting estimators are consistent, asymptotically normal, and use all information optimally. No nonparametric smoothing is required to implement our estimators. To illustrate our results in an empirical setting, we show how to estimate the effect of changes in compulsory schooling laws on age at first marriage, a variable that is censored for younger individuals. We also demonstrate how refreshment samples for this application can be created by combining cohort information across census datasets. Results from a small simulation experiment suggest that the estimator proposed in this paper can work very well in finite samples.

Suggested Citation

  • Paul J. Devereux & Gautam Tripathi, 2005. "Optimally Combining Censored and Uncensored Datasets," Working papers 2005-10, University of Connecticut, Department of Economics, revised Oct 2007.
  • Handle: RePEc:uct:uconnp:2005-10
    as

    Download full text from publisher

    File URL: https://media.economics.uconn.edu/working/2005-10r.pdf
    File Function: Full text (revised version)
    Download Restriction: no

    File URL: https://media.economics.uconn.edu/working/2005-10.pdf
    File Function: Full text (original version)
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Sanders Korenman & David Neumark, 1992. "Marriage, Motherhood, and Wages," Journal of Human Resources, University of Wisconsin Press, vol. 27(2), pages 233-255.
    2. Powell, James L, 1986. "Symmetrically Trimmed Least Squares Estimation for Tobit Models," Econometrica, Econometric Society, vol. 54(6), pages 1435-1460, November.
    3. Whitney K. Newey & Richard J. Smith, 2004. "Higher Order Properties of Gmm and Generalized Empirical Likelihood Estimators," Econometrica, Econometric Society, vol. 72(1), pages 219-255, January.
    4. Judith K. Hellerstein & Guido W. Imbens, 1999. "Imposing Moment Restrictions From Auxiliary Data By Weighting," The Review of Economics and Statistics, MIT Press, vol. 81(1), pages 1-14, February.
    5. Claudia Goldin & Lawrence Katz, 2003. "Mass Secondary Schooling and the State," NBER Working Papers 10075, National Bureau of Economic Research, Inc.
    6. Guggenberger, Patrik & Smith, Richard J., 2005. "Generalized Empirical Likelihood Estimators And Tests Under Partial, Weak, And Strong Identification," Econometric Theory, Cambridge University Press, vol. 21(4), pages 667-709, August.
    7. Claudia Goldin & Lawrence F. Katz, 2002. "The Power of the Pill: Oral Contraceptives and Women's Career and Marriage Decisions," Journal of Political Economy, University of Chicago Press, vol. 110(4), pages 730-770, August.
    8. Bergstrom, Ted & Schoeni, Robert F, 1996. "Income Prospects and Age-at-Marriage," Journal of Population Economics, Springer;European Society for Population Economics, vol. 9(2), pages 115-130, May.
    9. Heather Koball, 1998. "Have african american men become less committed to marriage? Explaining the twentieth century racial cross-over in men’s marriage timing," Demography, Springer;Population Association of America (PAA), vol. 35(2), pages 251-258, May.
    10. Powell, James L., 1984. "Least absolute deviations estimation for the censored regression model," Journal of Econometrics, Elsevier, vol. 25(3), pages 303-325, July.
    11. Guido W. Imbens & Richard H. Spady & Phillip Johnson, 1998. "Information Theoretic Approaches to Inference in Moment Condition Models," Econometrica, Econometric Society, vol. 66(2), pages 333-358, March.
    12. Xiaohong Chen & Han Hong & Elie Tamer, 2005. "Measurement Error Models with Auxiliary Data," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 72(2), pages 343-366.
    13. Guido W. Imbens & Tony Lancaster, 1994. "Combining Micro and Macro Data in Microeconometric Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 61(4), pages 655-680.
    14. Joshua D. Angrist & Alan B. Krueger, 1990. "The Effect of Age at School Entry on Educational Attainment: An Application of Instrumental Variables with Moments from Two Samples," NBER Working Papers 3571, National Bureau of Economic Research, Inc.
    15. Lleras-Muney, Adriana, 2002. "Were Compulsory Attendance and Child Labor Laws Effective? An Analysis from 1915 to 1939," Journal of Law and Economics, University of Chicago Press, vol. 45(2), pages 401-435, October.
    16. Powell, James L., 1986. "Censored regression quantiles," Journal of Econometrics, Elsevier, vol. 32(1), pages 143-155, June.
    17. Lance Lochner & Enrico Moretti, 2004. "The Effect of Education on Crime: Evidence from Prison Inmates, Arrests, and Self-Reports," American Economic Review, American Economic Association, vol. 94(1), pages 155-189, March.
    18. Oreopoulos, Philip, 2007. "Do dropouts drop out too soon? Wealth, health and happiness from compulsory schooling," Journal of Public Economics, Elsevier, vol. 91(11-12), pages 2213-2229, December.
    19. Yuichi Kitamura, 2001. "Asymptotic Optimality of Empirical Likelihood for Testing Moment Restrictions," Econometrica, Econometric Society, vol. 69(6), pages 1661-1672, November.
    20. Keisuke Hirano & Guido W. Imbens & Geert Ridder & Donald B. Rubin, 2001. "Combining Panel Data Sets with Attrition and Refreshment Samples," Econometrica, Econometric Society, vol. 69(6), pages 1645-1659, November.
    21. Sanders Korenman & David Neumark, 1991. "Does Marriage Really Make Men More Productive?," Journal of Human Resources, University of Wisconsin Press, vol. 26(2), pages 282-307.
    22. Severini, Thomas A. & Tripathi, Gautam, 2001. "A simplified approach to computing efficiency bounds in semiparametric models," Journal of Econometrics, Elsevier, vol. 102(1), pages 23-66, May.
    23. McFadden, Daniel & Ruud, Paul A, 1994. "Estimation by Simulation," The Review of Economics and Statistics, MIT Press, vol. 76(4), pages 591-608, November.
    24. Akerlof, George A, 1998. "Men without Children," Economic Journal, Royal Economic Society, vol. 108(447), pages 287-309, March.
    25. Ridder, Geert & Moffitt, Robert, 2007. "The Econometrics of Data Combination," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 75, Elsevier.
    26. SandraE. Black & PaulJ. Devereux & KjellG. Salvanes, 2008. "Staying in the Classroom and out of the maternity ward? The effect of compulsory schooling laws on teenage births," Economic Journal, Royal Economic Society, vol. 118(530), pages 1025-1054, July.
    27. Smith, Richard J, 1997. "Alternative Semi-parametric Likelihood Approaches to Generalised Method of Moments Estimation," Economic Journal, Royal Economic Society, vol. 107(441), pages 503-519, March.
    28. Daron Acemoglu & Joshua Angrist, 2001. "How Large Are Human Capital Externalities? Evidence from Compulsory Schooling Laws," NBER Chapters, in: NBER Macroeconomics Annual 2000, Volume 15, pages 9-74, National Bureau of Economic Research, Inc.
    29. Xiaohong Chen & Han Hong & Alessandro Tarozzi, 2008. "Semiparametric Efficiency in GMM Models of Nonclassical Measurement Errors, Missing Data and Treatment Effects," Cowles Foundation Discussion Papers 1644, Cowles Foundation for Research in Economics, Yale University.
    30. Geert Ridder & Yingyao Hu, 2004. "Estimation of Nonlinear Models with Measurement Error Using Marginal Information," Econometric Society 2004 North American Summer Meetings 21, Econometric Society.
    31. Manuel Arellano & Costas Meghir, 1992. "Female Labour Supply and On-the-Job Search: An Empirical Model Estimated Using Complementary Data Sets," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 59(3), pages 537-559.
    32. Yuichi Kitamura, 2006. "Empirical Likelihood Methods in Econometrics: Theory and Practice," Cowles Foundation Discussion Papers 1569, Cowles Foundation for Research in Economics, Yale University.
    33. Guido W. Imbens, 1997. "One-Step Estimators for Over-Identified Generalized Method of Moments Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(3), pages 359-383.
    34. Amemiya, Takeshi, 1984. "Tobit models: A survey," Journal of Econometrics, Elsevier, vol. 24(1-2), pages 3-61.
    35. Yuichi Kitamura, 2006. "Empirical Likelihood Methods in Econometrics: Theory and Practice," Levine's Bibliography 321307000000000307, UCLA Department of Economics.
    36. Nevo, Aviv, 2003. "Using Weights to Adjust for Sample Selection When Auxiliary Information Is Available," Journal of Business & Economic Statistics, American Statistical Association, vol. 21(1), pages 43-52, January.
    37. Yuichi Kitamura, 2006. "Empirical Likelihood Methods in Econometrics: Theory and Practice," CIRJE F-Series CIRJE-F-430, CIRJE, Faculty of Economics, University of Tokyo.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Maria K. Humlum & Jannie H.G. Kristoffersen & Rune Vejlin, 2012. "Timing of College Enrollment and Family Formation Decisions," Economics Working Papers 2012-01, Department of Economics and Business Economics, Aarhus University.
    2. Humlum, Maria Knoth & Kristoffersen, Jannie H.G. & Vejlin, Rune, 2017. "College admissions decisions, educational outcomes, and family formation," Labour Economics, Elsevier, vol. 48(C), pages 215-230.
    3. Powdthavee, Nattavudh & Adireksombat, Kampon, 2010. "From Classroom to Wedding Aisle: The Effect of a Nationwide Change in the Compulsory Schooling Law on Age at First Marriage in the UK," IZA Discussion Papers 5019, Institute of Labor Economics (IZA).
    4. Karimi, Seyed M. & Taghvatalab, Golnaz, 2020. "Access to higher education and the likelihood of being married," The Quarterly Review of Economics and Finance, Elsevier, vol. 78(C), pages 22-33.
    5. Xiaohong Chen & Han Hong & Denis Nekipelov, 2011. "Nonlinear Models of Measurement Errors," Journal of Economic Literature, American Economic Association, vol. 49(4), pages 901-937, December.
    6. Josefine Koebe & Jan Marcus, 2022. "The Length of Schooling and the Timing of Family Formation [Income Taxes and the Timing of Marital Decisions]," CESifo Economic Studies, CESifo Group, vol. 68(1), pages 1-45.
    7. Josefine Koebe & Jan Marcus, 2020. "The Impact of the Length of Schooling on the Timing of Family Formation," Discussion Papers of DIW Berlin 1896, DIW Berlin, German Institute for Economic Research.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bryan S. Graham & Cristine Campos De Xavier Pinto & Daniel Egel, 2012. "Inverse Probability Tilting for Moment Condition Models with Missing Data," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 79(3), pages 1053-1079.
    2. Parente, Paulo M.D.C. & Smith, Richard J., 2011. "Gel Methods For Nonsmooth Moment Indicators," Econometric Theory, Cambridge University Press, vol. 27(1), pages 74-113, February.
    3. Stefan Boes, 2007. "Count Data Models with Unobserved Heterogeneity: An Empirical Likelihood Approach," SOI - Working Papers 0704, Socioeconomic Institute - University of Zurich.
    4. Prosper Dovonon, 2016. "Large Sample Properties of the Three-Step Euclidean Likelihood Estimators under Model Misspecification," Econometric Reviews, Taylor & Francis Journals, vol. 35(4), pages 465-514, April.
    5. Giuseppe Ragusa, 2011. "Minimum Divergence, Generalized Empirical Likelihoods, and Higher Order Expansions," Econometric Reviews, Taylor & Francis Journals, vol. 30(4), pages 406-456, August.
    6. Yuichi Kitamura & Taisuke Otsu & Kirill Evdokimov, 2013. "Robustness, Infinitesimal Neighborhoods, and Moment Restrictions," Econometrica, Econometric Society, vol. 81(3), pages 1185-1201, May.
    7. Richard Smith, 2005. "Weak instruments and empirical likelihood: a discussion of the papers by DWK Andrews and JH Stock and Y Kitamura," CeMMAP working papers CWP13/05, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    8. Joachim Inkmann, 2010. "Estimating Firm Size Elasticities of Product and Process R&D," Economica, London School of Economics and Political Science, vol. 77(306), pages 384-402, April.
    9. Xu, Ke-Li, 2020. "Inference of local regression in the presence of nuisance parameters," Journal of Econometrics, Elsevier, vol. 218(2), pages 532-560.
    10. Esmeralda A. Ramalho & Richard J. Smith, 2013. "Discrete Choice Non-Response," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 80(1), pages 343-364.
    11. Canay, Ivan A., 2010. "EL inference for partially identified models: Large deviations optimality and bootstrap validity," Journal of Econometrics, Elsevier, vol. 156(2), pages 408-425, June.
    12. Yuichi Kitamura, 2006. "Empirical Likelihood Methods in Econometrics: Theory and Practice," CIRJE F-Series CIRJE-F-430, CIRJE, Faculty of Economics, University of Tokyo.
    13. Whang, Yoon-Jae, 2006. "Smoothed Empirical Likelihood Methods For Quantile Regression Models," Econometric Theory, Cambridge University Press, vol. 22(2), pages 173-205, April.
    14. Gordon Dahl, 2010. "Early teen marriage and future poverty," Demography, Springer;Population Association of America (PAA), vol. 47(3), pages 689-718, August.
    15. Jin, Fei & Lee, Lung-fei, 2019. "GEL estimation and tests of spatial autoregressive models," Journal of Econometrics, Elsevier, vol. 208(2), pages 585-612.
    16. Gregory A. Gilpin & Luke A. Pennig, 2012. "Compulsory Schooling Laws and In-School Crime: Are Delinquents Incapacitated?," Caepr Working Papers 2012-005, Center for Applied Economics and Policy Research, Economics Department, Indiana University Bloomington.
    17. F Bravo, 2008. "Effcient M-estimators with auxiliary information," Discussion Papers 08/26, Department of Economics, University of York.
    18. Black, Sandra E. & Devereux, Paul J. & Salvanes, Kjell G., 2004. "Fast Times at Ridgemont High? The Effect of Compulsory Schooling Laws on Teenage Births," IZA Discussion Papers 1416, Institute of Labor Economics (IZA).
    19. Xiao, Zhiguo, 2010. "The weighted method of moments approach for moment condition models," Economics Letters, Elsevier, vol. 107(2), pages 183-186, May.
    20. Murat G. Kirdar & Meltem Dayioglu & Ismet Koç, 2013. "Does Longer Compulsory Education Equalize Educational Attainment? Evidence From A Major Policy Reform," Working Papers 777, Economic Research Forum, revised Oct 2013.

    More about this item

    Keywords

    Censoring; Empirical Likelihood; GMM; Refreshment samples; Truncation;
    All these keywords.

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C24 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Truncated and Censored Models; Switching Regression Models; Threshold Regression Models
    • C34 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Truncated and Censored Models; Switching Regression Models
    • C51 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Construction and Estimation

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:uct:uconnp:2005-10. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Mark McConnel (email available below). General contact details of provider: https://edirc.repec.org/data/deuctus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.