IDEAS home Printed from https://ideas.repec.org/a/spr/stmapp/v32y2023i1d10.1007_s10260-022-00650-5.html
   My bibliography  Save this article

Maximum likelihood estimation of missing data probability for nonmonotone missing at random data

Author

Listed:
  • Yang Zhao

    (University of Regina)

Abstract

In general, statistical analysis with missing data requires specification of a model for the missing data probability and/or the covariate distribution. For nonmonotone missing data patterns, modeling and practical estimation of the missing data probability are very challenging. Recently a semiparametric likelihood model was developed to estimate parametric regression models for the missing data mechanism based on all the observed data, which can deal with arbitrary nonmonotone missing data patterns. However, due to the curse of dimensionality in the likelihood-based models, this method becomes impractical if the number of variables increases. This research generalizes the semiparametric likelihood model such that it can deal with any number of variables with arbitrary nonmonotone missing data patterns. It further introduces a semiparametric estimator of the missing data probability for the partially observed data, which can be used to assess the model fit. An EM algorithm with closed form expressions at each step are used to compute the estimates. Simulation studies in various settings indicate that the performance of the new method is acceptable for practical implementation. The missing data mechanism of a case-control study of hip fractures among male veterans is analyzed to illustrate the method.

Suggested Citation

  • Yang Zhao, 2023. "Maximum likelihood estimation of missing data probability for nonmonotone missing at random data," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 32(1), pages 197-209, March.
    • Handle: RePEc:spr:stmapp:v:32:y:2023:i:1:d:10.1007_s10260-022-00650-5
    DOI: 10.1007/s10260-022-00650-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10260-022-00650-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10260-022-00650-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Joseph G. Ibrahim & Ming-Hui Chen & Stuart R. Lipsitz & Amy H. Herring, 2005. "Missing-Data Methods for Generalized Linear Models: A Comparative Review," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 332-346, March.
    2. Hua Yun Chen, 2004. "Nonparametric and Semiparametric Models for Missing Covariates in Parametric Regression," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 1176-1189, December.
    3. BaoLuo Sun & Eric J. Tchetgen Tchetgen, 2018. "On Inverse Probability Weighting for Nonmonotone Missing at Random Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(521), pages 369-379, January.
    4. Yang Zhao, 2021. "Semiparametric model for regression analysis with nonmonotone missing data," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(2), pages 461-475, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang Zhao & Meng Liu, 2021. "Unified approach for regression models with nonmonotone missing at random data," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 105(1), pages 87-101, March.
    2. Yang Zhao, 2021. "Semiparametric model for regression analysis with nonmonotone missing data," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(2), pages 461-475, June.
    3. Yi Qian & Hui Xie, 2011. "No Customer Left Behind: A Distribution-Free Bayesian Approach to Accounting for Missing Xs in Marketing Models," Marketing Science, INFORMS, vol. 30(4), pages 717-736, July.
    4. Zhuoer Sun & Suojin Wang, 2019. "Semiparametric estimation in regression with missing covariates using single-index models," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 71(5), pages 1201-1232, October.
    5. Richard M. Golden & Steven S. Henley & Halbert White & T. Michael Kashner, 2019. "Consequences of Model Misspecification for Maximum Likelihood Estimation with Missing Data," Econometrics, MDPI, vol. 7(3), pages 1-27, September.
    6. Hua Yun Chen & Hui Xie & Yi Qian, 2011. "Multiple Imputation for Missing Values through Conditional Semiparametric Odds Ratio Models," Biometrics, The International Biometric Society, vol. 67(3), pages 799-809, September.
    7. Yi Qian & Hui Xie, 2014. "Which Brand Purchasers Are Lost to Counterfeiters? An Application of New Data Fusion Approaches," Marketing Science, INFORMS, vol. 33(3), pages 437-448, May.
    8. Ryo Kato & Takahiro Hoshino, 2020. "Semiparametric Bayesian multiple imputation for regression models with missing mixed continuous–discrete covariates," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 72(3), pages 803-825, June.
    9. Li Cai & Lijie Gu & Qihua Wang & Suojin Wang, 2021. "Simultaneous confidence bands for nonparametric regression with missing covariate data," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 73(6), pages 1249-1279, December.
    10. McDonough, Ian K. & Millimet, Daniel L., 2017. "Missing data, imputation, and endogeneity," Journal of Econometrics, Elsevier, vol. 199(2), pages 141-155.
    11. J. Andrew Royle, 2009. "Analysis of Capture–Recapture Models with Individual Covariates Using Data Augmentation," Biometrics, The International Biometric Society, vol. 65(1), pages 267-274, March.
    12. Xie Yanmei & Zhang Biao, 2017. "Empirical Likelihood in Nonignorable Covariate-Missing Data Problems," The International Journal of Biostatistics, De Gruyter, vol. 13(1), pages 1-20, May.
    13. Breunig, Christoph, 2015. "Testing missing at random using instrumental variables," SFB 649 Discussion Papers 2015-016, Humboldt University Berlin, Collaborative Research Center 649: Economic Risk.
    14. Zhiwei Zhang & Howard Rockette, 2006. "Semiparametric Maximum Likelihood for Missing Covariates in Parametric Regression," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 58(4), pages 687-706, December.
    15. Jiang, Depeng & Zhao, Puying & Tang, Niansheng, 2016. "A propensity score adjustment method for regression models with nonignorable missing covariates," Computational Statistics & Data Analysis, Elsevier, vol. 94(C), pages 98-119.
    16. Lei Jin & Suojin Wang, 2010. "A Model Validation Procedure when Covariate Data are Missing at Random," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 37(3), pages 403-421, September.
    17. Jonathan S. Schildcrout & Shawn P. Garbett & Patrick J. Heagerty, 2013. "Outcome Vector Dependent Sampling with Longitudinal Continuous Response Data: Stratified Sampling Based on Summary Statistics," Biometrics, The International Biometric Society, vol. 69(2), pages 405-416, June.
    18. J. F. Lawless, 2018. "Two-phase outcome-dependent studies for failure times and testing for effects of expensive covariates," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 24(1), pages 28-44, January.
    19. Hui Yao & Sungduk Kim & Ming-Hui Chen & Joseph G. Ibrahim & Arvind K. Shah & Jianxin Lin, 2015. "Bayesian Inference for Multivariate Meta-Regression With a Partially Observed Within-Study Sample Covariance Matrix," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(510), pages 528-544, June.
    20. Jiang, Wei & Josse, Julie & Lavielle, Marc, 2020. "Logistic regression with missing covariates—Parameter estimation, model selection and prediction within a joint-modeling framework," Computational Statistics & Data Analysis, Elsevier, vol. 145(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stmapp:v:32:y:2023:i:1:d:10.1007_s10260-022-00650-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.