IDEAS home Printed from https://ideas.repec.org/a/spr/compst/v31y2016i4d10.1007_s00180-016-0661-7.html
   My bibliography  Save this article

Stochastic EM algorithms for parametric and semiparametric mixture models for right-censored lifetime data

Author

Listed:
  • Laurent Bordes

    (Univ. Pau & Pays de l’Adour)

  • Didier Chauveau

    (Univ. d’Orléans)

Abstract

Mixture models in reliability bring a useful compromise between parametric and nonparametric models, when several failure modes are suspected. The classical methods for estimation in mixture models rarely handle the additional difficulty coming from the fact that lifetime data are often censored, in a deterministic or random way. We present in this paper several iterative methods based on EM and Stochastic EM methodologies, that allow us to estimate parametric or semiparametric mixture models for randomly right censored lifetime data, provided they are identifiable. We consider different levels of completion for the (incomplete) observed data, and provide genuine or EM-like algorithms for several situations. In particular, we show that simulating the missing data coming from the mixture allows to plug a standard R package for survival data analysis in an EM algorithm’s M-step. Moreover, in censored semiparametric situations, a stochastic step is the only practical solution allowing computation of nonparametric estimates of the unknown survival function. The effectiveness of the new proposed algorithms are demonstrated in simulation studies and an actual dataset example from aeronautic industry.

Suggested Citation

  • Laurent Bordes & Didier Chauveau, 2016. "Stochastic EM algorithms for parametric and semiparametric mixture models for right-censored lifetime data," Computational Statistics, Springer, vol. 31(4), pages 1513-1538, December.
  • Handle: RePEc:spr:compst:v:31:y:2016:i:4:d:10.1007_s00180-016-0661-7
    DOI: 10.1007/s00180-016-0661-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00180-016-0661-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s00180-016-0661-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lee, Gyemin & Scott, Clayton, 2012. "EM algorithms for multivariate Gaussian mixture models with truncated and censored data," Computational Statistics & Data Analysis, Elsevier, vol. 56(9), pages 2816-2829.
    2. Cao, Ricardo & Janssen, Paul & Veraverbeke, Noel, 2001. "Relative density estimation and local bandwidth selection for censored data," Computational Statistics & Data Analysis, Elsevier, vol. 36(4), pages 497-510, June.
    3. Bordes, Laurent & Chauveau, Didier & Vandekerkhove, Pierre, 2007. "A stochastic EM algorithm for a semiparametric mixture model," Computational Statistics & Data Analysis, Elsevier, vol. 51(11), pages 5429-5443, July.
    4. Eric Beutner & Laurent Bordes, 2011. "Estimators Based on Data‐Driven Generalized Weighted Cramér‐von Mises Distances under Censoring – with Applications to Mixture Models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 38(1), pages 108-129, March.
    5. Dirick, Lore & Claeskens, Gerda & Baesens, Bart, 2015. "An Akaike information criterion for multiple event mixture cure models," European Journal of Operational Research, Elsevier, vol. 241(2), pages 449-457.
    6. Akio Suzukawa & Hideyuki Imai & Yoshiharu Sato, 2001. "Kullback-Leibler Information Consistent Estimation for Censored Data," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 53(2), pages 262-276, June.
    7. Castet, Jean-Francois & Saleh, Joseph H., 2010. "Single versus mixture Weibull distributions for nonparametric satellite reliability," Reliability Engineering and System Safety, Elsevier, vol. 95(3), pages 295-300.
    8. Laurent Bordes & Céline Delmas & Pierre Vandekerkhove, 2006. "Semiparametric Estimation of a Two‐component Mixture Model where One Component is known," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 33(4), pages 733-752, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Semhar Michael & Tatjana Miljkovic & Volodymyr Melnykov, 2020. "Mixture modeling of data with multiple partial right-censoring levels," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 14(2), pages 355-378, June.
    2. Ducros, Florence & Pamphile, Patrick, 2018. "Bayesian estimation of Weibull mixture in heavily censored data setting," Reliability Engineering and System Safety, Elsevier, vol. 180(C), pages 453-462.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jaspers, Stijn & Aerts, Marc & Verbeke, Geert & Beloeil, Pierre-Alexandre, 2014. "A new semi-parametric mixture model for interval censored data, with applications in the field of antimicrobial resistance," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 30-42.
    2. Chauveau, Didier & Hoang, Vy Thuy Lynh, 2016. "Nonparametric mixture models with conditionally independent multivariate component densities," Computational Statistics & Data Analysis, Elsevier, vol. 103(C), pages 1-16.
    3. Seo, Byungtae, 2017. "The doubly smoothed maximum likelihood estimation for location-shifted semiparametric mixtures," Computational Statistics & Data Analysis, Elsevier, vol. 108(C), pages 27-39.
    4. Jiali Zheng & Xiyang Wang, 2022. "Estimation for a Class of Semiparametric Pareto Mixture Densities," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 84(2), pages 609-627, August.
    5. Xiang, Sijia & Yao, Weixin & Seo, Byungtae, 2016. "Semiparametric mixture: Continuous scale mixture approach," Computational Statistics & Data Analysis, Elsevier, vol. 103(C), pages 413-425.
    6. Madeleine Cule & Richard Samworth & Michael Stewart, 2010. "Maximum likelihood estimation of a multi‐dimensional log‐concave density," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(5), pages 545-607, November.
    7. Mazo, Gildas & Averyanov, Yaroslav, 2019. "Constraining kernel estimators in semiparametric copula mixture models," Computational Statistics & Data Analysis, Elsevier, vol. 138(C), pages 170-189.
    8. Dirick, Lore & Claeskens, Gerda & Vasnev, Andrey & Baesens, Bart, 2022. "A hierarchical mixture cure model with unobserved heterogeneity for credit risk," Econometrics and Statistics, Elsevier, vol. 22(C), pages 39-55.
    9. Dirick, Lore & Claeskens, Gerda & Baesens, Bart, 2015. "An Akaike information criterion for multiple event mixture cure models," European Journal of Operational Research, Elsevier, vol. 241(2), pages 449-457.
    10. Aldo M. Garay & Victor H. Lachos & Heleno Bolfarine & Celso R. B. Cabral, 2017. "Linear censored regression models with scale mixtures of normal distributions," Statistical Papers, Springer, vol. 58(1), pages 247-278, March.
    11. Stéphane Bonhomme & Koen Jochmans & Jean-Marc Robin, 2016. "Non-parametric estimation of finite mixtures from repeated measurements," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(1), pages 211-229, January.
    12. Jung, Seunghwa & Choi, Jihwan P., 2019. "Predicting system failure rates of SRAM-based FPGA on-board processors in space radiation environments," Reliability Engineering and System Safety, Elsevier, vol. 183(C), pages 374-386.
    13. Elisa–María Molanes-López & Ricardo Cao, 2008. "Relative density estimation for left truncated and right censored data," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 20(8), pages 693-720.
    14. Michele Bavaro & Federico Tullio, 2023. "Intergenerational mobility measurement with latent transition matrices," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 21(1), pages 25-45, March.
    15. Diego Tomassi & Liliana Forzani & Efstathia Bura & Ruth Pfeiffer, 2017. "Sufficient dimension reduction for censored predictors," Biometrics, The International Biometric Society, vol. 73(1), pages 220-231, March.
    16. Doksum, Kjell A. & Jiang, Jiancheng & Sun, Bo & Wang, Shuzhen, 2017. "Nearest neighbor estimates of regression," Computational Statistics & Data Analysis, Elsevier, vol. 110(C), pages 64-74.
    17. Róbert Csalódi & Zoltán Birkner & János Abonyi, 2021. "Learning Interpretable Mixture of Weibull Distributions—Exploratory Analysis of How Economic Development Influences the Incidence of COVID-19 Deaths," Data, MDPI, vol. 6(12), pages 1-11, November.
    18. Cavalcante, C.A.V. & Lopes, R.S. & Scarf, P.A., 2018. "A general inspection and opportunistic replacement policy for one-component systems of variable quality," European Journal of Operational Research, Elsevier, vol. 266(3), pages 911-919.
    19. repec:hal:spmain:info:hdl:2441/etefo8s8r89oamhnhiclqr530 is not listed on IDEAS
    20. repec:spo:wpmain:info:hdl:2441/etefo8s8r89oamhnhiclqr530 is not listed on IDEAS
    21. Rostyslav Maiboroda & Olena Sugakova, 2012. "Nonparametric density estimation for symmetric distributions by contaminated data," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 75(1), pages 109-126, January.
    22. Lessmann, Stefan & Baesens, Bart & Seow, Hsin-Vonn & Thomas, Lyn C., 2015. "Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research," European Journal of Operational Research, Elsevier, vol. 247(1), pages 124-136.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:compst:v:31:y:2016:i:4:d:10.1007_s00180-016-0661-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.