IDEAS home Printed from https://ideas.repec.org/a/spr/advdac/v8y2014i2p125-145.html
   My bibliography  Save this article

A comparison of some criteria for states selection in the latent Markov model for longitudinal data

Author

Listed:
  • S. Bacci
  • S. Pandolfi
  • F. Pennoni

Abstract

We compare different selection criteria to choose the number of latent states of a multivariate latent Markov model for longitudinal data. This model is based on an underlying Markov chain to represent the evolution of a latent characteristic of a group of individuals over time. Then, the response variables observed at different occasions are assumed to be conditionally independent given this chain. Maximum likelihood estimation of the model is carried out through an Expectation–Maximization algorithm based on forward–backward recursions which are well known in the hidden Markov literature for time series. The selection criteria we consider are based on penalized versions of the maximum log-likelihood or on the posterior probabilities of belonging to each latent state, that is, the conditional probability of the latent state given the observed data. Among the latter criteria, we propose an appropriate entropy measure tailored for the latent Markov models. We show the results of a Monte Carlo simulation study aimed at comparing the performance of the above states selection criteria on the basis of a wide set of model specifications. Copyright Springer-Verlag Berlin Heidelberg 2014

Suggested Citation

  • S. Bacci & S. Pandolfi & F. Pennoni, 2014. "A comparison of some criteria for states selection in the latent Markov model for longitudinal data," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 8(2), pages 125-145, June.
    • Handle: RePEc:spr:advdac:v:8:y:2014:i:2:p:125-145
    DOI: 10.1007/s11634-013-0154-2
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s11634-013-0154-2
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11634-013-0154-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Gilles Celeux & Gilda Soromenho, 1996. "An entropy criterion for assessing the number of clusters in a mixture model," Journal of Classification, Springer;The Classification Society, vol. 13(2), pages 195-212, September.
    2. Hamparsum Bozdogan, 1987. "Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions," Psychometrika, Springer;The Psychometric Society, vol. 52(3), pages 345-370, September.
    3. Gilles Celeux & Jean-Baptiste Durand, 2008. "Selecting hidden Markov model state number with cross-validated likelihood," Computational Statistics, Springer, vol. 23(4), pages 541-564, October.
    4. Francesco Bartolucci, 2006. "Likelihood inference for a class of latent Markov models under linear hypotheses on the transition probabilities," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(2), pages 155-178, April.
    5. Francesco Bartolucci & Fulvia Pennoni & Brian Francis, 2007. "A latent Markov model for detecting patterns of criminal activity," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 170(1), pages 115-132, January.
    6. Hathaway, Richard J., 1986. "Another interpretation of the EM algorithm for mixture distributions," Statistics & Probability Letters, Elsevier, vol. 4(2), pages 53-56, March.
    7. Clifford M. Hurvich & Chih‐Ling Tsai, 1993. "A Corrected Akaike Information Criterion For Vector Autoregressive Model Selection," Journal of Time Series Analysis, Wiley Blackwell, vol. 14(3), pages 271-279, May.
    8. Michele Costa & Luca De angelis, 2010. "Model selection in hidden Markov models : a simulation study," Quaderni di Dipartimento 7, Department of Statistics, University of Bologna.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Giorgio E. Montanari & Silvia Pandolfi, 2018. "Evaluation of long-term health care services through a latent Markov model with covariates," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 27(1), pages 151-173, March.
    2. Morgan, Grant B. & Hodge, Kari J. & Baggett, Aaron R., 2016. "Latent profile analysis with nonnormal mixtures: A Monte Carlo examination of model selection using fit indices," Computational Statistics & Data Analysis, Elsevier, vol. 93(C), pages 146-161.
    3. Leonard Paas, 2014. "Comments on: Latent Markov models: a review of a general framework for the analysis of longitudinal data with covariates," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 23(3), pages 473-477, September.
    4. Francesco Bartolucci & Fulvia Pennoni & Giorgio Vittadini, 2016. "Causal Latent Markov Model for the Comparison of Multiple Treatments in Observational Longitudinal Studies," Journal of Educational and Behavioral Statistics, , vol. 41(2), pages 146-179, April.
    5. Silvia Bacci & Francesco Bartolucci & Giulia Bettin & Claudia Pigini, 2017. "A mixture growth model for migrants' remittances: An application to the German Socio-Economic Panel," Mo.Fi.R. Working Papers 145, Money and Finance Research group (Mo.Fi.R.) - Univ. Politecnica Marche - Dept. Economic and Social Sciences.
    6. Dylan Molenaar & Paul Boeck, 2018. "Response Mixture Modeling: Accounting for Heterogeneity in Item Characteristics across Response Times," Psychometrika, Springer;The Psychometric Society, vol. 83(2), pages 279-297, June.
    7. Montanari, Giorgio E. & Doretti, Marco & Bartolucci, Francesco, 2017. "A multilevel latent Markov model for the evaluation of nursing homes' performance," MPRA Paper 80691, University Library of Munich, Germany.
    8. Fulvia Pennoni & Ewa Genge, 2020. "Analysing the course of public trust via hidden Markov models: a focus on the Polish society," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 29(2), pages 399-425, June.
    9. Giorgio Eduardo Montanari & Marco Doretti & Maria Francesca Marino, 2022. "Model-based two-way clustering of second-level units in ordinal multilevel latent Markov models," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 16(2), pages 457-485, June.
    10. Antonello Maruotti, 2015. "Handling non-ignorable dropouts in longitudinal data: a conditional model based on a latent Markov heterogeneity structure," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 24(1), pages 84-109, March.
    11. Silvia Bacci & Bruno Bertaccini, 2022. "A Mixture Hidden Markov Model to Mine Students’ University Curricula," Data, MDPI, vol. 7(2), pages 1-19, February.
    12. David Aristei & Silvia Bacci & Francesco Bartolucci & Silvia Pandolfi, 2021. "A bivariate finite mixture growth model with selection," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 15(3), pages 759-793, September.
    13. Simon DeDeo, 2016. "Conflict and Computation on Wikipedia: A Finite-State Machine Analysis of Editor Interactions," Future Internet, MDPI, vol. 8(3), pages 1-23, July.
    14. Renske E. Kuijpers & Ingmar Visser & Dylan Molenaar, 2021. "Testing the Within-State Distribution in Mixture Models for Responses and Response Times," Journal of Educational and Behavioral Statistics, , vol. 46(3), pages 348-373, June.
    15. Hans Jørn Juhl & Morten H. J. Fenger & John Thøgersen, 2017. "Will the Consistent Organic Food Consumer Step Forward? An Empirical Analysis," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 44(3), pages 519-535.
    16. Geir D. Berentsen & Jan Bulla & Antonello Maruotti & Bård Støve, 2022. "Modelling clusters of corporate defaults: Regime‐switching models significantly reduce the contagion source," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(3), pages 698-722, June.
    17. Giorgio E. Montanari & Marco Doretti, 2019. "Ranking Nursing Homes’ Performances Through a Latent Markov Model with Fixed and Random Effects," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 146(1), pages 307-326, November.
    18. F. Bartolucci & A. Farcomeni & F. Pennoni, 2014. "Latent Markov models: a review of a general framework for the analysis of longitudinal data with covariates," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 23(3), pages 433-465, September.
    19. Francesca Bassi & Fulvia Pennoni & Luca Rossetto, 2020. "The Italian market of sparkling wines: Latent variable models for brand positioning, customer loyalty, and transitions across brands' preferences," Agribusiness, John Wiley & Sons, Ltd., vol. 36(4), pages 542-567, October.
    20. Qi Chen & Wen Luo & Gregory J. Palardy & Ryan Glaman & Amber McEnturff, 2017. "The Efficacy of Common Fit Indices for Enumerating Classes in Growth Mixture Models When Nested Data Structure Is Ignored," SAGE Open, , vol. 7(1), pages 21582440177, March.
    21. Philippe Carette & Marie-Anne Guerry, 2022. "Markov models for duration-dependent transitions: selecting the states using duration values or duration intervals?," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 31(5), pages 1203-1223, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nicolas Depraetere & Martina Vandebroek, 2014. "Order selection in finite mixtures of linear regressions," Statistical Papers, Springer, vol. 55(3), pages 871-911, August.
    2. Francesco BARTOLUCCI & Silvia BACCI & Claudia PIGINI, 2015. "A Misspecification Test for Finite-Mixture Logistic Models for Clustered Binary and Ordered Responses," Working Papers 410, Universita' Politecnica delle Marche (I), Dipartimento di Scienze Economiche e Sociali.
    3. Omar N. Solinger & Woody van Olffen & Robert A. Roe & Joeri Hofmans, 2013. "On Becoming (Un)Committed: A Taxonomy and Test of Newcomer Onboarding Scenarios," Organization Science, INFORMS, vol. 24(6), pages 1640-1661, December.
    4. Francesco Bartolucci & Ivonne Solis-Trapala, 2010. "Multidimensional Latent Markov Models in a Developmental Study of Inhibitory Control and Attentional Flexibility in Early Childhood," Psychometrika, Springer;The Psychometric Society, vol. 75(4), pages 725-743, December.
    5. F. Bartolucci & A. Farcomeni & F. Pennoni, 2014. "Latent Markov models: a review of a general framework for the analysis of longitudinal data with covariates," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 23(3), pages 433-465, September.
    6. Francesco Bartolucci & Alessio Farcomeni, 2010. "A note on the mixture transition distribution and hidden Markov models," Journal of Time Series Analysis, Wiley Blackwell, vol. 31(2), pages 132-138, March.
    7. Ana Oliveira-Brochado & Francisco Vitorino Martins, 2008. "Determining the Number of Market Segments Using an Experimental Design," FEP Working Papers 263, Universidade do Porto, Faculdade de Economia do Porto.
    8. Kelava, Augustin & Kohler, Michael & Krzyżak, Adam & Schaffland, Tim Fabian, 2017. "Nonparametric estimation of a latent variable model," Journal of Multivariate Analysis, Elsevier, vol. 154(C), pages 112-134.
    9. Kim, Daeyoung & Kim, Jong-Min & Liao, Shu-Min & Jung, Yoon-Sung, 2013. "Mixture of D-vine copulas for modeling dependence," Computational Statistics & Data Analysis, Elsevier, vol. 64(C), pages 1-19.
    10. Ana Oliveira-Brochado & Francisco Vitorino Martins, 2014. "Identifying Small Market Segments with Mixture Regression Models," International Journal of Finance, Insurance and Risk Management, International Journal of Finance, Insurance and Risk Management, vol. 4(4), pages 812-812.
    11. Marianna Virtanen & Jussi Vahtera & Jenny Head & Rosemary Dray-Spira & Annaleena Okuloff & Adam G Tabak & Marcel Goldberg & Jenni Ervasti & Markus Jokela & Archana Singh-Manoux & Jaana Pentti & Marie , 2015. "Work Disability among Employees with Diabetes: Latent Class Analysis of Risk Factors in Three Prospective Cohort Studies," PLOS ONE, Public Library of Science, vol. 10(11), pages 1-14, November.
    12. Danks, Nicholas P. & Sharma, Pratyush N. & Sarstedt, Marko, 2020. "Model selection uncertainty and multimodel inference in partial least squares structural equation modeling (PLS-SEM)," Journal of Business Research, Elsevier, vol. 113(C), pages 13-24.
    13. Morgan, Grant B. & Hodge, Kari J. & Baggett, Aaron R., 2016. "Latent profile analysis with nonnormal mixtures: A Monte Carlo examination of model selection using fit indices," Computational Statistics & Data Analysis, Elsevier, vol. 93(C), pages 146-161.
    14. Bartolucci, Francesco & Farcomeni, Alessio, 2009. "A Multivariate Extension of the Dynamic Logit Model for Longitudinal Data Based on a Latent Markov Heterogeneity Structure," Journal of the American Statistical Association, American Statistical Association, vol. 104(486), pages 816-831.
    15. Carlos A. Medel & Sergio C. Salgado, 2013. "Does the Bic Estimate and Forecast Better than the Aic?," Revista de Analisis Economico – Economic Analysis Review, Universidad Alberto Hurtado/School of Economics and Business, vol. 28(1), pages 47-64, April.
    16. Zachary K. Collier & Haobai Zhang & Bridgette Johnson, 2021. "Finite Mixture Modeling for Program Evaluation: Resampling and Pre-processing Approaches," Evaluation Review, , vol. 45(6), pages 309-333, December.
    17. Kim, Daeyoung & Seo, Byungtae, 2014. "Assessment of the number of components in Gaussian mixture models in the presence of multiple local maximizers," Journal of Multivariate Analysis, Elsevier, vol. 125(C), pages 100-120.
    18. Hunt, Lynette A. & Basford, Kaye E., 2016. "Comparing classical criteria for selecting intra-class correlated features in Multimix," Computational Statistics & Data Analysis, Elsevier, vol. 103(C), pages 350-366.
    19. Catania, Leopoldo & Di Mari, Roberto, 2021. "Hierarchical Markov-switching models for multivariate integer-valued time-series," Journal of Econometrics, Elsevier, vol. 221(1), pages 118-137.
    20. De Angelis, L & Paas, L.J., 2009. "The dynamic analysis and prediction of stock markets through the latent Markov model," Serie Research Memoranda 0053, VU University Amsterdam, Faculty of Economics, Business Administration and Econometrics.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:advdac:v:8:y:2014:i:2:p:125-145. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.