IDEAS home Printed from https://ideas.repec.org/a/eee/jmvana/v63y1997i1p47-72.html
   My bibliography  Save this article

Classification of Binary Vectors by Stochastic Complexity

Author

Listed:
  • Gyllenberg, Mats
  • Koski, Timo
  • Verlaan, Martin

Abstract

Stochastic complexity is treated as a tool of classification, i.e., of inferring the number of classes, the class descriptions, and the class memberships for a given data set of binary vectors. The stochastic complexity is evaluated with respect to the family of statistical models defined by finite mixtures of multivariate Bernoulli distributions obtained by the principle of maximum entropy. It is shown that stochastic complexity is asymptotically related to the classification maximum likelihood estimate. The formulae for stochastic complexity have an interpretation as minimum code lengths for certain universal source codes for storing the binary data vectors and their assignments into the classes in a classification. There is also a decomposition of the classification uncertainty in a sum of an intraclass uncertainty, an interclass uncertainty, and a special parsimony term. It is shown that minimizing the stochastic complexity amounts to maximizing the information content of the classification. An algorithm of alternating minimization of stochastic complexity is given. We discuss the relation of the method to the AUTOCLASS system of Bayesian classification. The application of classification by stochastic complexity to an extensive data base of strains ofEnterobacteriaceaeis described.

Suggested Citation

  • Gyllenberg, Mats & Koski, Timo & Verlaan, Martin, 1997. "Classification of Binary Vectors by Stochastic Complexity," Journal of Multivariate Analysis, Elsevier, vol. 63(1), pages 47-72, October.
    • Handle: RePEc:eee:jmvana:v:63:y:1997:i:1:p:47-72
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0047-259X(97)91687-0
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Michael Windham, 1987. "Parameter modification for clustering criteria," Journal of Classification, Springer;The Classification Society, vol. 4(2), pages 191-214, September.
    2. Glenn Milligan & Martha Cooper, 1985. "An examination of procedures for determining the number of clusters in a data set," Psychometrika, Springer;The Psychometric Society, vol. 50(2), pages 159-179, June.
    3. Mats Gyllenberg & Timo Koski, 1996. "Numerical taxonomy and the principle of maximum entropy," Journal of Classification, Springer;The Classification Society, vol. 13(2), pages 213-229, September.
    4. Gilles Celeux & Gérard Govaert, 1991. "Clustering criteria for discrete data and latent class models," Journal of Classification, Springer;The Classification Society, vol. 8(2), pages 157-176, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jukka Corander & Mats Gyllenberg & Timo Koski, 2009. "Bayesian unsupervised classification framework based on stochastic partitions of data and a parallel search strategy," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 3(1), pages 3-24, June.
    2. de Oliveira, Viviane M. & Gomes, M.A.F. & Tsang, I.R., 2006. "Theoretical model for the evolution of the linguistic diversity," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 361(1), pages 361-370.
    3. Bouguila, Nizar, 2010. "On multivariate binary data clustering and feature weighting," Computational Statistics & Data Analysis, Elsevier, vol. 54(1), pages 120-134, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bock, Hans H., 1996. "Probabilistic models in cluster analysis," Computational Statistics & Data Analysis, Elsevier, vol. 23(1), pages 5-28, November.
    2. Bouguila, Nizar, 2010. "On multivariate binary data clustering and feature weighting," Computational Statistics & Data Analysis, Elsevier, vol. 54(1), pages 120-134, January.
    3. Liu, Pei-chen Barry & Hansen, Mark & Mukherjee, Avijit, 2008. "Scenario-based air traffic flow management: From theory to practice," Transportation Research Part B: Methodological, Elsevier, vol. 42(7-8), pages 685-702, August.
    4. Li, Pai-Ling & Chiou, Jeng-Min, 2011. "Identifying cluster number for subspace projected functional data clustering," Computational Statistics & Data Analysis, Elsevier, vol. 55(6), pages 2090-2103, June.
    5. Alessandra Cepparulo & Antonello Zanfei, 2019. "The diffusion of public eServices in European cities," Working Papers 1904, University of Urbino Carlo Bo, Department of Economics, Society & Politics - Scientific Committee - L. Stefanini & G. Travaglini, revised 2019.
    6. Noelia Caceres & Luis M. Romero & Francisco J. Morales & Antonio Reyes & Francisco G. Benitez, 2018. "Estimating traffic volumes on intercity road locations using roadway attributes, socioeconomic features and other work-related activity characteristics," Transportation, Springer, vol. 45(5), pages 1449-1473, September.
    7. Michele Cincera, 2005. "Firms' productivity growth and R&D spillovers: An analysis of alternative technological proximity measures," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 14(8), pages 657-682.
    8. Douglas L. Steinley & M. J. Brusco, 2019. "Using an Iterative Reallocation Partitioning Algorithm to Verify Test Multidimensionality," Journal of Classification, Springer;The Classification Society, vol. 36(3), pages 397-413, October.
    9. Javier Sevil-Serrano & Alberto Aibar-Solana & Ángel Abós & José Antonio Julián & Luis García-González, 2019. "Healthy or Unhealthy? The Cocktail of Health-Related Behavior Profiles in Spanish Adolescents," IJERPH, MDPI, vol. 16(17), pages 1-14, August.
    10. Jacques-Antoine Gauthier & Eric D. Widmer & Philipp Bucher & Cédric Notredame, 2009. "How Much Does It Cost?," Sociological Methods & Research, , vol. 38(1), pages 197-231, August.
    11. Jack DeWaard & Keuntae Kim & James Raymer, 2012. "Migration Systems in Europe: Evidence From Harmonized Flow Data," Demography, Springer;Population Association of America (PAA), vol. 49(4), pages 1307-1333, November.
    12. Vicente Rodríguez Montequín & Joaquín Villanueva Balsera & Sonia María Cousillas Fernández & Francisco Ortega Fernández, 2018. "Exploring Project Complexity through Project Failure Factors: Analysis of Cluster Patterns Using Self-Organizing Maps," Complexity, Hindawi, vol. 2018, pages 1-17, May.
    13. Goethner, Maximilian & Hornuf, Lars & Regner, Tobias, 2021. "Protecting investors in equity crowdfunding: An empirical analysis of the small investor protection act," Technological Forecasting and Social Change, Elsevier, vol. 162(C).
    14. Maria Lo Bue & Stephan Klasen, 2013. "Identifying Synergies and Complementarities Between MDGs: Results from Cluster Analysis," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 113(2), pages 647-670, September.
    15. Valérie Barraud-Didier & Marie-Christine Henninger & Pierre Triboulet, 2014. "La Participation des Adhérents Dans Leurs Coopératives Agricoles: Une Étude Exploratoire du Secteur Céréalier Français," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 62(1), pages 125-148, March.
    16. Latruffe, Laure & Dupuy, Aurelia & Desjeux, Yann, 2012. "What would farmers’ strategies be in a no-CAP situation? An illustration to France," 86th Annual Conference, April 16-18, 2012, Warwick University, Coventry, UK 134989, Agricultural Economics Society.
    17. J. Fernando Vera & Rodrigo Macías, 2021. "On the Behaviour of K-Means Clustering of a Dissimilarity Matrix by Means of Full Multidimensional Scaling," Psychometrika, Springer;The Psychometric Society, vol. 86(2), pages 489-513, June.
    18. Alessio Ishizaka & Philippe Nemery, 2013. "A Multi-Criteria Group Decision Framework for Partner Grouping When Sharing Facilities," Group Decision and Negotiation, Springer, vol. 22(4), pages 773-799, July.
    19. Peter Bryant, 1991. "Large-sample results for optimization-based clustering methods," Journal of Classification, Springer;The Classification Society, vol. 8(1), pages 31-44, January.
    20. Victor Kaufman & Anthony Rodriguez & Lisa C. Walsh & Edward Shafranske & Shelly P. Harrell, 2022. "Unique Ways in Which the Quality of Friendships Matter for Life Satisfaction," Journal of Happiness Studies, Springer, vol. 23(6), pages 2563-2580, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jmvana:v:63:y:1997:i:1:p:47-72. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.