IDEAS home Printed from https://ideas.repec.org/a/spr/advdac/v12y2018i2d10.1007_s11634-016-0278-2.html
   My bibliography  Save this article

Probabilistic clustering via Pareto solutions and significance tests

Author

Listed:
  • María Teresa Gallegos

    (University of Passau)

  • Gunter Ritter

    (University of Passau)

Abstract

The present paper proposes a new strategy for probabilistic (often called model-based) clustering. It is well known that local maxima of mixture likelihoods can be used to partition an underlying data set. However, local maxima are rarely unique. Therefore, it remains to select the reasonable solutions, and in particular the desired one. Credible partitions are usually recognized by separation (and cohesion) of their clusters. We use here the p values provided by the classical tests of Wilks, Hotelling, and Behrens–Fisher to single out those solutions that are well separated by location. It has been shown that reasonable solutions to a clustering problem are related to Pareto points in a plot of scale balance vs. model fit of all local maxima. We briefly review this theory and propose as solutions all well-fitting Pareto points in the set of local maxima separated by location in the above sense. We also design a new iterative, parameter-free cutting plane algorithm for the multivariate Behrens–Fisher problem.

Suggested Citation

  • María Teresa Gallegos & Gunter Ritter, 2018. "Probabilistic clustering via Pareto solutions and significance tests," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 12(2), pages 179-202, June.
    • Handle: RePEc:spr:advdac:v:12:y:2018:i:2:d:10.1007_s11634-016-0278-2
    DOI: 10.1007/s11634-016-0278-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11634-016-0278-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11634-016-0278-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kiefer, Nicholas M, 1978. "Discrete Parameter Variation: Efficient Estimation of a Switching Regression Model," Econometrica, Econometric Society, vol. 46(2), pages 427-434, March.
    2. María Gallegos & Gunter Ritter, 2009. "Trimming algorithms for clustering contaminated grouped data and their robustness," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 3(2), pages 135-167, September.
    3. Gallegos, María Teresa & Ritter, Gunter, 2013. "Strong consistency of k-parameters clustering," Journal of Multivariate Analysis, Elsevier, vol. 117(C), pages 14-31.
    4. Fritz, Heinrich & García-Escudero, Luis A. & Mayo-Iscar, Agustín, 2013. "A fast algorithm for robust constrained clustering," Computational Statistics & Data Analysis, Elsevier, vol. 61(C), pages 124-136.
    5. Chris Fraley & Adrian E. Raftery, 1999. "MCLUST: Software for Model-Based Cluster Analysis," Journal of Classification, Springer;The Classification Society, vol. 16(2), pages 297-306, July.
    6. H. Bock, 1985. "On some significance tests in cluster analysis," Journal of Classification, Springer;The Classification Society, vol. 2(1), pages 77-108, December.
    7. Bock H.H., 1985. "On some significance tests in cluster analysis," Journal of Classification, Springer;The Classification Society, vol. 2(1), pages 300-300, December.
    8. Gallegos, María Teresa & Ritter, Gunter, 2010. "Using combinatorial optimization in model-based trimmed clustering with cardinality constraints," Computational Statistics & Data Analysis, Elsevier, vol. 54(3), pages 637-654, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gallegos, María Teresa & Ritter, Gunter, 2013. "Strong consistency of k-parameters clustering," Journal of Multivariate Analysis, Elsevier, vol. 117(C), pages 14-31.
    2. Gallegos, María Teresa & Ritter, Gunter, 2010. "Using combinatorial optimization in model-based trimmed clustering with cardinality constraints," Computational Statistics & Data Analysis, Elsevier, vol. 54(3), pages 637-654, March.
    3. Roberto Rocci & Stefano Antonio Gattone & Roberto Di Mari, 2018. "A data driven equivariant approach to constrained Gaussian mixture modeling," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 12(2), pages 235-260, June.
    4. María Gallegos & Gunter Ritter, 2009. "Trimming algorithms for clustering contaminated grouped data and their robustness," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 3(2), pages 135-167, September.
    5. Fritz, Heinrich & García-Escudero, Luis A. & Mayo-Iscar, Agustín, 2012. "tclust: An R Package for a Trimming Approach to Cluster Analysis," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 47(i12).
    6. Pacáková, Z. & Poláčková, J., 2013. "Hierarchical Cluster Analysis – Various Approaches to Data Preparation," AGRIS on-line Papers in Economics and Informatics, Czech University of Life Sciences Prague, Faculty of Economics and Management, vol. 5(3), pages 1-11, September.
    7. Luis Angel García-Escudero & Alfonso Gordaliza & Francesca Greselin & Salvatore Ingrassia & Agustín Mayo-Iscar, 2018. "Eigenvalues and constraints in mixture modeling: geometric and computational issues," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 12(2), pages 203-233, June.
    8. Fritz, Heinrich & García-Escudero, Luis A. & Mayo-Iscar, Agustín, 2013. "A fast algorithm for robust constrained clustering," Computational Statistics & Data Analysis, Elsevier, vol. 61(C), pages 124-136.
    9. Tran, Minh Chau & Gan, Christopher & Hu, Baiding, 2014. "Credit Constraints and Impact on Farm Household Welfare: Evidence from Vietnam’s North Central Coast region," 2014 Conference, August 28-29, 2014, Nelson, New Zealand 187495, New Zealand Agricultural and Resource Economics Society.
    10. Georgiev, Iliyan, 2010. "Model-based asymptotic inference on the effect of infrequent large shocks on cointegrated variables," Journal of Econometrics, Elsevier, vol. 158(1), pages 37-50, September.
    11. Adrian O’Hagan & Arthur White, 2019. "Improved model-based clustering performance using Bayesian initialization averaging," Computational Statistics, Springer, vol. 34(1), pages 201-231, March.
    12. Li-Xuan Qin & Steven G. Self, 2006. "The Clustering of Regression Models Method with Applications in Gene Expression Data," Biometrics, The International Biometric Society, vol. 62(2), pages 526-533, June.
    13. Yang, Qing-Qing & Ching, Wai-Ki & Gu, Jia-Wen & Siu, Tak-Kuen, 2018. "Market-making strategy with asymmetric information and regime-switching," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 408-433.
    14. Houda Rharrabti Zaid, 2015. "Transmission du stress financier de la zone euro aux Pays de l’Europe Centrale et Orientale," EconomiX Working Papers 2015-37, University of Paris Nanterre, EconomiX.
    15. Maddalena Cavicchioli, 2016. "Statistical Analysis Of Mixture Vector Autoregressive Models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(4), pages 1192-1213, December.
    16. Yao, Weixin & Wei, Yan & Yu, Chun, 2014. "Robust mixture regression using the t-distribution," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 116-127.
    17. L. García-Escudero & A. Gordaliza & A. Mayo-Iscar, 2013. "Comments on: model-based clustering and classification with non-normal mixture distributions," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 22(4), pages 459-461, November.
    18. Ugo Fratesi & Giovanni Perucca, 2018. "Territorial capital and the resilience of European regions," The Annals of Regional Science, Springer;Western Regional Science Association, vol. 60(2), pages 241-264, March.
    19. Morduch, Jonathan J. & Stern, Hal S., 1997. "Using mixture models to detect sex bias in health outcomes in Bangladesh," Journal of Econometrics, Elsevier, vol. 77(1), pages 259-276, March.
    20. Kajal Lahiri & Chuanming Gao & Bernard Wixon, 2020. "Value of Sample Separation Information in a Sequential Probit Model," Arthaniti: Journal of Economic Theory and Practice, , vol. 19(2), pages 151-176, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:advdac:v:12:y:2018:i:2:d:10.1007_s11634-016-0278-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.