IDEAS home Printed from https://ideas.repec.org/a/bla/scjsta/v51y2024i2p485-512.html
   My bibliography  Save this article

Covariance‐based soft clustering of functional data based on the Wasserstein–Procrustes metric

Author

Listed:
  • Valentina Masarotto
  • Guido Masarotto

Abstract

We consider the problem of clustering functional data according to their covariance structure. We contribute a soft clustering methodology based on the Wasserstein–Procrustes distance, where the in‐between cluster variability is penalized by a term proportional to the entropy of the partition matrix. In this way, each covariance operator can be partially classified into more than one group. Such soft classification allows for clusters to overlap, and arises naturally in situations where the separation between all or some of the clusters is not well‐defined. We also discuss how to estimate the number of groups and to test for the presence of any cluster structure. The algorithm is illustrated using simulated and real data. An R implementation is available in the Appendix S1.

Suggested Citation

  • Valentina Masarotto & Guido Masarotto, 2024. "Covariance‐based soft clustering of functional data based on the Wasserstein–Procrustes metric," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 51(2), pages 485-512, June.
    • Handle: RePEc:bla:scjsta:v:51:y:2024:i:2:p:485-512
    DOI: 10.1111/sjos.12692
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/sjos.12692
    Download Restriction: no
    File URL: https://libkey.io/10.1111/sjos.12692?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Aneiros, Germán & Horová, Ivana & Hušková, Marie & Vieu, Philippe, 2022. "On functional data analysis and related topics," Journal of Multivariate Analysis, Elsevier, vol. 189(C).
    2. Valentina Masarotto & Victor M. Panaretos & Yoav Zemel, 2019. "Procrustes Metrics on Covariance Operators and Optimal Transportation of Gaussian Processes," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 81(1), pages 172-213, February.
    3. Davide Pigoli & John A. D. Aston & Ian L. Dryden & Piercesare Secchi, 2014. "Distances and inference for covariance operators," Biometrika, Biometrika Trust, vol. 101(2), pages 409-422.
    4. Stefan Fremdt & Josef G. Steinebach & Lajos Horváth & Piotr Kokoszka, 2013. "Testing the Equality of Covariance Operators in Functional Samples," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 40(1), pages 138-152, March.
    5. Shahin Tavakoli & Victor M. Panaretos, 2016. "Detecting and Localizing Differences in Functional Time Series Dynamics: A Case Study in Molecular Biophysics," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(515), pages 1020-1035, July.
    6. Davide Pigoli & Pantelis Z. Hadjipantelis & John S. Coleman & John A. D. Aston, 2018. "The statistical analysis of acoustic phonetic data: exploring differences between spoken Romance languages," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 67(5), pages 1103-1145, November.
    7. Batool, Fatima & Hennig, Christian, 2021. "Clustering with the Average Silhouette Width," Computational Statistics & Data Analysis, Elsevier, vol. 158(C).
    8. David Kraus & Victor M. Panaretos, 2012. "Dispersion operators and resistant second-order functional data analysis," Biometrika, Biometrika Trust, vol. 99(4), pages 813-832.
    9. Jia Guo & Bu Zhou & Jin-Ting Zhang, 2019. "New Tests for Equality of Several Covariance Functions for Functional Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(527), pages 1251-1263, July.
    10. Panaretos, Victor M. & Kraus, David & Maddocks, John H., 2010. "Second-Order Comparison of Gaussian Random Functions and the Geometry of DNA Minicircles," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 670-682.
    11. Graciela Boente & Daniela Rodriguez & Mariela Sued, 2018. "Testing equality between several populations covariance operators," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 70(4), pages 919-950, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Boente, Graciela & Rodriguez, Daniela & Sued, Mariela, 2019. "The spatial sign covariance operator: Asymptotic results and applications," Journal of Multivariate Analysis, Elsevier, vol. 170(C), pages 115-128.
    2. Qiu, Zhiping & Fan, Jiangyuan & Zhang, Jin-Ting & Chen, Jianwei, 2024. "Tests for equality of several covariance matrix functions for multivariate functional data," Journal of Multivariate Analysis, Elsevier, vol. 199(C).
    3. Valentina Masarotto & Victor M. Panaretos & Yoav Zemel, 2019. "Procrustes Metrics on Covariance Operators and Optimal Transportation of Gaussian Processes," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 81(1), pages 172-213, February.
    4. Holger Dette & Kevin Kokot, 2022. "Detecting relevant differences in the covariance operators of functional time series: a sup-norm approach," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 74(2), pages 195-231, April.
    5. Kraus, David, 2019. "Inferential procedures for partially observed functional data," Journal of Multivariate Analysis, Elsevier, vol. 173(C), pages 583-603.
    6. Hà Quang Minh, 2023. "Entropic Regularization of Wasserstein Distance Between Infinite-Dimensional Gaussian Measures and Gaussian Processes," Journal of Theoretical Probability, Springer, vol. 36(1), pages 201-296, March.
    7. Ghiglietti, Andrea & Paganoni, Anna Maria, 2017. "Exact tests for the means of Gaussian stochastic processes," Statistics & Probability Letters, Elsevier, vol. 131(C), pages 102-107.
    8. Holger Dette & Kevin Kokot & Stanislav Volgushev, 2020. "Testing relevant hypotheses in functional time series via self‐normalization," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(3), pages 629-660, July.
    9. Tomasz Górecki & Lajos Horváth & Piotr Kokoszka, 2020. "Tests of Normality of Functional Data," International Statistical Review, International Statistical Institute, vol. 88(3), pages 677-697, December.
    10. Jiang, Qing & Hušková, Marie & Meintanis, Simos G. & Zhu, Lixing, 2019. "Asymptotics, finite-sample comparisons and applications for two-sample tests with functional data," Journal of Multivariate Analysis, Elsevier, vol. 170(C), pages 202-220.
    11. Jia Guo & Bu Zhou & Jianwei Chen & Jin-Ting Zhang, 2019. "An $${{\varvec{L}}}^{2}$$L2-norm-based test for equality of several covariance functions: a further study," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 28(4), pages 1092-1112, December.
    12. Kokoszka Piotr & Miao Hong & Zheng Ben, 2017. "Testing for asymmetry in betas of cumulative returns: Impact of the financial crisis and crude oil price," Statistics & Risk Modeling, De Gruyter, vol. 34(1-2), pages 33-53, June.
    13. Dimitrios Pilavakis & Efstathios Paparoditis & Theofanis Sapatinas, 2020. "Testing equality of autocovariance operators for functional time series," Journal of Time Series Analysis, Wiley Blackwell, vol. 41(4), pages 571-589, July.
    14. Adam B. Kashlak & John A. D. Aston & Richard Nickl, 2019. "Inference on Covariance Operators via Concentration Inequalities: k-sample Tests, Classification, and Clustering via Rademacher Complexities," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 81(1), pages 214-243, February.
    15. Leucht, Anne & Paparoditis, Efstathios & Rademacher, Daniel & Sapatinas, Theofanis, 2022. "Testing equality of spectral density operators for functional processes," Journal of Multivariate Analysis, Elsevier, vol. 189(C).
    16. González–Rodríguez, Gil & Colubi, Ana & González–Manteiga, Wenceslao & Febrero–Bande, Manuel, 2024. "A consistent test of equality of distributions for Hilbert-valued random elements," Journal of Multivariate Analysis, Elsevier, vol. 202(C).
    17. Ahmad, Rauf, 2022. "Tests for proportionality of matrices with large dimension," Journal of Multivariate Analysis, Elsevier, vol. 189(C).
    18. Tao Zhang & Zhiwen Wang & Yanling Wan, 2021. "Functional test for high-dimensional covariance matrix, with application to mitochondrial calcium concentration," Statistical Papers, Springer, vol. 62(3), pages 1213-1230, June.
    19. Horváth, Lajos & Hušková, Marie & Rice, Gregory, 2013. "Test of independence for functional data," Journal of Multivariate Analysis, Elsevier, vol. 117(C), pages 100-119.
    20. Graciela Boente & Daniela Rodriguez & Mariela Sued, 2018. "Testing equality between several populations covariance operators," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 70(4), pages 919-950, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:scjsta:v:51:y:2024:i:2:p:485-512. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0303-6898 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.