IDEAS home Printed from https://ideas.repec.org/a/spr/compst/v38y2023i1d10.1007_s00180-022-01225-4.html
   My bibliography  Save this article

Speeding up the convergence of the alternating least squares algorithm using vector $$\varepsilon $$ ε acceleration and restarting for nonlinear principal component analysis

Author

Listed:
  • Masahiro Kuroda

    (Okayama University of Science)

  • Yuichi Mori

    (Okayama University of Science)

  • Masaya Iizuka

    (Okayama University)

Abstract

Principal component analysis (PCA) is a widely used descriptive multivariate technique in the analysis of quantitative data. When applying PCA to mixed quantitative and qualitative data, we utilize an optimal scaling technique for quantifying qualitative data. PCA with optimal scaling is called nonlinear PCA. The alternating least squares (ALS) algorithm is used for computing nonlinear PCA. The ALS algorithm is stable in convergence and simple in implementation; however, the algorithm tends to converge slowly for large data matrices owing to its linear convergence. Then the v $$\varepsilon $$ ε -ALS algorithm, which incorporates the vector $$\varepsilon $$ ε accelerator into the ALS algorithm, is used to accelerate the convergence of the ALS algorithm for nonlinear PCA. In this paper, we improve the v $$\varepsilon $$ ε -ALS algorithm via a restarting procedure and further reduce its number of iterations and computation time. The restarting procedure is performed under simple restarting conditions, and it speeds up the convergence of the v $$\varepsilon $$ ε -ALS algorithm. The v $$\varepsilon $$ ε -ALS algorithm with a restarting procedure is referred to as the v $$\varepsilon $$ ε R-ALS algorithm. Numerical experiments examine the performance of the v $$\varepsilon $$ ε R-ALS algorithm by comparing its number of iterations and computation time with those of the ALS and v $$\varepsilon $$ ε -ALS algorithms.

Suggested Citation

  • Masahiro Kuroda & Yuichi Mori & Masaya Iizuka, 2023. "Speeding up the convergence of the alternating least squares algorithm using vector $$\varepsilon $$ ε acceleration and restarting for nonlinear principal component analysis," Computational Statistics, Springer, vol. 38(1), pages 243-262, March.
  • Handle: RePEc:spr:compst:v:38:y:2023:i:1:d:10.1007_s00180-022-01225-4
    DOI: 10.1007/s00180-022-01225-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00180-022-01225-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s00180-022-01225-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Forrest Young & Yoshio Takane & Jan Leeuw, 1978. "The principal components of mixed measurement level multivariate data: An alternating least squares method with optimal scaling features," Psychometrika, Springer;The Psychometric Society, vol. 43(2), pages 279-281, June.
    2. Kuroda, Masahiro & Mori, Yuichi & Iizuka, Masaya & Sakakihara, Michio, 2011. "Acceleration of the alternating least squares algorithm for principal components analysis," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 143-153, January.
    3. Yoshio Takane & Kwanghee Jung & Heungsun Hwang, 2010. "An acceleration method for Ten Berge et al.’s algorithm for orthogonal INDSCAL," Computational Statistics, Springer, vol. 25(3), pages 409-428, September.
    4. Sébastien Loisel & Yoshio Takane, 2011. "Generalized GIPSCAL re-revisited: a fast convergent algorithm with acceleration by the minimal polynomial extrapolation," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 5(1), pages 57-75, April.
    5. J. Kruskal, 1964. "Nonmetric multidimensional scaling: A numerical method," Psychometrika, Springer;The Psychometric Society, vol. 29(2), pages 115-129, June.
    6. Krijnen, Wim P., 2006. "Convergence of the sequence of parameters generated by alternating least squares algorithms," Computational Statistics & Data Analysis, Elsevier, vol. 51(2), pages 481-489, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hansohm, Jürgen, 2007. "Algorithms and error estimations for monotone regression on partially preordered sets," Journal of Multivariate Analysis, Elsevier, vol. 98(5), pages 1043-1050, May.
    2. Kuroda, Masahiro & Mori, Yuichi & Iizuka, Masaya & Sakakihara, Michio, 2011. "Acceleration of the alternating least squares algorithm for principal components analysis," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 143-153, January.
    3. Katherine Morales & Miguel Flores & Yasmín Salazar Méndez, 2021. "Analysis of Principal Nonlinear Components for the Construction of a Socioeconomic Stratification Index in Ecuador," Revista Desarrollo y Sociedad, Universidad de los Andes,Facultad de Economía, CEDE, vol. 88(2), pages 43-82, July.
    4. Takane, Yoshio, 2016. "My Early Interactions with Jan and Some of His Lost Papers," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 73(i07).
    5. Michel Tenenhaus, 1988. "Canonical analysis of two convex polyhedral cones and applications," Psychometrika, Springer;The Psychometric Society, vol. 53(4), pages 503-524, December.
    6. Jacqueline Meulman, 1992. "The integration of multidimensional scaling and multivariate analysis with optimal transformations," Psychometrika, Springer;The Psychometric Society, vol. 57(4), pages 539-565, December.
    7. Beniaich, Adnane & Guimarães, Danielle Vieira & Avanzi, Junior Cesar & Silva, Bruno Montoani & Acuña-Guzman, Salvador Francisco & dos Santos, Wharley Pereira & Silva, Marx Leandro Naves, 2023. "Spontaneous vegetation as an alternative to cover crops in olive orchards reduces water erosion and improves soil physical properties under tropical conditions," Agricultural Water Management, Elsevier, vol. 279(C).
    8. Giuseppe Arbia & Giovanni Lafratta, 2002. "Anisotropic spatial sampling designs for urban pollution," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 51(2), pages 223-234, May.
    9. Samuel Shye, 2010. "The Motivation to Volunteer: A Systemic Quality of Life Theory," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 98(2), pages 183-200, September.
    10. Muñoz-Mas, Rafael & Vezza, Paolo & Alcaraz-Hernández, Juan Diego & Martínez-Capel, Francisco, 2016. "Risk of invasion predicted with support vector machines: A case study on northern pike (Esox Lucius, L.) and bleak (Alburnus alburnus, L.)," Ecological Modelling, Elsevier, vol. 342(C), pages 123-134.
    11. Markovsky, Ivan & Niranjan, Mahesan, 2010. "Approximate low-rank factorization with structured factors," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3411-3420, December.
    12. la Grange, Anthony & le Roux, Niël & Gardner-Lubbe, Sugnet, 2009. "BiplotGUI: Interactive Biplots in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 30(i12).
    13. Simensen, Trond & Halvorsen, Rune & Erikstad, Lars, 2018. "Methods for landscape characterisation and mapping: A systematic review," Land Use Policy, Elsevier, vol. 75(C), pages 557-569.
    14. Silvia Vilčeková & Ilija Zoran Apostoloski & Ľudmila Mečiarová & Eva Krídlová Burdová & Jozef Kiseľák, 2017. "Investigation of Indoor Air Quality in Houses of Macedonia," IJERPH, MDPI, vol. 14(1), pages 1-12, January.
    15. Funk, Patrick & Davis, Alex & Vaishnav, Parth & Dewitt, Barry & Fuchs, Erica, 2020. "Individual inconsistency and aggregate rationality: Overcoming inconsistencies in expert judgment at the technical frontier," Technological Forecasting and Social Change, Elsevier, vol. 155(C).
    16. Moris Triventi, 2014. "Higher education regimes: an empirical classification of higher education systems and its relationship with student accessibility," Quality & Quantity: International Journal of Methodology, Springer, vol. 48(3), pages 1685-1703, May.
    17. Jessica Dafflon & Pedro F. Da Costa & František Váša & Ricardo Pio Monti & Danilo Bzdok & Peter J. Hellyer & Federico Turkheimer & Jonathan Smallwood & Emily Jones & Robert Leech, 2022. "A guided multiverse study of neuroimaging analyses," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    18. Karim Abou-Moustafa & Frank P. Ferrie, 2018. "Local generalized quadratic distance metrics: application to the k-nearest neighbors classifier," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 12(2), pages 341-363, June.
    19. Camacho, Maximo & Perez-Quiros, Gabriel & Saiz, Lorena, 2006. "Are European business cycles close enough to be just one?," Journal of Economic Dynamics and Control, Elsevier, vol. 30(9-10), pages 1687-1706.
    20. Sébastien Loisel & Yoshio Takane, 2011. "Generalized GIPSCAL re-revisited: a fast convergent algorithm with acceleration by the minimal polynomial extrapolation," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 5(1), pages 57-75, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:compst:v:38:y:2023:i:1:d:10.1007_s00180-022-01225-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.