IDEAS home Printed from https://ideas.repec.org/a/eee/apmaco/v451y2023ics0096300323001789.html
   My bibliography  Save this article

A variable projection method for the general radial basis function neural network

Author

Listed:
  • Zheng, Sanpeng
  • Feng, Renzhong

Abstract

The variable projection (VP) method is a classical and effective method for the separable nonlinear least squares (SNLLS) problem. Training a radial basis function neural network (RBFNN) with only one output neuron by minimizing the sum of the squared errors (SSE) is an SNLLS problem, so that the classical VP method has been applied to RBFNN. However, the one-output-RBFNN (ORBFNN) is just one type of RBFNN, so that the paper proposes a new VP method for the general radial basis function neural network (GRBFNN) which has no limit of the number of the output neurons. The new VP method translates the problem corresponding to minimizing the SSE of GRBFNN into a lower-dimensional optimization problem. We prove theoretically that the set of stationary points of the objective function of the lower-dimensional problem is equivalent to that of the original objective function. In addition, the lower dimension leads to less guesses about the initial point for the new problem. The numerical experiments indicate that, with the same algorithm, minimizing the new objective function converges in fewer iterations and makes both a smaller training error and a testing error than minimizing the original objective function.

Suggested Citation

  • Zheng, Sanpeng & Feng, Renzhong, 2023. "A variable projection method for the general radial basis function neural network," Applied Mathematics and Computation, Elsevier, vol. 451(C).
    • Handle: RePEc:eee:apmaco:v:451:y:2023:i:c:s0096300323001789
    DOI: 10.1016/j.amc.2023.128009
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0096300323001789
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.amc.2023.128009?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dianne O’Leary & Bert Rust, 2013. "Variable projection for nonlinear least squares problems," Computational Optimization and Applications, Springer, vol. 54(3), pages 579-593, April.
    2. Sanpeng Zheng & Renzhong Feng & Aitong Huang, 2020. "The Optimal Shape Parameter for the Least Squares Approximation Based on the Radial Basis Function," Mathematics, MDPI, vol. 8(11), pages 1-20, November.
    3. R. Cavoretto & A. Rossi & M. S. Mukhametzhanov & Ya. D. Sergeyev, 2021. "On the search of the shape parameter in radial basis functions using univariate global optimization methods," Journal of Global Optimization, Springer, vol. 79(2), pages 305-327, February.
    4. Wenyu Sun & Ya-Xiang Yuan, 2006. "Optimization Theory and Methods," Springer Optimization and Its Applications, Springer, number 978-0-387-24976-6, June.
    5. Min Gan & C.L. Philip Chen & Long Chen & Chun-Yang Zhang, 2016. "Exploiting the interpretability and forecasting ability of the RBF-AR model for nonlinear time series," International Journal of Systems Science, Taylor & Francis Journals, vol. 47(8), pages 1868-1876, June.
    6. Han, Yongming & Fan, Chenyu & Geng, Zhiqiang & Ma, Bo & Cong, Di & Chen, Kai & Yu, Bin, 2020. "Energy efficient building envelope using novel RBF neural network integrated affinity propagation," Energy, Elsevier, vol. 209(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Li, Yang & Liu, Dejun & Yin, Zhexu & Chen, Yun & Meng, Jin, 2023. "Adaptive selection strategy of shape parameters for LRBF for solving partial differential equations," Applied Mathematics and Computation, Elsevier, vol. 440(C).
    2. Yasushi Narushima & Shummin Nakayama & Masashi Takemura & Hiroshi Yabe, 2023. "Memoryless Quasi-Newton Methods Based on the Spectral-Scaling Broyden Family for Riemannian Optimization," Journal of Optimization Theory and Applications, Springer, vol. 197(2), pages 639-664, May.
    3. Saha, Tanay & Rakshit, Suman & Khare, Swanand R., 2023. "Linearly structured quadratic model updating using partial incomplete eigendata," Applied Mathematics and Computation, Elsevier, vol. 446(C).
    4. Kovács, Péter & Fekete, Andrea M., 2019. "Nonlinear least-squares spline fitting with variable knots," Applied Mathematics and Computation, Elsevier, vol. 354(C), pages 490-501.
    5. Hai-Jun Wang & Qin Ni, 2010. "A Convex Approximation Method For Large Scale Linear Inequality Constrained Minimization," Asia-Pacific Journal of Operational Research (APJOR), World Scientific Publishing Co. Pte. Ltd., vol. 27(01), pages 85-101.
    6. Chen, Liang, 2016. "A high-order modified Levenberg–Marquardt method for systems of nonlinear equations with fourth-order convergence," Applied Mathematics and Computation, Elsevier, vol. 285(C), pages 79-93.
    7. Ji, Li-Qun, 2015. "An assessment of agricultural residue resources for liquid biofuel production in China," Renewable and Sustainable Energy Reviews, Elsevier, vol. 44(C), pages 561-575.
    8. Babaie-Kafaki, Saman & Ghanbari, Reza, 2014. "The Dai–Liao nonlinear conjugate gradient method with optimal parameter choices," European Journal of Operational Research, Elsevier, vol. 234(3), pages 625-630.
    9. Yutao Zheng & Bing Zheng, 2017. "Two New Dai–Liao-Type Conjugate Gradient Methods for Unconstrained Optimization Problems," Journal of Optimization Theory and Applications, Springer, vol. 175(2), pages 502-509, November.
    10. Natei Ermias Benti & Mesfin Diro Chaka & Addisu Gezahegn Semie, 2023. "Forecasting Renewable Energy Generation with Machine Learning and Deep Learning: Current Advances and Future Prospects," Sustainability, MDPI, vol. 15(9), pages 1-33, April.
    11. Xiaojing Zhu & Hiroyuki Sato, 2020. "Riemannian conjugate gradient methods with inverse retraction," Computational Optimization and Applications, Springer, vol. 77(3), pages 779-810, December.
    12. Li, Jinqing & Ma, Jun, 2019. "Maximum penalized likelihood estimation of additive hazards models with partly interval censoring," Computational Statistics & Data Analysis, Elsevier, vol. 137(C), pages 170-180.
    13. Chen, Wang & Yang, Xinmin & Zhao, Yong, 2023. "Memory gradient method for multiobjective optimization," Applied Mathematics and Computation, Elsevier, vol. 443(C).
    14. Abolfazl Gharaei & Alireza Amjadian & Ali Shavandi & Amir Amjadian, 2023. "An augmented Lagrangian approach with general constraints to solve nonlinear models of the large-scale reliable inventory systems," Journal of Combinatorial Optimization, Springer, vol. 45(2), pages 1-37, March.
    15. Roozbeh, Mahdi, 2016. "Robust ridge estimator in restricted semiparametric regression models," Journal of Multivariate Analysis, Elsevier, vol. 147(C), pages 127-144.
    16. F. Aragón Artacho & A. Belyakov & A. Dontchev & M. López, 2014. "Local convergence of quasi-Newton methods under metric regularity," Computational Optimization and Applications, Springer, vol. 58(1), pages 225-247, May.
    17. Chen, Chuin-Shan & Noorizadegan, Amir & Young, D.L. & Chen, C.S., 2023. "On the selection of a better radial basis function and its shape parameter in interpolation problems," Applied Mathematics and Computation, Elsevier, vol. 442(C).
    18. Hamid Esmaeili & Morteza Kimiaei, 2016. "A trust-region method with improved adaptive radius for systems of nonlinear equations," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 83(1), pages 109-125, February.
    19. Maximilian S. Ernst & Aaron Peikert & Andreas M. Brandmaier & Yves Rosseel, 2023. "A Note on the Connection Between Trek Rules and Separable Nonlinear Least Squares in Linear Structural Equation Models," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 98-116, March.
    20. Lijuan Zhao & Wenyu Sun, 2013. "A Conic Affine Scaling Dogleg Method For Nonlinear Optimization With Bound Constraints," Asia-Pacific Journal of Operational Research (APJOR), World Scientific Publishing Co. Pte. Ltd., vol. 30(03), pages 1-30.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:apmaco:v:451:y:2023:i:c:s0096300323001789. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/applied-mathematics-and-computation .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.