IDEAS home Printed from https://ideas.repec.org/a/eee/stapro/v190y2022ics0167715222001560.html
   My bibliography  Save this article

Nonparametric regression with modified ReLU networks

Author

Listed:
  • Beknazaryan, Aleksandr
  • Sang, Hailin

Abstract

We consider regression estimation with modified ReLU neural networks in which network weight matrices are first modified by a function α before being multiplied by input vectors. We give an example of continuous, piecewise linear function α for which the empirical risk minimizers over the classes of modified ReLU networks with l1 and squared l2 penalties attain, up to a logarithmic factor, the minimax rate of prediction of unknown β-smooth function.

Suggested Citation

  • Beknazaryan, Aleksandr & Sang, Hailin, 2022. "Nonparametric regression with modified ReLU networks," Statistics & Probability Letters, Elsevier, vol. 190(C).
    • Handle: RePEc:eee:stapro:v:190:y:2022:i:c:s0167715222001560
    DOI: 10.1016/j.spl.2022.109624
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167715222001560
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.spl.2022.109624?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Antoniadis, Anestis & Bigot, Jeremie & Sapatinas, Theofanis, 2001. "Wavelet Estimators in Nonparametric Regression: A Comparative Simulation Study," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 6(i06).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Autin, Florent & Freyermuth, Jean-Marc & von Sachs, Rainer, 2011. "Combining thresholding rules: a new way to improve the performance of wavelet estimators," LIDAM Discussion Papers ISBA 2011021, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    2. De Canditiis, Daniela, 2014. "A frame based shrinkage procedure for fast oscillating functions," Computational Statistics & Data Analysis, Elsevier, vol. 75(C), pages 142-150.
    3. Yu, Dengdeng & Zhang, Li & Mizera, Ivan & Jiang, Bei & Kong, Linglong, 2019. "Sparse wavelet estimation in quantile regression with multiple functional predictors," Computational Statistics & Data Analysis, Elsevier, vol. 136(C), pages 12-29.
    4. T. Palanisamy & J. Ravichandran, 2015. "A wavelet-based hybrid approach to estimate variance function in heteroscedastic regression models," Statistical Papers, Springer, vol. 56(3), pages 911-932, August.
    5. Autin, Florent & Freyermuth, Jean-Marc & von Sachs, Rainer, 2011. "Block-Threshold-Adapted Estimators via a maxiset approach," LIDAM Discussion Papers ISBA 2011017, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    6. Youssef Taleb & Edward A. K. Cohen, 2021. "Multiresolution analysis of point processes and statistical thresholding for Haar wavelet-based intensity estimation," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 73(2), pages 395-423, April.
    7. Antoniadis, Anestis & Sapatinas, Theofanis, 2003. "Wavelet methods for continuous-time prediction using Hilbert-valued autoregressive processes," Journal of Multivariate Analysis, Elsevier, vol. 87(1), pages 133-158, October.
    8. Lee, Kichun & Vidakovic, Brani, 2012. "Semi-supervised wavelet shrinkage," Computational Statistics & Data Analysis, Elsevier, vol. 56(6), pages 1681-1691.
    9. repec:ibn:ijspnl:v:8:y:2019:i:4:p:32 is not listed on IDEAS
    10. Céline Lacaux & Aurélie Muller-Gueudin & Radu Ranta & Samy Tindel, 2014. "Convergence and performance of the peeling wavelet denoising algorithm," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 77(4), pages 509-537, May.
    11. Nilotpal Sanyal & Marco A. R. Ferreira, 2017. "Bayesian Wavelet Analysis Using Nonlocal Priors with an Application to fMRI Analysis," Sankhya B: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 79(2), pages 361-388, November.
    12. Florent Autin & Jean-Marc Freyermuth & Rainer Von Sachs, 2014. "Block-threshold-adapted Estimators via a Maxiset Approach," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 41(1), pages 240-258, March.
    13. Vincent Rivoirard, 2004. "Thresholding procedure with priors based on Pareto distributions," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 13(1), pages 213-246, June.
    14. Helida Nurcahayani & I Nyoman Budiantara & Ismaini Zain, 2021. "The Curve Estimation of Combined Truncated Spline and Fourier Series Estimators for Multiresponse Nonparametric Regression," Mathematics, MDPI, vol. 9(10), pages 1-22, May.
    15. Serban, Nicoleta, 2010. "Noise reduction for enhanced component identification in multi-dimensional biomolecular NMR studies," Computational Statistics & Data Analysis, Elsevier, vol. 54(4), pages 1051-1065, April.
    16. Aminghafari, Mina & Cheze, Nathalie & Poggi, Jean-Michel, 2006. "Multivariate denoising using wavelets and principal component analysis," Computational Statistics & Data Analysis, Elsevier, vol. 50(9), pages 2381-2398, May.
    17. Fryzlewicz, Piotr, 2007. "Bivariate hard thresholding in wavelet function estimation," LSE Research Online Documents on Economics 25219, London School of Economics and Political Science, LSE Library.
    18. Fabien Navarro & Adrien Saumard, 2017. "Slope heuristics and V-Fold model selection in heteroscedastic regression using strongly localized bases," Working Papers 2017-67, Center for Research in Economics and Statistics.
    19. T. W. Randolph & Y. Yasui, 2006. "Multiscale Processing of Mass Spectrometry Data," Biometrics, The International Biometric Society, vol. 62(2), pages 589-597, June.
    20. Autin, F. & Freyermuth, Jean-Marc & von Sachs, Rainer, 2011. "Ideal denoising within a family of tree-structured wavelet estimators," LIDAM Discussion Papers ISBA 2011002, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    21. Jacinta Chan Phooi M’ng & Mohammadali Mehralizadeh, 2016. "Forecasting East Asian Indices Futures via a Novel Hybrid of Wavelet-PCA Denoising and Artificial Neural Network Models," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-29, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:stapro:v:190:y:2022:i:c:s0167715222001560. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.