IDEAS home Printed from https://ideas.repec.org/a/bla/scjsta/v35y2008i3p496-523.html
   My bibliography  Save this article

Prediction Error Estimation Under Bregman Divergence for Non‐Parametric Regression and Classification

Author

Listed:
  • CHUNMING ZHANG

Abstract

. Prediction error is critical to assess model fit and evaluate model prediction. We propose the cross‐validation (CV) and approximated CV methods for estimating prediction error under the Bregman divergence (BD), which embeds nearly all of the commonly used loss functions in the regression, classification procedures and machine learning literature. The approximated CV formulas are analytically derived, which facilitate fast estimation of prediction error under BD. We then study a data‐driven optimal bandwidth selector for local‐likelihood estimation that minimizes the overall prediction error or equivalently the covariance penalty. It is shown that the covariance penalty and CV methods converge to the same mean‐prediction‐error‐criterion. We also propose a lower‐bound scheme for computing the local logistic regression estimates and demonstrate that the algorithm monotonically enhances the target local likelihood and converges. The idea and methods are extended to the generalized varying‐coefficient models and additive models.

Suggested Citation

  • Chunming Zhang, 2008. "Prediction Error Estimation Under Bregman Divergence for Non‐Parametric Regression and Classification," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 35(3), pages 496-523, September.
  • Handle: RePEc:bla:scjsta:v:35:y:2008:i:3:p:496-523
    DOI: 10.1111/j.1467-9469.2008.00593.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1467-9469.2008.00593.x
    Download Restriction: no

    File URL: https://libkey.io/10.1111/j.1467-9469.2008.00593.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Bradley Efron, 2004. "The Estimation of Prediction Error: Covariance Penalties and Cross-Validation," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 619-632, January.
    2. Harrison, David Jr. & Rubinfeld, Daniel L., 1978. "Hedonic housing prices and the demand for clean air," Journal of Environmental Economics and Management, Elsevier, vol. 5(1), pages 81-102, March.
    3. Zhang, Chunming, 2003. "Calibrating the Degrees of Freedom for Automatic Data Smoothing and Effective Curve Checking," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 609-628, January.
    4. J. Fan & J. Chen, 1999. "One‐step local quasi‐likelihood estimation," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 61(4), pages 927-943.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Xia Cui & Heng Peng & Songqiao Wen & Lixing Zhu, 2013. "Component Selection in the Additive Regression Model," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 40(3), pages 491-510, September.
    2. Borra, Simone & Di Ciaccio, Agostino, 2010. "Measuring the prediction error. A comparison of cross-validation, bootstrap and covariance penalty methods," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 2976-2989, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jing Wang & Lijian Yang, 2009. "Efficient and fast spline-backfitted kernel smoothing of additive models," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 61(3), pages 663-690, September.
    2. Zhang, Chunming, 2010. "Statistical inference of minimum BD estimators and classifiers for varying-dimensional models," Journal of Multivariate Analysis, Elsevier, vol. 101(7), pages 1574-1593, August.
    3. Lucija Muehlenbachs & Elisheba Spiller & Christopher Timmins, 2015. "The Housing Market Impacts of Shale Gas Development," American Economic Review, American Economic Association, vol. 105(12), pages 3633-3659, December.
    4. Jianhong Shi & Qian Yang & Xiongya Li & Weixing Song, 2017. "Effects of measurement error on a class of single-index varying coefficient regression models," Computational Statistics, Springer, vol. 32(3), pages 977-1001, September.
    5. Theo Dijkstra, 2014. "Ridge regression and its degrees of freedom," Quality & Quantity: International Journal of Methodology, Springer, vol. 48(6), pages 3185-3193, November.
    6. Smith, Michael & Kohn, Robert, 1996. "Nonparametric regression using Bayesian variable selection," Journal of Econometrics, Elsevier, vol. 75(2), pages 317-343, December.
    7. Villalonga, Belen, 2004. "Intangible resources, Tobin's q, and sustainability of performance differences," Journal of Economic Behavior & Organization, Elsevier, vol. 54(2), pages 205-230, June.
    8. Long Feng & Changliang Zou & Zhaojun Wang & Lixing Zhu, 2015. "Robust comparison of regression curves," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 24(1), pages 185-204, March.
    9. Brockmeier, M., 1991. "Entwicklung und Aufhebung von Reinheitsgeboten im Nahrungsmittelbereich – Analyse und Bewertung," Proceedings “Schriften der Gesellschaft für Wirtschafts- und Sozialwissenschaften des Landbaues e.V.”, German Association of Agricultural Economists (GEWISOLA), vol. 27.
    10. Miles M Finney, 2017. "Air Quality and the Development of Los Angeles," The Review of Regional Studies, Southern Regional Science Association, vol. 47(3), pages 271-288, Fall.
    11. Terri Menke, 1987. "Economic Welfare and Urban Amenities Across Race-Sex Groups," Urban Studies, Urban Studies Journal Limited, vol. 24(2), pages 151-161, April.
    12. Sieds, 2012. "Complete Volume LXVI n.1 2012," RIEDS - Rivista Italiana di Economia, Demografia e Statistica - The Italian Journal of Economic, Demographic and Statistical Studies, SIEDS Societa' Italiana di Economia Demografia e Statistica, vol. 66(1), pages 1-296.
    13. Suneel Babu Chatla, 2023. "Nonparametric inference for additive models estimated via simplified smooth backfitting," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 75(1), pages 71-97, February.
    14. Miller, Steve & Startz, Richard, 2019. "Feasible generalized least squares using support vector regression," Economics Letters, Elsevier, vol. 175(C), pages 28-31.
    15. Chunfang Zhao & Yingliang Wu & Yunfeng Chen & Guohua Chen, 2023. "Multiscale Effects of Hedonic Attributes on Airbnb Listing Prices Based on MGWR: A Case Study of Beijing, China," Sustainability, MDPI, vol. 15(2), pages 1-21, January.
    16. Umberto Amato & Anestis Antoniadis & Italia De Feis & Irene Gijbels, 2021. "Penalised robust estimators for sparse and high-dimensional linear models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(1), pages 1-48, March.
    17. Prendergast, Luke A. & Li Wai Suen, Connie, 2011. "A new and practical influence measure for subsets of covariance matrix sample principal components with applications to high dimensional datasets," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 752-764, January.
    18. repec:asg:wpaper:1006 is not listed on IDEAS
    19. Tizheng Li & Xiaojuan Kang, 2022. "Variable selection of higher-order partially linear spatial autoregressive model with a diverging number of parameters," Statistical Papers, Springer, vol. 63(1), pages 243-285, February.
    20. Deac Dan Stelian & Schebesch Klaus Bruno, 2018. "Market Forecasts and Client Behavioral Data: Towards Finding Adequate Model Complexity," Studia Universitatis „Vasile Goldis” Arad – Economics Series, Sciendo, vol. 28(3), pages 50-75, September.
    21. James Hansen & James McDonald & Panayiotis Theodossiou & Brad Larsen, 2010. "Partially Adaptive Econometric Methods For Regression and Classification," Computational Economics, Springer;Society for Computational Economics, vol. 36(2), pages 153-169, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:scjsta:v:35:y:2008:i:3:p:496-523. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0303-6898 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.