Measuring the prediction error. A comparison of cross-validation, bootstrap and covariance penalty methods
Author
Abstract
Suggested Citation
Download full text from publisher
As the access to this document is restricted, you may want to search for a different version of it.
References listed on IDEAS
- Chunming Zhang, 2008. "Prediction Error Estimation Under Bregman Divergence for Non‐Parametric Regression and Classification," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 35(3), pages 496-523, September.
- Bradley Efron, 2004. "The Estimation of Prediction Error: Covariance Penalties and Cross-Validation," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 619-632, January.
- Daudin, Jean-Jacques & Mary-Huard, Tristan, 2008. "Estimation of the conditional risk in classification: The swapping method," Computational Statistics & Data Analysis, Elsevier, vol. 52(6), pages 3220-3232, February.
- Yoshua Bengio & Yves Grandvalet, 2003. "No unbiased Estimator of the Variance of K-Fold Cross-Validation," CIRANO Working Papers 2003s-22, CIRANO.
- Shen X. & Ye J., 2002. "Adaptive Model Selection," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 210-221, March.
- Kim, Ji-Hyun, 2009. "Estimating classification error rate: Repeated cross-validation, repeated hold-out and bootstrap," Computational Statistics & Data Analysis, Elsevier, vol. 53(11), pages 3735-3745, September.
- Wisnowski, James W. & Simpson, James R. & Montgomery, Douglas C. & Runger, George C., 2003. "Resampling methods for variable selection in robust regression," Computational Statistics & Data Analysis, Elsevier, vol. 43(3), pages 341-355, July.
Citations
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
Cited by:
- Mario Guevara & Rodrigo Vargas, 2019. "Downscaling satellite soil moisture using geomorphometry and machine learning," PLOS ONE, Public Library of Science, vol. 14(9), pages 1-20, September.
- Nader Salari & Shamarina Shohaimi & Farid Najafi & Meenakshii Nallappan & Isthrinayagy Karishnarajah, 2014. "A Novel Hybrid Classification Model of Genetic Algorithms, Modified k-Nearest Neighbor and Developed Backpropagation Neural Network," PLOS ONE, Public Library of Science, vol. 9(11), pages 1-50, November.
- Keunhyun Park & Sadegh Sabouri & Torrey Lyons & Guang Tian & Reid Ewing, 2020. "Intrazonal or interzonal? Improving intrazonal travel forecast in a four-step travel demand model," Transportation, Springer, vol. 47(5), pages 2087-2108, October.
- Ha, Tran Vinh & Asada, Takumi & Arimura, Mikiharu, 2019. "Determination of the influence factors on household vehicle ownership patterns in Phnom Penh using statistical and machine learning methods," Journal of Transport Geography, Elsevier, vol. 78(C), pages 70-86.
- Bergmeir, Christoph & Hyndman, Rob J. & Koo, Bonsoo, 2018. "A note on the validity of cross-validation for evaluating autoregressive time series prediction," Computational Statistics & Data Analysis, Elsevier, vol. 120(C), pages 70-83.
- Usta, Ilhan & Kantar, Yeliz Mert, 2011. "On the performance of the flexible maximum entropy distributions within partially adaptive estimation," Computational Statistics & Data Analysis, Elsevier, vol. 55(6), pages 2172-2182, June.
- George Chalamandaris & Nikos E. Vlachogiannakis, 2018. "Are financial ratios relevant for trading credit risk? Evidence from the CDS market," Annals of Operations Research, Springer, vol. 266(1), pages 395-440, July.
- Melissa Adelman & Francisco Haimovich & Andres Ham & Emmanuel Vazquez, 2018.
"Predicting school dropout with administrative data: new evidence from Guatemala and Honduras,"
Education Economics, Taylor & Francis Journals, vol. 26(4), pages 356-372, July.
- Adelman,Melissa Ann & Haimovich,Francisco & Ham,Andres & Vazquez,Emmanuel Jose, 2017. "Predicting school dropout with administrative data: new evidence from Guatemala and Honduras," Policy Research Working Paper Series 8142, The World Bank.
- Abbasabadi, Narjes & Ashayeri, Mehdi & Azari, Rahman & Stephens, Brent & Heidarinejad, Mohammad, 2019. "An integrated data-driven framework for urban energy use modeling (UEUM)," Applied Energy, Elsevier, vol. 253(C), pages 1-1.
- Conde, David & Fernández, Miguel & Salvador, Bonifacio & Rueda, Cristina, 2015. "dawai: An R Package for Discriminant Analysis with Additional Information," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 66(i10).
- Christoph Bergmeir & Rob J Hyndman & Bonsoo Koo, 2015. "A Note on the Validity of Cross-Validation for Evaluating Time Series Prediction," Monash Econometrics and Business Statistics Working Papers 10/15, Monash University, Department of Econometrics and Business Statistics.
- Khaled Yousef Almansi & Abdul Rashid Mohamed Shariff & Bahareh Kalantar & Ahmad Fikri Abdullah & Sharifah Norkhadijah Syed Ismail & Naonori Ueda, 2022. "Performance Evaluation of Hospital Site Suitability Using Multilayer Perceptron (MLP) and Analytical Hierarchy Process (AHP) Models in Malacca, Malaysia," Sustainability, MDPI, vol. 14(7), pages 1-36, March.
- Bergmeir, Christoph & Costantini, Mauro & Benítez, José M., 2014. "On the usefulness of cross-validation for directional forecast evaluation," Computational Statistics & Data Analysis, Elsevier, vol. 76(C), pages 132-143.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Hirose, Kei & Tateishi, Shohei & Konishi, Sadanori, 2013. "Tuning parameter selection in sparse regression modeling," Computational Statistics & Data Analysis, Elsevier, vol. 59(C), pages 28-40.
- Yi, Feng & Zou, Hui, 2013. "SURE-tuned tapering estimation of large covariance matrices," Computational Statistics & Data Analysis, Elsevier, vol. 58(C), pages 339-351.
- Zhang, Bo & Shen, Xiaotong & Mumford, Sunni L., 2012. "Generalized degrees of freedom and adaptive model selection in linear mixed-effects models," Computational Statistics & Data Analysis, Elsevier, vol. 56(3), pages 574-586.
- Philip Reiss & Lei Huang & Joseph Cavanaugh & Amy Roy, 2012. "Resampling-based information criteria for best-subset regression," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 64(6), pages 1161-1186, December.
- Theo Dijkstra, 2014. "Ridge regression and its degrees of freedom," Quality & Quantity: International Journal of Methodology, Springer, vol. 48(6), pages 3185-3193, November.
- Sieds, 2012. "Complete Volume LXVI n.1 2012," RIEDS - Rivista Italiana di Economia, Demografia e Statistica - The Italian Journal of Economic, Demographic and Statistical Studies, SIEDS Societa' Italiana di Economia Demografia e Statistica, vol. 66(1), pages 1-296.
- Airola, Antti & Pahikkala, Tapio & Waegeman, Willem & De Baets, Bernard & Salakoski, Tapio, 2011. "An experimental comparison of cross-validation techniques for estimating the area under the ROC curve," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1828-1844, April.
- Hettihewa, Samanthala & Saha, Shrabani & Zhang, Hanxiong, 2018. "Does an aging population influence stock markets? Evidence from New Zealand," Economic Modelling, Elsevier, vol. 75(C), pages 142-158.
- Mendez, Guillermo & Lohr, Sharon, 2011. "Estimating residual variance in random forest regression," Computational Statistics & Data Analysis, Elsevier, vol. 55(11), pages 2937-2950, November.
- Yanagihara, Hirokazu & Satoh, Kenichi, 2010. "An unbiased Cp criterion for multivariate ridge regression," Journal of Multivariate Analysis, Elsevier, vol. 101(5), pages 1226-1238, May.
- G. Saharidis & I. Androulakis & M. Ierapetritou, 2011. "Model building using bi-level optimization," Journal of Global Optimization, Springer, vol. 49(1), pages 49-67, January.
- Zhang, Xinyu & Yu, Jihai, 2018. "Spatial weights matrix selection and model averaging for spatial autoregressive models," Journal of Econometrics, Elsevier, vol. 203(1), pages 1-18.
- Chunming Zhang, 2008. "Prediction Error Estimation Under Bregman Divergence for Non‐Parametric Regression and Classification," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 35(3), pages 496-523, September.
- John J Nay & Yevgeniy Vorobeychik, 2016. "Predicting Human Cooperation," PLOS ONE, Public Library of Science, vol. 11(5), pages 1-19, May.
- So-Won Choi & Bo-Guk Seo & Eul-Bum Lee, 2023. "Machine Learning-Based Tap Temperature Prediction and Control for Optimized Power Consumption in Stainless Electric Arc Furnaces (EAF) of Steel Plants," Sustainability, MDPI, vol. 15(8), pages 1-31, April.
- Giessing, Alexander & He, Xuming, 2019. "On the predictive risk in misspecified quantile regression," Journal of Econometrics, Elsevier, vol. 213(1), pages 235-260.
- Matthew Tuson & Berwin Turlach & Kevin Murray & Mei Ruu Kok & Alistair Vickery & David Whyatt, 2021. "Predicting Future Geographic Hotspots of Potentially Preventable Hospitalisations Using All Subset Model Selection and Repeated K-Fold Cross-Validation," IJERPH, MDPI, vol. 18(19), pages 1-21, September.
- James Younker, 2022. "Calculating Effective Degrees of Freedom for Forecast Combinations and Ensemble Models," Discussion Papers 2022-19, Bank of Canada.
- Wang, You-Gan & Hin, Lin-Yee, 2010. "Modeling strategies in longitudinal data analysis: Covariate, variance function and correlation structure selection," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3359-3370, December.
- Daudin, Jean-Jacques & Mary-Huard, Tristan, 2008. "Estimation of the conditional risk in classification: The swapping method," Computational Statistics & Data Analysis, Elsevier, vol. 52(6), pages 3220-3232, February.
More about this item
Keywords
Prediction error Extra-sample error In-sample error Optimism Cross-validation Leave-one-out Bootstrap Covariance penalty Regression trees Projection pursuit regression Neural networks;Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:54:y:2010:i:12:p:2976-2989. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.