IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v276y2019i1d10.1007_s10479-018-2847-6.html
   My bibliography  Save this article

Relaxed support vector regression

Author

Listed:
  • Orestis P. Panagopoulos

    (California State University, Stanislaus)

  • Petros Xanthopoulos

    (Stetson University)

  • Talayeh Razzaghi

    (New Mexico State University)

  • Onur Şeref

    (Virginia Polytechnic Institute and State University)

Abstract

Datasets with outliers pose a serious challenge in regression analysis. In this paper, a new regression method called relaxed support vector regression (RSVR) is proposed for such datasets. RSVR is based on the concept of constraint relaxation which leads to increased robustness in datasets with outliers. RSVR is formulated using both linear and quadratic loss functions. Numerical experiments on benchmark datasets and computational comparisons with other popular regression methods depict the behavior of our proposed method. RSVR achieves better overall performance than support vector regression (SVR) in measures such as RMSE and $$R^2_{adj}$$ R adj 2 while being on par with other state-of-the-art regression methods such as robust regression (RR). Additionally, RSVR provides robustness for higher dimensional datasets which is a limitation of RR, the robust equivalent of ordinary least squares regression. Moreover, RSVR can be used on datasets that contain varying levels of noise.

Suggested Citation

  • Orestis P. Panagopoulos & Petros Xanthopoulos & Talayeh Razzaghi & Onur Şeref, 2019. "Relaxed support vector regression," Annals of Operations Research, Springer, vol. 276(1), pages 191-210, May.
    • Handle: RePEc:spr:annopr:v:276:y:2019:i:1:d:10.1007_s10479-018-2847-6
    DOI: 10.1007/s10479-018-2847-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10479-018-2847-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10479-018-2847-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Huang, Chien-Ming & Lee, Yuh-Jye & Lin, Dennis K.J. & Huang, Su-Yun, 2007. "Model selection for support vector machines via uniform design," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 335-346, September.
    2. Onur Şeref & Wanpracha Chaovalitwongse & J. Brooks, 2014. "Relaxing support vectors for classification," Annals of Operations Research, Springer, vol. 216(1), pages 229-255, May.
    3. Johansen, Soren, 1988. "Statistical analysis of cointegration vectors," Journal of Economic Dynamics and Control, Elsevier, vol. 12(2-3), pages 231-254.
    4. Panagopoulos, Orestis P. & Pappu, Vijay & Xanthopoulos, Petros & Pardalos, Panos M., 2016. "Constrained subspace classifier for high dimensional datasets," Omega, Elsevier, vol. 59(PA), pages 40-46.
    5. Harrison, David Jr. & Rubinfeld, Daniel L., 1978. "Hedonic housing prices and the demand for clean air," Journal of Environmental Economics and Management, Elsevier, vol. 5(1), pages 81-102, March.
    6. Claudio Cifarelli & Mario R. Guarracino & Onur Seref & Salvatore Cuciniello & Panos M. Pardalos, 2007. "Incremental Classification with Generalized Eigenvalues," Journal of Classification, Springer;The Classification Society, vol. 24(2), pages 205-219, September.
    7. Petros Xanthopoulos & Mario Guarracino & Panos Pardalos, 2014. "Robust generalized eigenvalue classifier with ellipsoidal uncertainty," Annals of Operations Research, Springer, vol. 216(1), pages 327-342, May.
    8. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    9. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mike G. Tsionas, 2021. "Multi-criteria optimization in regression," Annals of Operations Research, Springer, vol. 306(1), pages 7-25, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Umberto Amato & Anestis Antoniadis & Italia De Feis & Irene Gijbels, 2021. "Penalised robust estimators for sparse and high-dimensional linear models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(1), pages 1-48, March.
    2. Yinjun Chen & Hao Ming & Hu Yang, 2024. "Efficient variable selection for high-dimensional multiplicative models: a novel LPRE-based approach," Statistical Papers, Springer, vol. 65(6), pages 3713-3737, August.
    3. Narayan, Seema & Smyth, Russell, 2015. "The financial econometrics of price discovery and predictability," International Review of Financial Analysis, Elsevier, vol. 42(C), pages 380-393.
    4. JinXing Che & YouLong Yang, 2017. "Stochastic correlation coefficient ensembles for variable selection," Journal of Applied Statistics, Taylor & Francis Journals, vol. 44(10), pages 1721-1742, July.
    5. Fakhri J. Hasanov & Muhammad Javid & Frederick L. Joutz, 2022. "Saudi Non-Oil Exports before and after COVID-19: Historical Impacts of Determinants and Scenario Analysis," Sustainability, MDPI, vol. 14(4), pages 1-38, February.
    6. Costa, Alexandre Bonnet R. & Ferreira, Pedro Cavalcanti G. & Gaglianone, Wagner P. & Guillén, Osmani Teixeira C. & Issler, João Victor & Lin, Yihao, 2021. "Machine learning and oil price point and density forecasting," Energy Economics, Elsevier, vol. 102(C).
    7. Liang, Chong & Schienle, Melanie, 2019. "Determination of vector error correction models in high dimensions," Journal of Econometrics, Elsevier, vol. 208(2), pages 418-441.
    8. Xuan Liu & Jianbao Chen, 2021. "Variable Selection for the Spatial Autoregressive Model with Autoregressive Disturbances," Mathematics, MDPI, vol. 9(12), pages 1-20, June.
    9. Krüger, Jens & Ruths Sion, Sebastian, 2019. "Improving oil price forecasts by sparse VAR methods," Darmstadt Discussion Papers in Economics 237, Darmstadt University of Technology, Department of Law and Economics.
    10. Onur Şeref & Talayeh Razzaghi & Petros Xanthopoulos, 2017. "Weighted relaxed support vector machines," Annals of Operations Research, Springer, vol. 249(1), pages 235-271, February.
    11. Mai Dao & Min Wang & Souparno Ghosh & Keying Ye, 2022. "Bayesian variable selection and estimation in quantile regression using a quantile-specific prior," Computational Statistics, Springer, vol. 37(3), pages 1339-1368, July.
    12. Tutz, Gerhard & Pößnecker, Wolfgang & Uhlmann, Lorenz, 2015. "Variable selection in general multinomial logit models," Computational Statistics & Data Analysis, Elsevier, vol. 82(C), pages 207-222.
    13. Carstensen, Kai & Heinrich, Markus & Reif, Magnus & Wolters, Maik H., 2020. "Predicting ordinary and severe recessions with a three-state Markov-switching dynamic factor model," International Journal of Forecasting, Elsevier, vol. 36(3), pages 829-850.
    14. Hou-Tai Chang & Ping-Huai Wang & Wei-Fang Chen & Chen-Ju Lin, 2022. "Risk Assessment of Early Lung Cancer with LDCT and Health Examinations," IJERPH, MDPI, vol. 19(8), pages 1-12, April.
    15. Wang, Qiao & Zhou, Wei & Cheng, Yonggang & Ma, Gang & Chang, Xiaolin & Miao, Yu & Chen, E, 2018. "Regularized moving least-square method and regularized improved interpolating moving least-square method with nonsingular moment matrices," Applied Mathematics and Computation, Elsevier, vol. 325(C), pages 120-145.
    16. Mkhadri, Abdallah & Ouhourane, Mohamed, 2013. "An extended variable inclusion and shrinkage algorithm for correlated variables," Computational Statistics & Data Analysis, Elsevier, vol. 57(1), pages 631-644.
    17. Lucian Belascu & Alexandra Horobet & Georgiana Vrinceanu & Consuela Popescu, 2021. "Performance Dissimilarities in European Union Manufacturing: The Effect of Ownership and Technological Intensity," Sustainability, MDPI, vol. 13(18), pages 1-19, September.
    18. Candelon, B. & Hurlin, C. & Tokpavi, S., 2012. "Sampling error and double shrinkage estimation of minimum variance portfolios," Journal of Empirical Finance, Elsevier, vol. 19(4), pages 511-527.
    19. Andrea Carriero & Todd E. Clark & Massimiliano Marcellino, 2022. "Specification Choices in Quantile Regression for Empirical Macroeconomics," Working Papers 22-25, Federal Reserve Bank of Cleveland.
    20. Kim, Hyun Hak & Swanson, Norman R., 2018. "Mining big data using parsimonious factor, machine learning, variable selection and shrinkage methods," International Journal of Forecasting, Elsevier, vol. 34(2), pages 339-354.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:276:y:2019:i:1:d:10.1007_s10479-018-2847-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.