IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v313y2024i3p858-870.html
   My bibliography  Save this article

Convex support vector regression

Author

Listed:
  • Liao, Zhiqiang
  • Dai, Sheng
  • Kuosmanen, Timo

Abstract

Nonparametric regression subject to convexity or concavity constraints is increasingly popular in economics, finance, operations research, machine learning, and statistics. However, the conventional convex regression based on the least squares loss function often suffers from overfitting and outliers. This paper proposes to address these two issues by introducing the convex support vector regression (CSVR) method, which effectively combines the key elements of convex regression and support vector regression. Numerical experiments demonstrate the performance of CSVR in prediction accuracy and robustness that compares favorably with other state-of-the-art methods.

Suggested Citation

  • Liao, Zhiqiang & Dai, Sheng & Kuosmanen, Timo, 2024. "Convex support vector regression," European Journal of Operational Research, Elsevier, vol. 313(3), pages 858-870.
    • Handle: RePEc:eee:ejores:v:313:y:2024:i:3:p:858-870
    DOI: 10.1016/j.ejor.2023.05.009
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221723003715
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2023.05.009?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Tanaka, Kenta & Managi, Shunsuke, 2021. "Industrial agglomeration effect for energy efficiency in Japanese production plants," Energy Policy, Elsevier, vol. 156(C).
    2. Timo Kuosmanen & Andrew Johnson & Antti Saastamoinen, 2015. "Stochastic Nonparametric Approach to Efficiency Analysis: A Unified Framework," International Series in Operations Research & Management Science, in: Joe Zhu (ed.), Data Envelopment Analysis, edition 127, chapter 7, pages 191-244, Springer.
    3. Preciado Arreola, José Luis & Johnson, Andrew L. & Chen, Xun C. & Morita, Hiroshi, 2020. "Estimating stochastic production frontiers: A one-stage multivariate semiparametric Bayesian concave regression method," European Journal of Operational Research, Elsevier, vol. 287(2), pages 699-711.
    4. Dimitris Bertsimas & Nishanth Mundru, 2021. "Sparse Convex Regression," INFORMS Journal on Computing, INFORMS, vol. 33(1), pages 262-279, January.
    5. Rahul Mazumder & Arkopal Choudhury & Garud Iyengar & Bodhisattva Sen, 2019. "A Computational Framework for Multivariate Convex Regression and Its Variants," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(525), pages 318-331, January.
    6. Daisuke Yagi & Yining Chen & Andrew L. Johnson & Timo Kuosmanen, 2020. "Shape-Constrained Kernel-Weighted Least Squares: Estimating Production Functions for Chilean Manufacturing Industries," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(1), pages 43-54, January.
    7. Lee, Chia-Yen & Johnson, Andrew L. & Moreno-Centeno, Erick & Kuosmanen, Timo, 2013. "A more efficient algorithm for Convex Nonparametric Least Squares," European Journal of Operational Research, Elsevier, vol. 227(2), pages 391-400.
    8. Timo Kuosmanen, 2008. "Representation theorem for convex nonparametric least squares," Econometrics Journal, Royal Economic Society, vol. 11(2), pages 308-325, July.
    9. Kuosmanen, Timo & Zhou, Xun & Dai, Sheng, 2020. "How much climate policy has cost for OECD countries?," World Development, Elsevier, vol. 125(C).
    10. Kelley Pace, R. & Barry, Ronald, 1997. "Sparse spatial autoregressions," Statistics & Probability Letters, Elsevier, vol. 33(3), pages 291-297, May.
    11. Kuosmanen, Timo, 2012. "Stochastic semi-nonparametric frontier estimation of electricity distribution networks: Application of the StoNED method in the Finnish regulatory model," Energy Economics, Elsevier, vol. 34(6), pages 2189-2199.
    12. Lin, Dongyu & Foster, Dean P. & Ungar, Lyle H., 2011. "VIF Regression: A Fast Regression Algorithm for Large Data," Journal of the American Statistical Association, American Statistical Association, vol. 106(493), pages 232-247.
    13. Johnson, Andrew L. & Kuosmanen, Timo, 2012. "One-stage and two-stage DEA estimation of the effects of contextual variables," European Journal of Operational Research, Elsevier, vol. 220(2), pages 559-570.
    14. Promit Ghosal & Bodhisattva Sen, 2017. "On Univariate Convex Regression," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 79(2), pages 215-253, August.
    15. Timo Kuosmanen & Andrew L. Johnson, 2010. "Data Envelopment Analysis as Nonparametric Least-Squares Regression," Operations Research, INFORMS, vol. 58(1), pages 149-160, February.
    16. Wang, Yongqiao & Wang, Shouyang & Dang, Chuangyin & Ge, Wenxiu, 2014. "Nonparametric quantile frontier estimation under shape restriction," European Journal of Operational Research, Elsevier, vol. 232(3), pages 671-678.
    17. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    18. Kuosmanen, Timo & Zhou, Xun, 2021. "Shadow prices and marginal abatement costs: Convex quantile regression approach," European Journal of Operational Research, Elsevier, vol. 289(2), pages 666-675.
    19. Eunji Lim, 2014. "On Convergence Rates of Convex Regression in Multiple Dimensions," INFORMS Journal on Computing, INFORMS, vol. 26(3), pages 616-628, August.
    20. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    21. Timo Kuosmanen and Andrew L. Johnson, 2020. "Conditional Yardstick Competition in Energy Regulation," The Energy Journal, International Association for Energy Economics, vol. 0(Special I).
    22. Xi Chen & Qihang Lin & Bodhisattva Sen, 2020. "On Degrees of Freedom of Projection Estimators With Applications to Multivariate Nonparametric Regression," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 115(529), pages 173-186, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhiqiang Liao, 2024. "Variable selection in convex nonparametric least squares via structured Lasso: An application to the Swedish electricity distribution networks," Papers 2409.01911, arXiv.org, revised Nov 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dai, Sheng & Kuosmanen, Timo & Zhou, Xun, 2023. "Generalized quantile and expectile properties for shape constrained nonparametric estimation," European Journal of Operational Research, Elsevier, vol. 310(2), pages 914-927.
    2. Dai, Sheng, 2023. "Variable selection in convex quantile regression: L1-norm or L0-norm regularization?," European Journal of Operational Research, Elsevier, vol. 305(1), pages 338-355.
    3. Wen, Xiaojie & Yao, Shunbo & Sauer, Johannes, 2022. "Shadow prices and abatement cost of soil erosion in Shaanxi Province, China: Convex expectile regression approach," Ecological Economics, Elsevier, vol. 201(C).
    4. Zhiqiang Liao, 2024. "Variable selection in convex nonparametric least squares via structured Lasso: An application to the Swedish electricity distribution networks," Papers 2409.01911, arXiv.org, revised Nov 2024.
    5. Cristina Polo & Julián Ramajo & Alejandro Ricci‐Risquete, 2021. "A stochastic semi‐non‐parametric analysis of regional efficiency in the European Union," Regional Science Policy & Practice, Wiley Blackwell, vol. 13(1), pages 7-24, February.
    6. K. Hervé Dakpo & Yann Desjeux & Laure Latruffe, 2023. "Cost of abating excess nitrogen on wheat plots in France: An assessment with multi‐technology modelling," Journal of Agricultural Economics, Wiley Blackwell, vol. 74(3), pages 800-815, September.
    7. Lee, Chia-Yen & Wang, Ke, 2019. "Nash marginal abatement cost estimation of air pollutant emissions using the stochastic semi-nonparametric frontier," European Journal of Operational Research, Elsevier, vol. 273(1), pages 390-400.
    8. Tsionas, Mike G., 2023. "Joint production in stochastic non-parametric envelopment of data with firm-specific directions," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1336-1347.
    9. Kuosmanen, Timo & Zhou, Xun, 2021. "Shadow prices and marginal abatement costs: Convex quantile regression approach," European Journal of Operational Research, Elsevier, vol. 289(2), pages 666-675.
    10. Timo Kuosmanen & Yong Tan & Sheng Dai, 2023. "Performance analysis of English hospitals during the first and second waves of the coronavirus pandemic," Health Care Management Science, Springer, vol. 26(3), pages 447-460, September.
    11. Jose M. Cordero & Cristina Polo & Daniel Santín, 2020. "Assessment of new methods for incorporating contextual variables into efficiency measures: a Monte Carlo simulation," Operational Research, Springer, vol. 20(4), pages 2245-2265, December.
    12. Kuosmanen, Timo & Johnson, Andrew, 2017. "Modeling joint production of multiple outputs in StoNED: Directional distance function approach," European Journal of Operational Research, Elsevier, vol. 262(2), pages 792-801.
    13. Timo Kuosmanen & Sheng Dai, 2023. "Modeling economies of scope in joint production: Convex regression of input distance function," Papers 2311.11637, arXiv.org.
    14. Layer, Kevin & Johnson, Andrew L. & Sickles, Robin C. & Ferrier, Gary D., 2020. "Direction selection in stochastic directional distance functions," European Journal of Operational Research, Elsevier, vol. 280(1), pages 351-364.
    15. Duras, Toni & Javed, Farrukh & Månsson, Kristofer & Sjölander, Pär & Söderberg, Magnus, 2023. "Using machine learning to select variables in data envelopment analysis: Simulations and application using electricity distribution data," Energy Economics, Elsevier, vol. 120(C).
    16. Tsionas, Mike G., 2022. "Convex non-parametric least squares, causal structures and productivity," European Journal of Operational Research, Elsevier, vol. 303(1), pages 370-387.
    17. Wei-han Liu & Qian Long Kweh, 2022. "Reexamining nonlinear effects of intellectual capital on firm efficiency," Annals of Operations Research, Springer, vol. 315(2), pages 1319-1344, August.
    18. Lee, Chia-Yen & Johnson, Andrew L. & Moreno-Centeno, Erick & Kuosmanen, Timo, 2013. "A more efficient algorithm for Convex Nonparametric Least Squares," European Journal of Operational Research, Elsevier, vol. 227(2), pages 391-400.
    19. Eskelinen, Juha & Kuosmanen, Timo, 2013. "Intertemporal efficiency analysis of sales teams of a bank: Stochastic semi-nonparametric approach," Journal of Banking & Finance, Elsevier, vol. 37(12), pages 5163-5175.
    20. Tsionas, Mike, 2022. "Efficiency estimation using probabilistic regression trees with an application to Chilean manufacturing industries," International Journal of Production Economics, Elsevier, vol. 249(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:313:y:2024:i:3:p:858-870. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.