IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v305y2023i1p338-355.html
   My bibliography  Save this article

Variable selection in convex quantile regression: L1-norm or L0-norm regularization?

Author

Listed:
  • Dai, Sheng

Abstract

The curse of dimensionality is a recognized challenge in nonparametric estimation. This paper develops a new L0-norm regularization approach to the convex quantile and expectile regressions for subset selection. We show how to use mixed-integer programming to solve the proposed L0-norm regularization approach in practice and build a link to the commonly used L1-norm regularization approach. A Monte Carlo study is performed to compare the finite sample performances of the proposed L0-penalized convex quantile and expectile regression approaches with the L1-norm regularization approaches. The proposed approach is further applied to benchmark the sustainable development performance of the OECD countries and empirically analyze the accuracy in the dimensionality reduction of variables. The results from the simulation and application illustrate that the proposed L0-norm regularization approach can more effectively address the curse of dimensionality than the L1-norm regularization approach in multidimensional spaces.

Suggested Citation

  • Dai, Sheng, 2023. "Variable selection in convex quantile regression: L1-norm or L0-norm regularization?," European Journal of Operational Research, Elsevier, vol. 305(1), pages 338-355.
    • Handle: RePEc:eee:ejores:v:305:y:2023:i:1:p:338-355
    DOI: 10.1016/j.ejor.2022.05.041
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221722004313
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2022.05.041?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Behr, Andreas, 2010. "Quantile regression for robust bank efficiency score estimation," European Journal of Operational Research, Elsevier, vol. 200(2), pages 568-581, January.
    2. Zou, Hui, 2006. "The Adaptive Lasso and Its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1418-1429, December.
    3. Timo Kuosmanen & Andrew Johnson & Antti Saastamoinen, 2015. "Stochastic Nonparametric Approach to Efficiency Analysis: A Unified Framework," International Series in Operations Research & Management Science, in: Joe Zhu (ed.), Data Envelopment Analysis, edition 127, chapter 7, pages 191-244, Springer.
    4. Wilson, Paul W., 2018. "Dimension reduction in nonparametric models of production," European Journal of Operational Research, Elsevier, vol. 267(1), pages 349-367.
    5. Charles, Vincent & Aparicio, Juan & Zhu, Joe, 2019. "The curse of dimensionality of decision-making units: A simple approach to increase the discriminatory power of data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 279(3), pages 929-940.
    6. Lee, Chia-Yen & Cai, Jia-Ying, 2020. "LASSO variable selection in data envelopment analysis with small datasets," Omega, Elsevier, vol. 91(C).
    7. Le-Yu Chen & Sokbae Lee, 2021. "Binary classification with covariate selection through ℓ0-penalised empirical risk minimisation," The Econometrics Journal, Royal Economic Society, vol. 24(1), pages 103-120.
    8. Dimitris Bertsimas & Nishanth Mundru, 2021. "Sparse Convex Regression," INFORMS Journal on Computing, INFORMS, vol. 33(1), pages 262-279, January.
    9. Rahul Mazumder & Arkopal Choudhury & Garud Iyengar & Bodhisattva Sen, 2019. "A Computational Framework for Multivariate Convex Regression and Its Variants," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(525), pages 318-331, January.
    10. Lee, Chia-Yen & Johnson, Andrew L. & Moreno-Centeno, Erick & Kuosmanen, Timo, 2013. "A more efficient algorithm for Convex Nonparametric Least Squares," European Journal of Operational Research, Elsevier, vol. 227(2), pages 391-400.
    11. Adler, Nicole & Yazhemsky, Ekaterina, 2010. "Improving discrimination in data envelopment analysis: PCA-DEA or variable reduction," European Journal of Operational Research, Elsevier, vol. 202(1), pages 273-284, April.
    12. Timo Kuosmanen, 2008. "Representation theorem for convex nonparametric least squares," Econometrics Journal, Royal Economic Society, vol. 11(2), pages 308-325, July.
    13. Benítez-Peña, Sandra & Bogetoft, Peter & Romero Morales, Dolores, 2020. "Feature Selection in Data Envelopment Analysis: A Mathematical Optimization approach," Omega, Elsevier, vol. 96(C).
    14. Kuosmanen, Timo & Zhou, Xun & Dai, Sheng, 2020. "How much climate policy has cost for OECD countries?," World Development, Elsevier, vol. 125(C).
    15. Jesús T. Pastor & JosÉ L. Ruiz & Inmaculada Sirvent, 2002. "A Statistical Test for Nested Radial Dea Models," Operations Research, INFORMS, vol. 50(4), pages 728-735, August.
    16. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    17. Nataraja, Niranjan R. & Johnson, Andrew L., 2011. "Guidelines for using variable selection techniques in data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 215(3), pages 662-669, December.
    18. Lavergne, Pascal & Patilea, Valentin, 2008. "Breaking the curse of dimensionality in nonparametric testing," Journal of Econometrics, Elsevier, vol. 143(1), pages 103-122, March.
    19. Dai, Sheng & Zhou, Xun & Kuosmanen, Timo, 2020. "Forward-looking assessment of the GHG abatement cost: Application to China," Energy Economics, Elsevier, vol. 88(C).
    20. Tsionas, Mike G., 2022. "Convex non-parametric least squares, causal structures and productivity," European Journal of Operational Research, Elsevier, vol. 303(1), pages 370-387.
    21. Homburg, Carsten, 2001. "Using data envelopment analysis to benchmark activities," International Journal of Production Economics, Elsevier, vol. 73(1), pages 51-58, August.
    22. Bodin Singpai & Desheng Wu, 2020. "Using a DEA–AutoML Approach to Track SDG Achievements," Sustainability, MDPI, vol. 12(23), pages 1-26, December.
    23. Koenker, Roger W & Bassett, Gilbert, Jr, 1978. "Regression Quantiles," Econometrica, Econometric Society, vol. 46(1), pages 33-50, January.
    24. Chen, Ya & Tsionas, Mike G. & Zelenyuk, Valentin, 2021. "LASSO+DEA for small and big wide data," Omega, Elsevier, vol. 102(C).
    25. Evropi‐Sofia Dalampira & Stefanos A. Nastis, 2020. "Mapping Sustainable Development Goals: A network analysis framework," Sustainable Development, John Wiley & Sons, Ltd., vol. 28(1), pages 46-55, January.
    26. Timo Kuosmanen & Andrew L. Johnson, 2010. "Data Envelopment Analysis as Nonparametric Least-Squares Regression," Operations Research, INFORMS, vol. 58(1), pages 149-160, February.
    27. Wagner, Janet M. & Shimshak, Daniel G., 2007. "Stepwise selection of variables in data envelopment analysis: Procedures and managerial perspectives," European Journal of Operational Research, Elsevier, vol. 180(1), pages 57-67, July.
    28. Wang, Yongqiao & Wang, Shouyang & Dang, Chuangyin & Ge, Wenxiu, 2014. "Nonparametric quantile frontier estimation under shape restriction," European Journal of Operational Research, Elsevier, vol. 232(3), pages 671-678.
    29. Yongjun Li & Xiao Shi & Min Yang & Liang Liang, 2017. "Variable selection in data envelopment analysis via Akaike’s information criteria," Annals of Operations Research, Springer, vol. 253(1), pages 453-476, June.
    30. Dyson, R. G. & Allen, R. & Camanho, A. S. & Podinovski, V. V. & Sarrico, C. S. & Shale, E. A., 2001. "Pitfalls and protocols in DEA," European Journal of Operational Research, Elsevier, vol. 132(2), pages 245-259, July.
    31. Kuosmanen, Timo & Zhou, Xun, 2021. "Shadow prices and marginal abatement costs: Convex quantile regression approach," European Journal of Operational Research, Elsevier, vol. 289(2), pages 666-675.
    32. Xi Chen & Qihang Lin & Bodhisattva Sen, 2020. "On Degrees of Freedom of Projection Estimators With Applications to Multivariate Nonparametric Regression," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 115(529), pages 173-186, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Fu, Saiji & Tian, Yingjie & Tang, Long, 2023. "Robust regression under the general framework of bounded loss functions," European Journal of Operational Research, Elsevier, vol. 310(3), pages 1325-1339.
    2. Dai, Sheng & Kuosmanen, Timo & Zhou, Xun, 2023. "Non-crossing convex quantile regression," Economics Letters, Elsevier, vol. 233(C).
    3. Dai, Sheng & Kuosmanen, Timo & Zhou, Xun, 2023. "Generalized quantile and expectile properties for shape constrained nonparametric estimation," European Journal of Operational Research, Elsevier, vol. 310(2), pages 914-927.
    4. Zhiqiang Liao, 2024. "Variable selection in convex nonparametric least squares via structured Lasso: An application to the Swedish electricity distribution networks," Papers 2409.01911, arXiv.org, revised Nov 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dai, Sheng & Kuosmanen, Timo & Zhou, Xun, 2023. "Generalized quantile and expectile properties for shape constrained nonparametric estimation," European Journal of Operational Research, Elsevier, vol. 310(2), pages 914-927.
    2. Liao, Zhiqiang & Dai, Sheng & Kuosmanen, Timo, 2024. "Convex support vector regression," European Journal of Operational Research, Elsevier, vol. 313(3), pages 858-870.
    3. Lee, Chia-Yen & Cai, Jia-Ying, 2020. "LASSO variable selection in data envelopment analysis with small datasets," Omega, Elsevier, vol. 91(C).
    4. Wen, Xiaojie & Yao, Shunbo & Sauer, Johannes, 2022. "Shadow prices and abatement cost of soil erosion in Shaanxi Province, China: Convex expectile regression approach," Ecological Economics, Elsevier, vol. 201(C).
    5. Esteve, Miriam & Aparicio, Juan & Rodriguez-Sala, Jesus J. & Zhu, Joe, 2023. "Random Forests and the measurement of super-efficiency in the context of Free Disposal Hull," European Journal of Operational Research, Elsevier, vol. 304(2), pages 729-744.
    6. Kuosmanen, Timo & Zhou, Xun, 2021. "Shadow prices and marginal abatement costs: Convex quantile regression approach," European Journal of Operational Research, Elsevier, vol. 289(2), pages 666-675.
    7. Raul Moragues & Juan Aparicio & Miriam Esteve, 2023. "Ranking the Importance of Variables in a Nonparametric Frontier Analysis Using Unsupervised Machine Learning Techniques," Mathematics, MDPI, vol. 11(11), pages 1-24, June.
    8. Imad Bou-Hamad & Abdel Latef Anouze & Ibrahim H. Osman, 2022. "A cognitive analytics management framework to select input and output variables for data envelopment analysis modeling of performance efficiency of banks using random forest and entropy of information," Annals of Operations Research, Springer, vol. 308(1), pages 63-92, January.
    9. Anna Łozowicka & Bartłomiej Lach, 2022. "CI-DEA: A Way to Improve the Discriminatory Power of DEA—Using the Example of the Efficiency Assessment of the Digitalization in the Life of the Generation 50+," Sustainability, MDPI, vol. 14(6), pages 1-22, March.
    10. Villanueva-Cantillo, Jeyms & Munoz-Marquez, Manuel, 2021. "Methodology for calculating critical values of relevance measures in variable selection methods in data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 290(2), pages 657-670.
    11. Peyrache, Antonio & Rose, Christiern & Sicilia, Gabriela, 2020. "Variable selection in Data Envelopment Analysis," European Journal of Operational Research, Elsevier, vol. 282(2), pages 644-659.
    12. K. Hervé Dakpo & Yann Desjeux & Laure Latruffe, 2023. "Cost of abating excess nitrogen on wheat plots in France: An assessment with multi‐technology modelling," Journal of Agricultural Economics, Wiley Blackwell, vol. 74(3), pages 800-815, September.
    13. Shirong Zhao & Guangshun Qiao, 2022. "The shadow prices of CO2, SO2 and NOx for U.S. coal power industry 2010–2017: a convex quantile regression method," Journal of Productivity Analysis, Springer, vol. 57(3), pages 243-253, June.
    14. Zhiqiang Liao, 2024. "Variable selection in convex nonparametric least squares via structured Lasso: An application to the Swedish electricity distribution networks," Papers 2409.01911, arXiv.org, revised Nov 2024.
    15. Sheng Dai & Natalia Kuosmanen & Timo Kuosmanen & Juuso Liesio, 2023. "Optimal resource allocation: Convex quantile regression approach," Papers 2311.06590, arXiv.org.
    16. Benítez-Peña, Sandra & Bogetoft, Peter & Romero Morales, Dolores, 2020. "Feature Selection in Data Envelopment Analysis: A Mathematical Optimization approach," Omega, Elsevier, vol. 96(C).
    17. Chen, Ya & Tsionas, Mike G. & Zelenyuk, Valentin, 2021. "LASSO+DEA for small and big wide data," Omega, Elsevier, vol. 102(C).
    18. Ya Chen & Mike Tsionas & Valentin Zelenyuk, 2020. "LASSO DEA for small and big data," CEPA Working Papers Series WP092020, School of Economics, University of Queensland, Australia.
    19. Duras, Toni & Javed, Farrukh & Månsson, Kristofer & Sjölander, Pär & Söderberg, Magnus, 2023. "Using machine learning to select variables in data envelopment analysis: Simulations and application using electricity distribution data," Energy Economics, Elsevier, vol. 120(C).
    20. Wang, Yongqiao & Wang, Shouyang & Dang, Chuangyin & Ge, Wenxiu, 2014. "Nonparametric quantile frontier estimation under shape restriction," European Journal of Operational Research, Elsevier, vol. 232(3), pages 671-678.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:305:y:2023:i:1:p:338-355. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.