IDEAS home Printed from https://ideas.repec.org/a/hin/complx/2032987.html
   My bibliography  Save this article

A Two-Stage Regularization Method for Variable Selection and Forecasting in High-Order Interaction Model

Author

Listed:
  • Yao Dong
  • He Jiang

Abstract

Forecasting models with high-order interaction has become popular in many applications since researchers gradually notice that an additive linear model is not adequate for accurate forecasting. However, the excessive number of variables with low sample size in the model poses critically challenges to predication accuracy. To enhance the forecasting accuracy and training speed simultaneously, an interpretable model is essential in knowledge recovery. To deal with ultra-high dimensionality, this paper investigates and studies a two-stage procedure to demand sparsity within high-order interaction model. In each stage, square root hard ridge (SRHR) method is applied to discover the relevant variables. The application of square root loss function facilitates the parameter tuning work. On the other hand, hard ridge penalty function is able to handle both the high multicollinearity and selection inconsistency. The real data experiments reveal the superior performances to other comparing approaches.

Suggested Citation

  • Yao Dong & He Jiang, 2018. "A Two-Stage Regularization Method for Variable Selection and Forecasting in High-Order Interaction Model," Complexity, Hindawi, vol. 2018, pages 1-12, November.
    • Handle: RePEc:hin:complx:2032987
    DOI: 10.1155/2018/2032987
    as

    Download full text from publisher

    File URL: http://downloads.hindawi.com/journals/8503/2018/2032987.pdf
    Download Restriction: no
    File URL: http://downloads.hindawi.com/journals/8503/2018/2032987.xml
    Download Restriction: no
    File URL: https://libkey.io/10.1155/2018/2032987?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jiahua Chen & Zehua Chen, 2008. "Extended Bayesian information criteria for model selection with large model spaces," Biometrika, Biometrika Trust, vol. 95(3), pages 759-771.
    2. Radchenko, Peter & James, Gareth M., 2010. "Variable Selection Using Adaptive Nonlinear Interaction Structures in High Dimensions," Journal of the American Statistical Association, American Statistical Association, vol. 105(492), pages 1541-1553.
    3. NESTEROV, Yu., 2007. "Gradient methods for minimizing composite objective function," LIDAM Discussion Papers CORE 2007076, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    4. A. Belloni & V. Chernozhukov & L. Wang, 2011. "Square-root lasso: pivotal recovery of sparse signals via conic programming," Biometrika, Biometrika Trust, vol. 98(4), pages 791-806.
    5. A. Antoniadis, 1997. "Wavelets in statistics: A review," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 6(2), pages 97-130, August.
    6. Ye, Ya-Fen & Shao, Yuan-Hai & Deng, Nai-Yang & Li, Chun-Na & Hua, Xiang-Yu, 2017. "Robust Lp-norm least squares support vector regression with feature selection," Applied Mathematics and Computation, Elsevier, vol. 305(C), pages 32-52.
    7. Choi, Nam Hee & Li, William & Zhu, Ji, 2010. "Variable Selection With the Strong Heredity Constraint and Its Oracle Property," Journal of the American Statistical Association, American Statistical Association, vol. 105(489), pages 354-364.
    8. Yiyuan She & Zhifeng Wang & He Jiang, 2018. "Group Regularized Estimation Under Structural Hierarchy," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(521), pages 445-454, January.
    9. Ning Hao & Hao Helen Zhang, 2014. "Interaction Screening for Ultrahigh-Dimensional Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(507), pages 1285-1301, September.
    10. Friedman, Jerome H., 2012. "Fast sparse regression and classification," International Journal of Forecasting, Elsevier, vol. 28(3), pages 722-738.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wang, Cheng & Chen, Haozhe & Jiang, Binyan, 2024. "HiQR: An efficient algorithm for high-dimensional quadratic regression with penalties," Computational Statistics & Data Analysis, Elsevier, vol. 192(C).
    2. He Jiang, 2022. "A novel robust structural quadratic forecasting model and applications," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(6), pages 1156-1180, September.
    3. Gregor Stiglic & Petra Povalej Brzan & Nino Fijacko & Fei Wang & Boris Delibasic & Alexandros Kalousis & Zoran Obradovic, 2015. "Comprehensible Predictive Modeling Using Regularized Logistic Regression and Comorbidity Based Features," PLOS ONE, Public Library of Science, vol. 10(12), pages 1-11, December.
    4. Yawei He & Zehua Chen, 2016. "The EBIC and a sequential procedure for feature selection in interactive linear models with high-dimensional data," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 68(1), pages 155-180, February.
    5. Jiang, He & Luo, Shihua & Dong, Yao, 2021. "Simultaneous feature selection and clustering based on square root optimization," European Journal of Operational Research, Elsevier, vol. 289(1), pages 214-231.
    6. Ning Hao & Hao Helen Zhang, 2017. "A Note on High-Dimensional Linear Regression With Interactions," The American Statistician, Taylor & Francis Journals, vol. 71(4), pages 291-297, October.
    7. Feng Li & Yajie Li & Sanying Feng, 2021. "Estimation for Varying Coefficient Models with Hierarchical Structure," Mathematics, MDPI, vol. 9(2), pages 1-18, January.
    8. Sanying Feng & Menghan Zhang & Tiejun Tong, 2022. "Variable selection for functional linear models with strong heredity constraint," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 74(2), pages 321-339, April.
    9. Umberto Amato & Anestis Antoniadis & Italia De Feis & Irene Gijbels, 2021. "Penalised robust estimators for sparse and high-dimensional linear models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(1), pages 1-48, March.
    10. Loann David Denis Desboulets, 2018. "A Review on Variable Selection in Regression Analysis," Econometrics, MDPI, vol. 6(4), pages 1-27, November.
    11. Luke Mosley & Idris A. Eckley & Alex Gibberd, 2022. "Sparse temporal disaggregation," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(4), pages 2203-2233, October.
    12. Ryan A. Peterson & Joseph E. Cavanaugh, 2022. "Ranked sparsity: a cogent regularization framework for selecting and estimating feature interactions and polynomials," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 106(3), pages 427-454, September.
    13. Alain Hecq & Luca Margaritella & Stephan Smeekes, 2023. "Granger Causality Testing in High-Dimensional VARs: A Post-Double-Selection Procedure," Journal of Financial Econometrics, Oxford University Press, vol. 21(3), pages 915-958.
    14. Byron Botha & Rulof Burger & Kevin Kotzé & Neil Rankin & Daan Steenkamp, 2023. "Big data forecasting of South African inflation," Empirical Economics, Springer, vol. 65(1), pages 149-188, July.
    15. Ting‐Huei Chen & Hanaa Boughal, 2021. "A penalized structural equation modeling method accounting for secondary phenotypes for variable selection on genetically regulated expression from PrediXcan for Alzheimer's disease," Biometrics, The International Biometric Society, vol. 77(1), pages 362-371, March.
    16. Achim Ahrens & Christian B. Hansen & Mark E. Schaffer, 2020. "lassopack: Model selection and prediction with regularized regression in Stata," Stata Journal, StataCorp LLC, vol. 20(1), pages 176-235, March.
    17. Kaixu Yang & Tapabrata Maiti, 2022. "Ultrahigh‐dimensional generalized additive model: Unified theory and methods," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 49(3), pages 917-942, September.
    18. Wang, Lu & Shen, Jincheng & Thall, Peter F., 2014. "A modified adaptive Lasso for identifying interactions in the Cox model with the heredity constraint," Statistics & Probability Letters, Elsevier, vol. 93(C), pages 126-133.
    19. Pun, Chi Seng & Hadimaja, Matthew Zakharia, 2021. "A self-calibrated direct approach to precision matrix estimation and linear discriminant analysis in high dimensions," Computational Statistics & Data Analysis, Elsevier, vol. 155(C).
    20. Radchenko, Peter, 2015. "High dimensional single index models," Journal of Multivariate Analysis, Elsevier, vol. 139(C), pages 266-282.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hin:complx:2032987. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Mohamed Abdelhakeem (email available below). General contact details of provider: https://www.hindawi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.