IDEAS home Printed from https://ideas.repec.org/a/spr/stpapr/v65y2024i9d10.1007_s00362-024-01615-4.html
   My bibliography  Save this article

Model-free feature screening based on Hellinger distance for ultrahigh dimensional data

Author

Listed:
  • Jiujing Wu

    (Capital Normal University)

  • Hengjian Cui

    (Capital Normal University)

Abstract

With the explosive development of data acquisition and processing technology, feature dimensions increase exponentially with sample size, posing significant challenges for data analysis. It is crucial to accurately identify useful features from thousands available. In this paper, we develop an omnibus model-free feature screening procedure based on the Hellinger distance, offering several appealing merits. First, we define the Hellinger distance index for discrete response variables in discriminant analysis. Second, this procedure consistently works for continuous response variables, where the responses are discretized using a slice-and-fused technique. Third, it is robust against potential outliers and model misspecification. Theoretically, the procedure for both discrete and continuous response variables exhibits sure screening and ranking consistency properties under mild conditions. Numerical studies show that this procedure is highly competitive in heavy-tailed and skewed data, as well as maintaining comparability with existing approaches for light-tailed data, indicating robust performance across various data types. The real data sets, one with discrete and the other with continuous response variables demonstrate the effectiveness of the proposed method.

Suggested Citation

  • Jiujing Wu & Hengjian Cui, 2024. "Model-free feature screening based on Hellinger distance for ultrahigh dimensional data," Statistical Papers, Springer, vol. 65(9), pages 5903-5930, December.
  • Handle: RePEc:spr:stpapr:v:65:y:2024:i:9:d:10.1007_s00362-024-01615-4
    DOI: 10.1007/s00362-024-01615-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00362-024-01615-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s00362-024-01615-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hengjian Cui & Runze Li & Wei Zhong, 2015. "Model-Free Feature Screening for Ultrahigh Dimensional Discriminant Analysis," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(510), pages 630-641, June.
    2. Runze Li & Wei Zhong & Liping Zhu, 2012. "Feature Screening via Distance Correlation Learning," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(499), pages 1129-1139, September.
    3. He, Shengmei & Ma, Shuangge & Xu, Wangli, 2019. "A modified mean-variance feature-screening procedure for ultrahigh-dimensional discriminant analysis," Computational Statistics & Data Analysis, Elsevier, vol. 137(C), pages 155-169.
    4. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    5. Jinhan Xie & Yuanyuan Lin & Xiaodong Yan & Niansheng Tang, 2020. "Category-Adaptive Variable Screening for Ultra-High Dimensional Heterogeneous Categorical Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 115(530), pages 747-760, April.
    6. N. Neykov & P. Filzmoser & P. Neytchev, 2014. "Erratum to: Ultrahigh dimensional variable selection through the penalized maximum trimmed likelihood estimator," Statistical Papers, Springer, vol. 55(3), pages 917-918, August.
    7. Qing Mai & Hui Zou, 2013. "The Kolmogorov filter for variable screening in high-dimensional binary classification," Biometrika, Biometrika Trust, vol. 100(1), pages 229-234.
    8. Jun Lu & Lu Lin, 2020. "Model-free conditional screening via conditional distance correlation," Statistical Papers, Springer, vol. 61(1), pages 225-244, February.
    9. Rui Pan & Hansheng Wang & Runze Li, 2016. "Ultrahigh-Dimensional Multiclass Linear Discriminant Analysis by Pairwise Sure Independence Screening," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(513), pages 169-179, March.
    10. Cui, Hengjian & Zhong, Wei, 2019. "A distribution-free test of independence based on mean variance index," Computational Statistics & Data Analysis, Elsevier, vol. 139(C), pages 117-133.
    11. Fan, Jianqing & Feng, Yang & Song, Rui, 2011. "Nonparametric Independence Screening in Sparse Ultra-High-Dimensional Additive Models," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 544-557.
    12. Yan, Xiaodong & Tang, Niansheng & Xie, Jinhan & Ding, Xianwen & Wang, Zhiqiang, 2018. "Fused mean–variance filter for feature screening," Computational Statistics & Data Analysis, Elsevier, vol. 122(C), pages 18-32.
    13. N. Neykov & P. Filzmoser & P. Neytchev, 2014. "Ultrahigh dimensional variable selection through the penalized maximum trimmed likelihood estimator," Statistical Papers, Springer, vol. 55(1), pages 187-207, February.
    14. Jing Zhang & Yanyan Liu & Hengjian Cui, 2021. "Model-free feature screening via distance correlation for ultrahigh dimensional survival data," Statistical Papers, Springer, vol. 62(6), pages 2711-2738, December.
    15. Tingyou Zhou & Liping Zhu & Chen Xu & Runze Li, 2020. "Model-Free Forward Screening Via Cumulative Divergence," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 115(531), pages 1393-1405, July.
    16. Danyang Huang & Runze Li & Hansheng Wang, 2014. "Feature Screening for Ultrahigh Dimensional Categorical Data With Applications," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 32(2), pages 237-244, April.
    17. Jianqing Fan & Jinchi Lv, 2008. "Sure independence screening for ultrahigh dimensional feature space," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(5), pages 849-911, November.
    18. Zhong, Wei & Wang, Jiping & Chen, Xiaolin, 2021. "Censored mean variance sure independence screening for ultrahigh dimensional survival data," Computational Statistics & Data Analysis, Elsevier, vol. 159(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yuan, Qingcong & Chen, Xianyan & Ke, Chenlu & Yin, Xiangrong, 2022. "Independence index sufficient variable screening for categorical responses," Computational Statistics & Data Analysis, Elsevier, vol. 174(C).
    2. He, Shengmei & Ma, Shuangge & Xu, Wangli, 2019. "A modified mean-variance feature-screening procedure for ultrahigh-dimensional discriminant analysis," Computational Statistics & Data Analysis, Elsevier, vol. 137(C), pages 155-169.
    3. Sheng, Ying & Wang, Qihua, 2020. "Model-free feature screening for ultrahigh dimensional classification," Journal of Multivariate Analysis, Elsevier, vol. 178(C).
    4. Dong, Yuexiao & Yu, Zhou & Zhu, Liping, 2020. "Model-free variable selection for conditional mean in regression," Computational Statistics & Data Analysis, Elsevier, vol. 152(C).
    5. Guo, Chaohui & Lv, Jing & Wu, Jibo, 2021. "Composite quantile regression for ultra-high dimensional semiparametric model averaging," Computational Statistics & Data Analysis, Elsevier, vol. 160(C).
    6. Yang, Baoying & Yin, Xiangrong & Zhang, Nan, 2019. "Sufficient variable selection using independence measures for continuous response," Journal of Multivariate Analysis, Elsevier, vol. 173(C), pages 480-493.
    7. Fengli Song & Peng Lai & Baohua Shen, 2020. "Robust composite weighted quantile screening for ultrahigh dimensional discriminant analysis," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 83(7), pages 799-820, October.
    8. Shuaishuai Chen & Jun Lu, 2023. "Quantile-Composited Feature Screening for Ultrahigh-Dimensional Data," Mathematics, MDPI, vol. 11(10), pages 1-21, May.
    9. Lai, Peng & Liu, Yiming & Liu, Zhi & Wan, Yi, 2017. "Model free feature screening for ultrahigh dimensional data with responses missing at random," Computational Statistics & Data Analysis, Elsevier, vol. 105(C), pages 201-216.
    10. Zhang, Jing & Wang, Qihua & Kang, Jian, 2020. "Feature screening under missing indicator imputation with non-ignorable missing response," Computational Statistics & Data Analysis, Elsevier, vol. 149(C).
    11. Xiaochao Xia & Hao Ming, 2022. "A Flexibly Conditional Screening Approach via a Nonparametric Quantile Partial Correlation," Mathematics, MDPI, vol. 10(24), pages 1-32, December.
    12. He, Yong & Zhang, Liang & Ji, Jiadong & Zhang, Xinsheng, 2019. "Robust feature screening for elliptical copula regression model," Journal of Multivariate Analysis, Elsevier, vol. 173(C), pages 568-582.
    13. Tang, Niansheng & Xia, Linli & Yan, Xiaodong, 2019. "Feature screening in ultrahigh-dimensional partially linear models with missing responses at random," Computational Statistics & Data Analysis, Elsevier, vol. 133(C), pages 208-227.
    14. Liming Wang & Xingxiang Li & Xiaoqing Wang & Peng Lai, 2022. "Unified mean-variance feature screening for ultrahigh-dimensional regression," Computational Statistics, Springer, vol. 37(4), pages 1887-1918, September.
    15. Ping Wang & Lu Lin, 2023. "Conditional characteristic feature screening for massive imbalanced data," Statistical Papers, Springer, vol. 64(3), pages 807-834, June.
    16. Baiguo An & Guozhong Feng & Jianhua Guo, 2022. "Interaction Identification and Clique Screening for Classification with Ultra-high Dimensional Discrete Features," Journal of Classification, Springer;The Classification Society, vol. 39(1), pages 122-146, March.
    17. Lyu Ni & Fang Fang & Fangjiao Wan, 2017. "Adjusted Pearson Chi-Square feature screening for multi-classification with ultrahigh dimensional data," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 80(6), pages 805-828, November.
    18. Sweata Sen & Damitri Kundu & Kiranmoy Das, 2023. "Variable selection for categorical response: a comparative study," Computational Statistics, Springer, vol. 38(2), pages 809-826, June.
    19. Lu, Jun & Lin, Lu & Wang, WenWu, 2021. "Partition-based feature screening for categorical data via RKHS embeddings," Computational Statistics & Data Analysis, Elsevier, vol. 157(C).
    20. Adriano Zanin Zambom & Gregory J. Matthews, 2021. "Sure independence screening in the presence of missing data," Statistical Papers, Springer, vol. 62(2), pages 817-845, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stpapr:v:65:y:2024:i:9:d:10.1007_s00362-024-01615-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.