IDEAS home Printed from https://ideas.repec.org/a/bla/biomet/v77y2021i3p996-1010.html
   My bibliography  Save this article

A weak‐signal‐assisted procedure for variable selection and statistical inference with an informative subsample

Author

Listed:
  • Fang Fang
  • Jiwei Zhao
  • S. Ejaz Ahmed
  • Annie Qu

Abstract

This paper is motivated from an HIV‐1 drug resistance study where we encounter three analytical challenges: to analyze data with an informative subsample, to take into account the weak signals, and to detect important signals and also conduct statistical inference. We start with an initial estimation method, which adopts a penalized pairwise conditional likelihood approach for variable selection. This initial estimator incorporates the informative subsample issue. To accounting for the effect of weak signals, we use a key idea of partial ridge regression. We also propose a one‐step estimation method for each of the signal coefficients and then construct confidence intervals accordingly. We apply the proposed method to the Stanford HIV‐1 drug resistance study and compare the results with existing approaches. We also conduct comprehensive simulation studies to demonstrate the superior performance of our proposed method.

Suggested Citation

  • Fang Fang & Jiwei Zhao & S. Ejaz Ahmed & Annie Qu, 2021. "A weak‐signal‐assisted procedure for variable selection and statistical inference with an informative subsample," Biometrics, The International Biometric Society, vol. 77(3), pages 996-1010, September.
  • Handle: RePEc:bla:biomet:v:77:y:2021:i:3:p:996-1010
    DOI: 10.1111/biom.13346
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/biom.13346
    Download Restriction: no

    File URL: https://libkey.io/10.1111/biom.13346?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Bradley Efron, 2014. "Estimation and Accuracy After Model Selection," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(507), pages 991-1007, September.
    2. Xiaoli Gao & S. E. Ahmed & Yang Feng, 2017. "Post selection shrinkage estimation for high‐dimensional data analysis," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 33(2), pages 97-120, March.
    3. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    4. Cun-Hui Zhang & Stephanie S. Zhang, 2014. "Confidence intervals for low dimensional parameters in high dimensional linear models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 76(1), pages 217-242, January.
    5. Kung‐Yee Liang & Jing Qin, 2000. "Regression analysis under non‐standard situations: a pairwise pseudolikelihood approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 62(4), pages 773-786.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Liu, Yu & Zhuang, Xiaoyang, 2023. "Shrinkage estimation of semi-parametric spatial autoregressive panel data model with fixed effects," Statistics & Probability Letters, Elsevier, vol. 194(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-dimensional econometrics and regularized GMM," CeMMAP working papers CWP35/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    2. Haixiang Zhang & Jian Huang & Liuquan Sun, 2022. "Projection‐based and cross‐validated estimation in high‐dimensional Cox model," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 49(1), pages 353-372, March.
    3. Chenchuan (Mark) Li & Ulrich K. Müller, 2021. "Linear regression with many controls of limited explanatory power," Quantitative Economics, Econometric Society, vol. 12(2), pages 405-442, May.
    4. Hansen, Christian & Liao, Yuan, 2019. "The Factor-Lasso And K-Step Bootstrap Approach For Inference In High-Dimensional Economic Applications," Econometric Theory, Cambridge University Press, vol. 35(3), pages 465-509, June.
    5. Maur,Jean-Christophe & Nedeljkovic,Milan & Von Uexkull,Jan Erik, 2022. "FDI and Trade Outcomes at the Industry Level—A Data-Driven Approach," Policy Research Working Paper Series 9901, The World Bank.
    6. Victor Chernozhukov & Christian Hansen & Martin Spindler, 2015. "Valid Post-Selection and Post-Regularization Inference: An Elementary, General Approach," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 649-688, August.
    7. Lu Tang & Peter X.‐K. Song, 2021. "Poststratification fusion learning in longitudinal data analysis," Biometrics, The International Biometric Society, vol. 77(3), pages 914-928, September.
    8. Victor Chernozhukov & Christian Hansen & Martin Spindler, 2015. "Post-Selection and Post-Regularization Inference in Linear Models with Many Controls and Instruments," American Economic Review, American Economic Association, vol. 105(5), pages 486-490, May.
    9. Guo, Xu & Li, Runze & Liu, Jingyuan & Zeng, Mudong, 2023. "Statistical inference for linear mediation models with high-dimensional mediators and application to studying stock reaction to COVID-19 pandemic," Journal of Econometrics, Elsevier, vol. 235(1), pages 166-179.
    10. Toshio Honda, 2021. "The de-biased group Lasso estimation for varying coefficient models," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 73(1), pages 3-29, February.
    11. Lu Xia & Bin Nan & Yi Li, 2023. "Debiased lasso for generalized linear models with a diverging number of covariates," Biometrics, The International Biometric Society, vol. 79(1), pages 344-357, March.
    12. Huang, Yuan & Li, Changcheng & Li, Runze & Yang, Songshan, 2022. "An overview of tests on high-dimensional means," Journal of Multivariate Analysis, Elsevier, vol. 188(C).
    13. Susan M. Shortreed & Ashkan Ertefaie, 2017. "Outcome‐adaptive lasso: Variable selection for causal inference," Biometrics, The International Biometric Society, vol. 73(4), pages 1111-1122, December.
    14. Gueuning, Thomas & Claeskens, Gerda, 2016. "Confidence intervals for high-dimensional partially linear single-index models," Journal of Multivariate Analysis, Elsevier, vol. 149(C), pages 13-29.
    15. Jingxuan Luo & Lili Yue & Gaorong Li, 2023. "Overview of High-Dimensional Measurement Error Regression Models," Mathematics, MDPI, vol. 11(14), pages 1-22, July.
    16. Wang, Yining & Wang, Jialei & Balakrishnan, Sivaraman & Singh, Aarti, 2019. "Rate optimal estimation and confidence intervals for high-dimensional regression with missing covariates," Journal of Multivariate Analysis, Elsevier, vol. 174(C).
    17. Farrell, Max H., 2015. "Robust inference on average treatment effects with possibly more covariates than observations," Journal of Econometrics, Elsevier, vol. 189(1), pages 1-23.
    18. Lan, Wei & Zhong, Ping-Shou & Li, Runze & Wang, Hansheng & Tsai, Chih-Ling, 2016. "Testing a single regression coefficient in high dimensional linear models," Journal of Econometrics, Elsevier, vol. 195(1), pages 154-168.
    19. Cai, Xizhen & Zhu, Yeying & Huang, Yuan & Ghosh, Debashis, 2022. "High-dimensional causal mediation analysis based on partial linear structural equation models," Computational Statistics & Data Analysis, Elsevier, vol. 174(C).
    20. Panxu Yuan & Xiao Guo, 2022. "High-dimensional inference for linear model with correlated errors," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 85(1), pages 21-52, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:biomet:v:77:y:2021:i:3:p:996-1010. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0006-341X .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.