IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v54y2010i12p3181-3193.html
   My bibliography  Save this article

A diagnostic method for simultaneous feature selection and outlier identification in linear regression

Author

Listed:
  • Menjoge, Rajiv S.
  • Welsch, Roy E.

Abstract

A diagnostic method along the lines of forward search is proposed to simultaneously study the effect of individual observations and features on the inferences made in linear regression. The method operates by appending dummy variables to the data matrix and performing backward selection on the augmented matrix. It outputs sequences of feature-outlier combinations which can be evaluated by plots similar to those of forward search and includes the capacity to incorporate prior knowledge, in order to mitigate issues such as collinearity. It also allows for alternative ways to understand the selection of the final model. The method is evaluated on five data sets and yields promising results.

Suggested Citation

  • Menjoge, Rajiv S. & Welsch, Roy E., 2010. "A diagnostic method for simultaneous feature selection and outlier identification in linear regression," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3181-3193, December.
    • Handle: RePEc:eee:csdana:v:54:y:2010:i:12:p:3181-3193
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(10)00079-4
    Download Restriction: Full text for ScienceDirect subscribers only.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Sung-Soo Kim & Sung Park & W. J. Krzanowski, 2008. "Simultaneous variable selection and outlier identification in linear regression using the mean-shift outlier model," Journal of Applied Statistics, Taylor & Francis Journals, vol. 35(3), pages 283-291.
    2. Anthony C. Atkinson, 2002. "Forward search added-variable t-tests and the effect of masked outliers on model selection," Biometrika, Biometrika Trust, vol. 89(4), pages 939-946, December.
    3. McCann, Lauren & Welsch, Roy E., 2007. "Robust variable selection using least angle regression and elemental set sampling," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 249-257, September.
    4. Ronchetti, Elvezio, 1985. "Robust model selection in regression," Statistics & Probability Letters, Elsevier, vol. 3(1), pages 21-23, February.
    5. Marco Riani & Anthony C. Atkinson & Andrea Cerioli, 2009. "Finding an unknown number of multivariate outliers," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 71(2), pages 447-466, April.
    6. Sung-Soo Kim & W. Krzanowski, 2007. "Detecting multiple outliers in linear regression using a cluster method combined with graphical visualization," Computational Statistics, Springer, vol. 22(1), pages 109-119, April.
    7. Hoeting, Jennifer & Raftery, Adrian E. & Madigan, David, 1996. "A method for simultaneous variable selection and outlier identification in linear regression," Computational Statistics & Data Analysis, Elsevier, vol. 22(3), pages 251-270, July.
    8. Khan, Jafar A. & Van Aelst, Stefan & Zamar, Ruben H., 2007. "Robust Linear Model Selection Based on Least Angle Regression," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 1289-1299, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Luca Insolia & Ana Kenney & Martina Calovi & Francesca Chiaromonte, 2021. "Robust Variable Selection with Optimality Guarantees for High-Dimensional Logistic Regression," Stats, MDPI, vol. 4(3), pages 1-17, August.
    2. Sue-Fen Huang & Ching-Hsue Cheng, 2013. "GMADM-based attributes selection method in developing prediction model," Quality & Quantity: International Journal of Methodology, Springer, vol. 47(6), pages 3335-3347, October.
    3. A.A.M. Nurunnabi & Ali S. Hadi & A.H.M.R. Imon, 2014. "Procedures for the identification of multiple influential observations in linear regression," Journal of Applied Statistics, Taylor & Francis Journals, vol. 41(6), pages 1315-1331, June.
    4. Thompson, Ryan, 2022. "Robust subset selection," Computational Statistics & Data Analysis, Elsevier, vol. 169(C).
    5. Luca Insolia & Ana Kenney & Francesca Chiaromonte & Giovanni Felici, 2022. "Simultaneous feature selection and outlier detection with optimality guarantees," Biometrics, The International Biometric Society, vol. 78(4), pages 1592-1603, December.
    6. A.A.M. Nurunnabi & M. Nasser & A.H.M.R. Imon, 2016. "Identification and classification of multiple outliers, high leverage points and influential observations in linear regression," Journal of Applied Statistics, Taylor & Francis Journals, vol. 43(3), pages 509-525, March.
    7. Tianxiang Liu & Ting Kei Pong & Akiko Takeda, 2019. "A refined convergence analysis of $$\hbox {pDCA}_{e}$$ pDCA e with applications to simultaneous sparse recovery and outlier detection," Computational Optimization and Applications, Springer, vol. 73(1), pages 69-100, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Riani, Marco & Atkinson, Anthony C., 2010. "Robust model selection with flexible trimming," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3300-3312, December.
    2. Nikolas Kuschnig & Gregor Zens & Jesús Crespo Cuaresma, 2021. "Hidden in Plain Sight: Influential Sets in Linear Models," CESifo Working Paper Series 8981, CESifo.
    3. Sung-Soo Kim & Sung Park & W. J. Krzanowski, 2008. "Simultaneous variable selection and outlier identification in linear regression using the mean-shift outlier model," Journal of Applied Statistics, Taylor & Francis Journals, vol. 35(3), pages 283-291.
    4. Andreas Alfons & Wolfgang Baaske & Peter Filzmoser & Wolfgang Mader & Roland Wieser, 2011. "Robust variable selection with application to quality of life research," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 20(1), pages 65-82, March.
    5. Anthony C. Atkinson & Marco Riani & Aldo Corbellini, 2020. "The analysis of transformations for profit‐and‐loss data," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 69(2), pages 251-275, April.
    6. Domenico Perrotta & Marco Riani & Francesca Torti, 2009. "New robust dynamic plots for regression mixture detection," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 3(3), pages 263-279, December.
    7. Marc Aerts & Niel Hens & Jeffrey Simonoff, 2010. "Model selection in regression based on pre-smoothing," Journal of Applied Statistics, Taylor & Francis Journals, vol. 37(9), pages 1455-1472.
    8. Khan, Jafar A. & Van Aelst, Stefan & Zamar, Ruben H., 2010. "Fast robust estimation of prediction error based on resampling," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3121-3130, December.
    9. M. Habshah & M. R. Norazan & A.H.M. Rahmatullah Imon, 2009. "The performance of diagnostic-robust generalized potentials for the identification of multiple high leverage points in linear regression," Journal of Applied Statistics, Taylor & Francis Journals, vol. 36(5), pages 507-520.
    10. Cerioli, Andrea & Farcomeni, Alessio & Riani, Marco, 2014. "Strong consistency and robustness of the Forward Search estimator of multivariate location and scatter," Journal of Multivariate Analysis, Elsevier, vol. 126(C), pages 167-183.
    11. Alfons, Andreas & Croux, Christophe & Gelper, Sarah, 2016. "Robust groupwise least angle regression," Computational Statistics & Data Analysis, Elsevier, vol. 93(C), pages 421-435.
    12. Salibian-Barrera, Matias & Van Aelst, Stefan, 2008. "Robust model selection using fast and robust bootstrap," Computational Statistics & Data Analysis, Elsevier, vol. 52(12), pages 5121-5135, August.
    13. Thompson, Ryan, 2022. "Robust subset selection," Computational Statistics & Data Analysis, Elsevier, vol. 169(C).
    14. Umberto Amato & Anestis Antoniadis & Italia De Feis & Irene Gijbels, 2021. "Penalised robust estimators for sparse and high-dimensional linear models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(1), pages 1-48, March.
    15. Elbakry, Ashraf E. & Nwachukwu, Jacinta C. & Abdou, Hussein A. & Elshandidy, Tamer, 2017. "Comparative evidence on the value relevance of IFRS-based accounting information in Germany and the UK," Journal of International Accounting, Auditing and Taxation, Elsevier, vol. 28(C), pages 10-30.
    16. Diego Vidaurre & Concha Bielza & Pedro Larrañaga, 2012. "Lazy lasso for local regression," Computational Statistics, Springer, vol. 27(3), pages 531-550, September.
    17. Fernandez, Carmen & Ley, Eduardo & Steel, Mark F. J., 2001. "Benchmark priors for Bayesian model averaging," Journal of Econometrics, Elsevier, vol. 100(2), pages 381-427, February.
    18. Mark F. J. Steel, 2020. "Model Averaging and Its Use in Economics," Journal of Economic Literature, American Economic Association, vol. 58(3), pages 644-719, September.
    19. James C. Rockey, 2007. "Which Democracies Pay Higher Wages?," Bristol Economics Discussion Papers 07/600, School of Economics, University of Bristol, UK.
    20. Tri-Dzung Nguyen & Roy Welsch, 2010. "Outlier detection and robust covariance estimation using mathematical programming," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 4(4), pages 301-334, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:54:y:2010:i:12:p:3181-3193. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.