IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v49y2024i4p595-629.html
   My bibliography  Save this article

Identifying Informative Predictor Variables With Random Forests

Author

Listed:
  • Yannick Rothacher
  • Carolin Strobl

    (University of Zurich)

Abstract

Random forests are a nonparametric machine learning method, which is currently gaining popularity in the behavioral sciences. Despite random forests’ potential advantages over more conventional statistical methods, a remaining question is how reliably informative predictor variables can be identified by means of random forests. The present study aims at giving a comprehensible introduction to the topic of variable selection with random forests and providing an overview of the currently proposed selection methods. Using simulation studies, the variable selection methods are examined regarding their statistical properties, and comparisons between their performances and the performance of a conventional linear model are drawn. Advantages and disadvantages of the examined methods are discussed, and practical recommendations for the use of random forests for variable selection are given.

Suggested Citation

  • Yannick Rothacher & Carolin Strobl, 2024. "Identifying Informative Predictor Variables With Random Forests," Journal of Educational and Behavioral Statistics, , vol. 49(4), pages 595-629, August.
    • Handle: RePEc:sae:jedbes:v:49:y:2024:i:4:p:595-629
    DOI: 10.3102/10769986231193327
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/10769986231193327
    Download Restriction: no
    File URL: https://libkey.io/10.3102/10769986231193327?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Kursa, Miron B. & Rudnicki, Witold R., 2010. "Feature Selection with the Boruta Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 36(i11).
    2. Hapfelmeier, A. & Ulm, K., 2013. "A new variable selection approach using Random Forests," Computational Statistics & Data Analysis, Elsevier, vol. 60(C), pages 50-69.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Leogrande, Angelo, 2024. "Strategie innovative per la logistica: il valore del kitting e assembly nel settore idrotermosanitario," MPRA Paper 122746, University Library of Munich, Germany.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jin Li & Maggie Tran & Justy Siwabessy, 2016. "Selecting Optimal Random Forest Predictive Models: A Case Study on Predicting the Spatial Distribution of Seabed Hardness," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-29, February.
    2. Tong, Jianfeng & Liu, Zhenxing & Zhang, Yong & Zheng, Xiujuan & Jin, Junyang, 2023. "Improved multi-gate mixture-of-experts framework for multi-step prediction of gas load," Energy, Elsevier, vol. 282(C).
    3. Asma Shaheen & Javed Iqbal, 2018. "Spatial Distribution and Mobility Assessment of Carcinogenic Heavy Metals in Soil Profiles Using Geostatistics and Random Forest, Boruta Algorithm," Sustainability, MDPI, vol. 10(3), pages 1-20, March.
    4. Ramón Ferri-García & María del Mar Rueda, 2022. "Variable selection in Propensity Score Adjustment to mitigate selection bias in online surveys," Statistical Papers, Springer, vol. 63(6), pages 1829-1881, December.
    5. Weijun Wang & Dan Zhao & Liguo Fan & Yulong Jia, 2019. "Study on Icing Prediction of Power Transmission Lines Based on Ensemble Empirical Mode Decomposition and Feature Selection Optimized Extreme Learning Machine," Energies, MDPI, vol. 12(11), pages 1-21, June.
    6. Manuel J. García Rodríguez & Vicente Rodríguez Montequín & Francisco Ortega Fernández & Joaquín M. Villanueva Balsera, 2019. "Public Procurement Announcements in Spain: Regulations, Data Analysis, and Award Price Estimator Using Machine Learning," Complexity, Hindawi, vol. 2019, pages 1-20, November.
    7. Sangjin Kim & Jong-Min Kim, 2019. "Two-Stage Classification with SIS Using a New Filter Ranking Method in High Throughput Data," Mathematics, MDPI, vol. 7(6), pages 1-16, May.
    8. Foutzopoulos, Giorgos & Pandis, Nikolaos & Tsagris, Michail, 2024. "Predicting full retirement attainment of NBA players," MPRA Paper 121540, University Library of Munich, Germany.
    9. Zhao-Yue Chen & Hervé Petetin & Raúl Fernando Méndez Turrubiates & Hicham Achebak & Carlos Pérez García-Pando & Joan Ballester, 2024. "Population exposure to multiple air pollutants and its compound episodes in Europe," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    10. Schrader, Silja & Graham, Sonia & Campbell, Rebecca & Height, Kaitlyn & Hawkes, Gina, 2024. "Grower attitudes and practices toward area-wide management of cropping weeds in Australia," Land Use Policy, Elsevier, vol. 137(C).
    11. Piotr Pomorski & Denise Gorse, 2023. "Improving Portfolio Performance Using a Novel Method for Predicting Financial Regimes," Papers 2310.04536, arXiv.org.
    12. Caperna, Giulio & Colagrossi, Marco & Geraci, Andrea & Mazzarella, Gianluca, 2022. "A babel of web-searches: Googling unemployment during the pandemic," Labour Economics, Elsevier, vol. 74(C).
    13. Lkhagvadorj Munkhdalai & Tsendsuren Munkhdalai & Oyun-Erdene Namsrai & Jong Yun Lee & Keun Ho Ryu, 2019. "An Empirical Comparison of Machine-Learning Methods on Bank Client Credit Assessments," Sustainability, MDPI, vol. 11(3), pages 1-23, January.
    14. Hakan Pabuccu & Adrian Barbu, 2023. "Feature Selection with Annealing for Forecasting Financial Time Series," Papers 2303.02223, arXiv.org, revised Feb 2024.
    15. Abolfazl Mollalo & Kiara M. Rivera & Behzad Vahedi, 2020. "Artificial Neural Network Modeling of Novel Coronavirus (COVID-19) Incidence Rates across the Continental United States," IJERPH, MDPI, vol. 17(12), pages 1-13, June.
    16. Chunyang Huang & Shaoliang Zhang, 2023. "Explainable artificial intelligence model for identifying Market Value in Professional Soccer Players," Papers 2311.04599, arXiv.org, revised Nov 2023.
    17. Faisal Alsayegh & Moh A Alkhamis & Fatima Ali & Sreeja Attur & Nicholas M Fountain-Jones & Mohammad Zubaid, 2022. "Anemia or other comorbidities? using machine learning to reveal deeper insights into the drivers of acute coronary syndromes in hospital admitted patients," PLOS ONE, Public Library of Science, vol. 17(1), pages 1-15, January.
    18. Basso, Franco & Cox, Tomás & Pezoa, Raúl & Maldonado, Tomás & Varas, Mauricio, 2024. "Characterizing last-mile freight transportation using mobile phone data: The case of Santiago, Chile," Transportation Research Part A: Policy and Practice, Elsevier, vol. 186(C).
    19. Andrea Albergoni & Florentina J. Hettinga & Wim Stut & Francesco Sartor, 2020. "Factors Influencing Walking and Exercise Adherence in Healthy Older Adults Using Monitoring and Interfacing Technology: Preliminary Evidence," IJERPH, MDPI, vol. 17(17), pages 1-18, August.
    20. Franck Ramaharo & Fitiavana Randriamifidy, 2023. "Determinants of renewable energy consumption in Madagascar: Evidence from feature selection algorithms," Papers 2401.13671, arXiv.org.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:49:y:2024:i:4:p:595-629. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.