IDEAS home Printed from https://ideas.repec.org/a/spr/compst/v37y2022i4d10.1007_s00180-021-01182-4.html
   My bibliography  Save this article

Models under which random forests perform badly; consequences for applications

Author

Listed:
  • José A. Ferreira

    (National Institute for Public Health and the Environment (RIVM))

Abstract

We give examples of data-generating models under which Breiman’s random forest may be extremely slow to converge to the optimal predictor or even fail to be consistent. The evidence provided for these properties is based on mostly intuitive arguments, similar to those used earlier with simpler examples, and on numerical experiments. Although one can always choose models under which random forests perform very badly, we show that simple methods based on statistics of ‘variable use’ and ‘variable importance’ can often be used to construct a much better predictor based on a ‘many-armed’ random forest obtained by forcing initial splits on variables which the default version of the algorithm tends to ignore.

Suggested Citation

  • José A. Ferreira, 2022. "Models under which random forests perform badly; consequences for applications," Computational Statistics, Springer, vol. 37(4), pages 1839-1854, September.
    • Handle: RePEc:spr:compst:v:37:y:2022:i:4:d:10.1007_s00180-021-01182-4
    DOI: 10.1007/s00180-021-01182-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00180-021-01182-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00180-021-01182-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ruoqing Zhu & Donglin Zeng & Michael R. Kosorok, 2015. "Reinforcement Learning Trees," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1770-1784, December.
    2. Gérard Biau & Erwan Scornet, 2016. "A random forest guided tour," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(2), pages 197-227, June.
    3. Gérard Biau & Erwan Scornet, 2016. "Rejoinder on: A random forest guided tour," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(2), pages 264-268, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yiyi Huo & Yingying Fan & Fang Han, 2023. "On the adaptation of causal forests to manifold data," Papers 2311.16486, arXiv.org, revised Dec 2023.
    2. Susan Athey & Julie Tibshirani & Stefan Wager, 2016. "Generalized Random Forests," Papers 1610.01271, arXiv.org, revised Apr 2018.
    3. Hou, Lei & Elsworth, Derek & Zhang, Fengshou & Wang, Zhiyuan & Zhang, Jianbo, 2023. "Evaluation of proppant injection based on a data-driven approach integrating numerical and ensemble learning models," Energy, Elsevier, vol. 264(C).
    4. Ma, Zhikai & Huo, Qian & Wang, Wei & Zhang, Tao, 2023. "Voltage-temperature aware thermal runaway alarming framework for electric vehicles via deep learning with attention mechanism in time-frequency domain," Energy, Elsevier, vol. 278(C).
    5. Patrick Krennmair & Timo Schmid, 2022. "Flexible domain prediction using mixed effects random forests," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(5), pages 1865-1894, November.
    6. Jie Shi & Arno P. J. M. Siebes & Siamak Mehrkanoon, 2023. "TransCORALNet: A Two-Stream Transformer CORAL Networks for Supply Chain Credit Assessment Cold Start," Papers 2311.18749, arXiv.org.
    7. Bourdouxhe, Axel & Wibail, Lionel & Claessens, Hugues & Dufrêne, Marc, 2023. "Modeling potential natural vegetation: A new light on an old concept to guide nature conservation in fragmented and degraded landscapes," Ecological Modelling, Elsevier, vol. 481(C).
    8. Manuel J. García Rodríguez & Vicente Rodríguez Montequín & Francisco Ortega Fernández & Joaquín M. Villanueva Balsera, 2019. "Public Procurement Announcements in Spain: Regulations, Data Analysis, and Award Price Estimator Using Machine Learning," Complexity, Hindawi, vol. 2019, pages 1-20, November.
    9. Borup, Daniel & Christensen, Bent Jesper & Mühlbach, Nicolaj Søndergaard & Nielsen, Mikkel Slot, 2023. "Targeting predictors in random forest regression," International Journal of Forecasting, Elsevier, vol. 39(2), pages 841-868.
    10. Akshita Bassi & Aditya Manchanda & Rajwinder Singh & Mahesh Patel, 2023. "A comparative study of machine learning algorithms for the prediction of compressive strength of rice husk ash-based concrete," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 118(1), pages 209-238, August.
    11. Sachin Kumar & Zairu Nisha & Jagvinder Singh & Anuj Kumar Sharma, 2022. "Sensor network driven novel hybrid model based on feature selection and SVR to predict indoor temperature for energy consumption optimisation in smart buildings," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 13(6), pages 3048-3061, December.
    12. Yong-Chao Su & Cheng-Yu Wu & Cheng-Hong Yang & Bo-Sheng Li & Sin-Hua Moi & Yu-Da Lin, 2021. "Machine Learning Data Imputation and Prediction of Foraging Group Size in a Kleptoparasitic Spider," Mathematics, MDPI, vol. 9(4), pages 1-16, February.
    13. Diogenis A. Kiziridis & Anna Mastrogianni & Magdalini Pleniou & Elpida Karadimou & Spyros Tsiftsis & Fotios Xystrakis & Ioannis Tsiripidis, 2022. "Acceleration and Relocation of Abandonment in a Mediterranean Mountainous Landscape: Drivers, Consequences, and Management Implications," Land, MDPI, vol. 11(3), pages 1-23, March.
    14. Escribano, Álvaro & Wang, Dandan, 2021. "Mixed random forest, cointegration, and forecasting gasoline prices," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1442-1462.
    15. Hunish Bansal & Basavraj Chinagundi & Prashant Singh Rana & Neeraj Kumar, 2022. "An Ensemble Machine Learning Technique for Detection of Abnormalities in Knee Movement Sustainability," Sustainability, MDPI, vol. 14(20), pages 1-16, October.
    16. Yigit Aydede & Jan Ditzen, 2022. "Identifying the regional drivers of influenza-like illness in Nova Scotia with dominance analysis," Papers 2212.06684, arXiv.org.
    17. Siyoon Kwon & Hyoseob Noh & Il Won Seo & Sung Hyun Jung & Donghae Baek, 2021. "Identification Framework of Contaminant Spill in Rivers Using Machine Learning with Breakthrough Curve Analysis," IJERPH, MDPI, vol. 18(3), pages 1-26, January.
    18. Sylwester Bejger, 2024. "Machine Learning in Cartel Screening—The Case of Parallel Pricing in a Fuel Wholesale Market," Energies, MDPI, vol. 17(16), pages 1-17, August.
    19. Lotfi Boudabsa & Damir Filipovi'c, 2022. "Ensemble learning for portfolio valuation and risk management," Papers 2204.05926, arXiv.org.
    20. Yan, Ran & Wang, Shuaian & Du, Yuquan, 2020. "Development of a two-stage ship fuel consumption prediction and reduction model for a dry bulk ship," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 138(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:compst:v:37:y:2022:i:4:d:10.1007_s00180-021-01182-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.