IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v179y2023ics0167947322002080.html
   My bibliography  Save this article

Tree-based ensembles for multi-output regression: Comparing multivariate approaches with separate univariate ones

Author

Listed:
  • Schmid, Lena
  • Gerharz, Alexander
  • Groll, Andreas
  • Pauly, Markus

Abstract

Tree-based ensembles such as the Random Forest are modern classics among statistical learning methods. In particular, they are used for predicting univariate responses. In case of multiple outputs the question arises whether it is better to separately fit univariate models or directly follow a multivariate approach. For the latter, several possibilities exist that are, e.g. based on modified splitting or stopping rules for multi-output regression. These methods are compared in extensive simulations and a real data example to help in answering the primary question when to use multivariate ensemble techniques instead of univariate ones.

Suggested Citation

  • Schmid, Lena & Gerharz, Alexander & Groll, Andreas & Pauly, Markus, 2023. "Tree-based ensembles for multi-output regression: Comparing multivariate approaches with separate univariate ones," Computational Statistics & Data Analysis, Elsevier, vol. 179(C).
    • Handle: RePEc:eee:csdana:v:179:y:2023:i:c:s0167947322002080
    DOI: 10.1016/j.csda.2022.107628
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947322002080
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2022.107628?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Wright, Marvin N. & Ziegler, Andreas, 2017. "ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 77(i01).
    2. Dine, Abdessamad & Larocque, Denis & Bellavance, François, 2009. "Multivariate trees for mixed outcomes," Computational Statistics & Data Analysis, Elsevier, vol. 53(11), pages 3795-3804, September.
    3. David R. Larsen & Paul L. Speckman, 2004. "Multivariate Regression Trees for Analysis of Abundance Data," Biometrics, The International Biometric Society, vol. 60(2), pages 543-549, June.
    4. Zimmermann, Georg & Pauly, Markus & Bathke, Arne C., 2020. "Multivariate analysis of covariance with potentially singular covariance matrices and non-normal responses," Journal of Multivariate Analysis, Elsevier, vol. 177(C).
    5. Siciliano, Roberta & Mola, Francesco, 2000. "Multivariate data analysis and modeling through classification and regression trees," Computational Statistics & Data Analysis, Elsevier, vol. 32(3-4), pages 285-301, January.
    6. Hsiao, Wei-Cheng & Shih, Yu-Shan, 2007. "Splitting variable selection for multivariate regression trees," Statistics & Probability Letters, Elsevier, vol. 77(3), pages 265-271, February.
    7. Keon Lee, Seong, 2005. "On generalized multivariate decision tree by using GEE," Computational Statistics & Data Analysis, Elsevier, vol. 49(4), pages 1105-1119, June.
    8. Konietschke, Frank & Bathke, Arne C. & Harrar, Solomon W. & Pauly, Markus, 2015. "Parametric and nonparametric bootstrap methods for general MANOVA," Journal of Multivariate Analysis, Elsevier, vol. 140(C), pages 291-301.
    9. Dennis Dobler & Sarah Friedrich & Markus Pauly, 2020. "Nonparametric MANOVA in meaningful effects," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 72(4), pages 997-1022, August.
    10. Véronique Cariou, 2006. "Extension of multivariate regression trees to interval data. Application to electricity load profiling," Computational Statistics, Springer, vol. 21(2), pages 325-341, June.
    11. Kocev, Dragi & Džeroski, Sašo & White, Matt D. & Newell, Graeme R. & Griffioen, Peter, 2009. "Using single- and multi-target regression trees and ensembles to model a compound index of vegetation condition," Ecological Modelling, Elsevier, vol. 220(8), pages 1159-1168.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Federico Zincenko, 2023. "Nonparametric estimation of conditional densities by generalized random forests," Papers 2309.13251, arXiv.org, revised May 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Piccarreta, Raffaella, 2010. "Binary trees for dissimilarity data," Computational Statistics & Data Analysis, Elsevier, vol. 54(6), pages 1516-1524, June.
    2. Wei-Yin Loh, 2014. "Fifty Years of Classification and Regression Trees," International Statistical Review, International Statistical Institute, vol. 82(3), pages 329-348, December.
    3. Baumeister, Marléne & Ditzhaus, Marc & Pauly, Markus, 2024. "Quantile-based MANOVA: A new tool for inferring multivariate data in factorial designs," Journal of Multivariate Analysis, Elsevier, vol. 199(C).
    4. Hsiao, Wei-Cheng & Shih, Yu-Shan, 2007. "Splitting variable selection for multivariate regression trees," Statistics & Probability Letters, Elsevier, vol. 77(3), pages 265-271, February.
    5. Antonio D’Ambrosio & Willem J. Heiser, 2016. "A Recursive Partitioning Method for the Prediction of Preference Rankings Based Upon Kemeny Distances," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 774-794, September.
    6. Dine, Abdessamad & Larocque, Denis & Bellavance, François, 2009. "Multivariate trees for mixed outcomes," Computational Statistics & Data Analysis, Elsevier, vol. 53(11), pages 3795-3804, September.
    7. Backer, David & Billing, Trey, 2024. "Forecasting the prevalence of child acute malnutrition using environmental and conflict conditions as leading indicators," World Development, Elsevier, vol. 176(C).
    8. Friedrich, Sarah & Pauly, Markus, 2018. "MATS: Inference for potentially singular and heteroscedastic MANOVA," Journal of Multivariate Analysis, Elsevier, vol. 165(C), pages 166-179.
    9. Mariana Oliveira & Luís Torgo & Vítor Santos Costa, 2021. "Evaluation Procedures for Forecasting with Spatiotemporal Data," Mathematics, MDPI, vol. 9(6), pages 1-27, March.
    10. Smaga, Łukasz, 2015. "Wald-type statistics using {2}-inverses for hypothesis testing in general factorial designs," Statistics & Probability Letters, Elsevier, vol. 107(C), pages 215-220.
    11. Bokelmann, Björn & Lessmann, Stefan, 2024. "Improving uplift model evaluation on randomized controlled trial data," European Journal of Operational Research, Elsevier, vol. 313(2), pages 691-707.
    12. Joel Podgorski & Oliver Kracht & Luis Araguas-Araguas & Stefan Terzer-Wassmuth & Jodie Miller & Ralf Straub & Rolf Kipfer & Michael Berg, 2024. "Groundwater vulnerability to pollution in Africa’s Sahel region," Nature Sustainability, Nature, vol. 7(5), pages 558-567, May.
    13. Dickhaus, Thorsten & Sirotko-Sibirskaya, Natalia, 2019. "Simultaneous statistical inference in dynamic factor models: Chi-square approximation and model-based bootstrap," Computational Statistics & Data Analysis, Elsevier, vol. 129(C), pages 30-46.
    14. Chakravorty, Bhaskar & Arulampalam, Wiji & Bhatiya, Apurav Yash & Imbert, Clément & Rathelot, Roland, 2024. "Can information about jobs improve the effectiveness of vocational training? Experimental evidence from India," Journal of Development Economics, Elsevier, vol. 169(C).
    15. Arjan S. Gosal & Janine A. McMahon & Katharine M. Bowgen & Catherine H. Hoppe & Guy Ziv, 2021. "Identifying and Mapping Groups of Protected Area Visitors by Environmental Awareness," Land, MDPI, vol. 10(6), pages 1-14, May.
    16. Albert Stuart Reece & Gary Kenneth Hulse, 2022. "European Epidemiological Patterns of Cannabis- and Substance-Related Congenital Neurological Anomalies: Geospatiotemporal and Causal Inferential Study," IJERPH, MDPI, vol. 20(1), pages 1-35, December.
    17. Giorgos Foutzopoulos & Nikolaos Pandis & Michail Tsagris, 2024. "Predicting Full Retirement Attainment of NBA Players," Working Papers 2403, University of Crete, Department of Economics.
    18. Michael Parzinger & Lucia Hanfstaengl & Ferdinand Sigg & Uli Spindler & Ulrich Wellisch & Markus Wirnsberger, 2020. "Residual Analysis of Predictive Modelling Data for Automated Fault Detection in Building’s Heating, Ventilation and Air Conditioning Systems," Sustainability, MDPI, vol. 12(17), pages 1-18, August.
    19. Pier Perri & Peter Heijden, 2012. "A Property of the CHAID Partitioning Method for Dichotomous Randomized Response Data and Categorical Predictors," Journal of Classification, Springer;The Classification Society, vol. 29(1), pages 76-90, April.
    20. Van Belle, Jente & Guns, Tias & Verbeke, Wouter, 2021. "Using shared sell-through data to forecast wholesaler demand in multi-echelon supply chains," European Journal of Operational Research, Elsevier, vol. 288(2), pages 466-479.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:179:y:2023:i:c:s0167947322002080. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.