IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v87y2022i2d10.1007_s11336-021-09803-z.html
   My bibliography  Save this article

A Systematic Study into the Factors that Affect the Predictive Accuracy of Multilevel VAR(1) Models

Author

Listed:
  • Ginette Lafit

    (KU Leuven – University of Leuven)

  • Kristof Meers

    (KU Leuven – University of Leuven)

  • Eva Ceulemans

    (KU Leuven – University of Leuven)

Abstract

The use of multilevel VAR(1) models to unravel within-individual process dynamics is gaining momentum in psychological research. These models accommodate the structure of intensive longitudinal datasets in which repeated measurements are nested within individuals. They estimate within-individual auto- and cross-regressive relationships while incorporating and using information about the distributions of these effects across individuals. An important quality feature of the obtained estimates pertains to how well they generalize to unseen data. Bulteel and colleagues (Psychol Methods 23(4):740–756, 2018a) showed that this feature can be assessed through a cross-validation approach, yielding a predictive accuracy measure. In this article, we follow up on their results, by performing three simulation studies that allow to systematically study five factors that likely affect the predictive accuracy of multilevel VAR(1) models: (i) the number of measurement occasions per person, (ii) the number of persons, (iii) the number of variables, (iv) the contemporaneous collinearity between the variables, and (v) the distributional shape of the individual differences in the VAR(1) parameters (i.e., normal versus multimodal distributions). Simulation results show that pooling information across individuals and using multilevel techniques prevent overfitting. Also, we show that when variables are expected to show strong contemporaneous correlations, performing multilevel VAR(1) in a reduced variable space can be useful. Furthermore, results reveal that multilevel VAR(1) models with random effects have a better predictive performance than person-specific VAR(1) models when the sample includes groups of individuals that share similar dynamics.

Suggested Citation

  • Ginette Lafit & Kristof Meers & Eva Ceulemans, 2022. "A Systematic Study into the Factors that Affect the Predictive Accuracy of Multilevel VAR(1) Models," Psychometrika, Springer;The Psychometric Society, vol. 87(2), pages 432-476, June.
    • Handle: RePEc:spr:psycho:v:87:y:2022:i:2:d:10.1007_s11336-021-09803-z
    DOI: 10.1007/s11336-021-09803-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11336-021-09803-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11336-021-09803-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Henk Kiers & Jos Berge, 1994. "The Harris-Kaiser independent cluster rotation as a method for rotation to simple component weights," Psychometrika, Springer;The Psychometric Society, vol. 59(1), pages 81-90, March.
    2. John Y. Campbell & Samuel B. Thompson, 2008. "Predicting Excess Stock Returns Out of Sample: Can Anything Beat the Historical Average?," The Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1509-1531, July.
    3. Tanja Krone & Casper J. Albers & Marieke E. Timmerman, 2017. "A comparative simulation study of AR(1) estimators in short time series," Quality & Quantity: International Journal of Methodology, Springer, vol. 51(1), pages 1-21, January.
    4. Eva Ceulemans & Iven Mechelen, 2005. "Hierarchical classes models for three-way three-mode binary data: interrelations and model selection," Psychometrika, Springer;The Psychometric Society, vol. 70(3), pages 461-480, September.
    5. Bates, Douglas & Mächler, Martin & Bolker, Ben & Walker, Steve, 2015. "Fitting Linear Mixed-Effects Models Using lme4," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 67(i01).
    6. John Horn, 1965. "A rationale and test for the number of factors in factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 30(2), pages 179-185, June.
    7. Jan Schepers & Eva Ceulemans & Iven Mechelen, 2008. "Selecting Among Multi-Mode Partitioning Models of Different Complexities: A Comparison of Four Model Selection Criteria," Journal of Classification, Springer;The Classification Society, vol. 25(1), pages 67-85, June.
    8. Marieke Timmerman & Henk Kiers, 2003. "Four simultaneous component models for the analysis of multivariate time series from more than one subject to model intraindividual and interindividual differences," Psychometrika, Springer;The Psychometric Society, vol. 68(1), pages 105-121, March.
    9. Henk Kiers & Age Smilde, 2007. "A comparison of various methods for multivariate regression with highly collinear variables," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 16(2), pages 193-228, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tom Wilderjans & E. Ceulemans & I. Mechelen, 2012. "The SIMCLAS Model: Simultaneous Analysis of Coupled Binary Data Matrices with Noise Heterogeneity Between and Within Data Blocks," Psychometrika, Springer;The Psychometric Society, vol. 77(4), pages 724-740, October.
    2. Kim De Roover & Eva Ceulemans & Marieke Timmerman & John Nezlek & Patrick Onghena, 2013. "Modeling Differences in the Dimensionality of Multiblock Data by Means of Clusterwise Simultaneous Component Analysis," Psychometrika, Springer;The Psychometric Society, vol. 78(4), pages 648-668, October.
    3. Tan, Xueping & Sirichand, Kavita & Vivian, Andrew & Wang, Xinyu, 2022. "Forecasting European carbon returns using dimension reduction techniques: Commodity versus financial fundamentals," International Journal of Forecasting, Elsevier, vol. 38(3), pages 944-969.
    4. Dirk Depril & Iven Mechelen & Tom Wilderjans, 2012. "Lowdimensional Additive Overlapping Clustering," Journal of Classification, Springer;The Classification Society, vol. 29(3), pages 297-320, October.
    5. JANSSENS, Jochen & DE CORTE, Annelies & SÖRENSEN, Kenneth, 2016. "Water distribution network design optimisation with respect to reliability," Working Papers 2016007, University of Antwerp, Faculty of Business and Economics.
    6. Gkillas, Konstantinos & Gupta, Rangan & Pierdzioch, Christian, 2020. "Forecasting realized oil-price volatility: The role of financial stress and asymmetric loss," Journal of International Money and Finance, Elsevier, vol. 104(C).
    7. Maria Lidia Mascia & Mirian Agus & Łukasz Tomczyk & Natale Salvatore Bonfiglio & Diego Bellini & Maria Pietronilla Penna, 2023. "Smartphone Distraction: Italian Validation of the Smartphone Distraction Scale (SDS)," IJERPH, MDPI, vol. 20(15), pages 1-15, August.
    8. Faria, Gonçalo & Verona, Fabio, 2023. "Forecast combination in the frequency domain," Bank of Finland Research Discussion Papers 1/2023, Bank of Finland.
    9. Papapostolou, Nikos C. & Pouliasis, Panos K. & Nomikos, Nikos K. & Kyriakou, Ioannis, 2016. "Shipping investor sentiment and international stock return predictability," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 96(C), pages 81-94.
    10. Patrick Hylton & Ben Kisby & Paul Goddard, 2018. "Young People’s Citizen Identities: A Q-Methodological Analysis of English Youth Perceptions of Citizenship in Britain," Societies, MDPI, vol. 8(4), pages 1-21, December.
    11. Raymond Hernandez & Elizabeth A. Pyatak & Cheryl L. P. Vigen & Haomiao Jin & Stefan Schneider & Donna Spruijt-Metz & Shawn C. Roll, 2021. "Understanding Worker Well-Being Relative to High-Workload and Recovery Activities across a Whole Day: Pilot Testing an Ecological Momentary Assessment Technique," IJERPH, MDPI, vol. 18(19), pages 1-17, October.
    12. Wang, Yubao & Huang, Xiaozhou & Huang, Zhendong, 2024. "Energy-related uncertainty and Chinese stock market returns," Finance Research Letters, Elsevier, vol. 62(PB).
    13. Esther Eiling & Raymond Kan & Ali Sharifkhani, 2018. "Sectoral Labor Reallocation and Return Predictability," Working Papers 2018-006, Human Capital and Economic Opportunity Working Group.
    14. Elisabeth Beckmann & Lukas Olbrich & Joseph Sakshaug, 2024. "Multivariate assessment of interviewer-related errors in a cross-national economic survey (Lukas Olbrich, Elisabeth Beckmann, Joseph W. Sakshaug)," Working Papers 253, Oesterreichische Nationalbank (Austrian Central Bank).
    15. Su, Yuandong & Lu, Xinjie & Zeng, Qing & Huang, Dengshi, 2022. "Good air quality and stock market returns," Research in International Business and Finance, Elsevier, vol. 62(C).
    16. Söderlind, Paul, 2009. "The C-CAPM without ex post data," Journal of Macroeconomics, Elsevier, vol. 31(4), pages 721-729, December.
    17. Orazio Attanasio & Sarah Cattan & Emla Fitzsimons & Costas Meghir & Marta Rubio-Codina, 2020. "Estimating the Production Function for Human Capital: Results from a Randomized Controlled Trial in Colombia," American Economic Review, American Economic Association, vol. 110(1), pages 48-85, January.
    18. Zongwu Cai & Haiqiang Chen & Xiaosai Liao, 2020. "A New Robust Inference for Predictive Quantile Regression," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 202002, University of Kansas, Department of Economics, revised Feb 2020.
    19. Salisu, Afees A. & Ademuyiwa, Idris & Isah, Kazeem O., 2018. "Revisiting the forecasting accuracy of Phillips curve: The role of oil price," Energy Economics, Elsevier, vol. 70(C), pages 334-356.
    20. Clark, Todd E. & McCracken, Michael W., 2012. "In-sample tests of predictive ability: A new approach," Journal of Econometrics, Elsevier, vol. 170(1), pages 1-14.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:87:y:2022:i:2:d:10.1007_s11336-021-09803-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.