IDEAS home Printed from https://ideas.repec.org/a/eee/intell/v90y2022ics016028962100088x.html
   My bibliography  Save this article

Process differences as a function of test modifications: Construct validity of Raven's advanced progressive matrices under standard, abbreviated and/or speeded conditions – A meta-analysis

Author

Listed:
  • Tatel, Corey E.
  • Tidler, Zachary R.
  • Ackerman, Phillip L.

Abstract

Historically, there has been substantial disagreement about the importance of speed vs. level in determining individual differences in intelligence – a disagreement that persists across various different modern assessment measures of intellectual abilities. The current investigation considers whether changes to the administration constraints (time limitations or speededness, and total test length) of the Raven's Advanced Progressive Matrices test – which has been identified as a measure highly saturated with general intelligence – results in differences to the underlying ability determinants of test performance. A review of empirical studies was conducted, where versions of Raven's Advanced Progressive Matrices Tests were administered under various time constraints and item lengths. Meta-analytic techniques were used to determine whether introducing speed constraints or shortening the length of the test changes the construct validity of the tests (as indicated by differences in convergent and discriminant correlations with other ability traits). The meta-analysis combined results from 142 studies composed of a total of 26,848 participants. Substantial differences were found for correlations of Raven's Advanced Progressive Matrices and Spatial Visualization (as large as ρ̂ = 0.26), Memory (as large as ρ̂ = 0.08), and Perceptual Speed (as large as ρ̂ = 0.34) abilities under speeded conditions and shorter test lengths. Examinees may draw on different strategies for test performance, that in turn, draw on different combinations of abilities, when the test is abbreviated or significant time constraints are introduced. Implications for using this test under different conditions are discussed.

Suggested Citation

  • Tatel, Corey E. & Tidler, Zachary R. & Ackerman, Phillip L., 2022. "Process differences as a function of test modifications: Construct validity of Raven's advanced progressive matrices under standard, abbreviated and/or speeded conditions – A meta-analysis," Intelligence, Elsevier, vol. 90(C).
    • Handle: RePEc:eee:intell:v:90:y:2022:i:c:s016028962100088x
    DOI: 10.1016/j.intell.2021.101604
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S016028962100088X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.intell.2021.101604?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. du Pont, Alta & Karbin, Zoe & Rhee, Soo Hyun & Corley, Robin P. & Hewitt, John K. & Friedman, Naomi P., 2020. "Differential associations between rumination and intelligence subtypes," Intelligence, Elsevier, vol. 78(C).
    2. Martinez, David, 2019. "Immediate and long-term memory and their relation to crystallized and fluid intelligence," Intelligence, Elsevier, vol. 76(C).
    3. Burgoyne, Alexander P. & Harris, Lauren Julius & Hambrick, David Z., 2019. "Predicting piano skill acquisition in beginners: The role of general intelligence, music aptitude, and mindset," Intelligence, Elsevier, vol. 76(C), pages 1-1.
    4. Burgoyne, Alexander P. & Harris, Lauren Julius & Hambrick, David Z., 2019. "Predicting piano skill acquisition in beginners: The role of general intelligence, music aptitude, and mindset," Intelligence, Elsevier, vol. 76(C).
    5. Miroshnik, Kirill G. & Shcherbakova, Olga V., 2019. "The proportion and creativity of “old” and “new” ideas: Are they related to fluid intelligence?," Intelligence, Elsevier, vol. 76(C).
    6. Ren, Xuezhu & Tong, Yan & Peng, Peng & Wang, Tengfei, 2020. "Critical thinking predicts academic performance beyond general cognitive ability: Evidence from adults and children," Intelligence, Elsevier, vol. 82(C).
    7. David Moher & Alessandro Liberati & Jennifer Tetzlaff & Douglas G Altman & The PRISMA Group, 2009. "Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement," PLOS Medicine, Public Library of Science, vol. 6(7), pages 1-6, July.
    8. Martinez, David, 2019. "Immediate and long-term memory and their relation to crystallized and fluid intelligence," Intelligence, Elsevier, vol. 76(C), pages 1-1.
    9. Viechtbauer, Wolfgang, 2010. "Conducting Meta-Analyses in R with the metafor Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 36(i03).
    10. Frederic Lord, 1956. "A study of speed factors in tests and academic grades," Psychometrika, Springer;The Psychometric Society, vol. 21(1), pages 31-50, March.
    11. Klaas Sijtsma, 2009. "On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha," Psychometrika, Springer;The Psychometric Society, vol. 74(1), pages 107-120, March.
    12. repec:cup:judgdm:v:7:y:2012:i:1:p:25-47 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Miroshnik, Kirill G. & Forthmann, Boris & Karwowski, Maciej & Benedek, Mathias, 2023. "The relationship of divergent thinking with broad retrieval ability and processing speed: A meta-analysis," Intelligence, Elsevier, vol. 98(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Piers Steel & Sjoerd Beugelsdijk & Herman Aguinis, 2021. "The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 52(1), pages 23-44, February.
    2. Mahesh Shumsher Rughooputh & Rui Zeng & Ying Yao, 2015. "Protein Diet Restriction Slows Chronic Kidney Disease Progression in Non-Diabetic and in Type 1 Diabetic Patients, but Not in Type 2 Diabetic Patients: A Meta-Analysis of Randomized Controlled Trials ," PLOS ONE, Public Library of Science, vol. 10(12), pages 1-17, December.
    3. Christopher Winchester & Kelsey E. Medeiros, 2023. "In Bounds but Out of the Box: A Meta-Analysis Clarifying the Effect of Ethicality on Creativity," Journal of Business Ethics, Springer, vol. 183(3), pages 713-743, March.
    4. Kelly R Moran & Sara Y Del Valle, 2016. "A Meta-Analysis of the Association between Gender and Protective Behaviors in Response to Respiratory Epidemics and Pandemics," PLOS ONE, Public Library of Science, vol. 11(10), pages 1-25, October.
    5. Sandra Feijóo & Raquel Rodríguez-Fernández, 2021. "A Meta-Analytical Review of Gender-Based School Bullying in Spain," IJERPH, MDPI, vol. 18(23), pages 1-13, December.
    6. Xizheng Xu & Zhiqiang Liu & Shaoying Gong & Yunpeng Wu, 2022. "The Relationship between Empathy and Attachment in Children and Adolescents: Three-Level Meta-Analyses," IJERPH, MDPI, vol. 19(3), pages 1-18, January.
    7. Kathrin Wunsch & Janis Fiedler & Philip Bachert & Alexander Woll, 2021. "The Tridirectional Relationship among Physical Activity, Stress, and Academic Performance in University Students: A Systematic Review and Meta-Analysis," IJERPH, MDPI, vol. 18(2), pages 1-18, January.
    8. Alan da Silveira Fleck & Margaux L. Sadoine & Stéphane Buteau & Eva Suarthana & Maximilien Debia & Audrey Smargiassi, 2021. "Environmental and Occupational Short-Term Exposure to Airborne Particles and FEV 1 and FVC in Healthy Adults: A Systematic Review and Meta-Analysis," IJERPH, MDPI, vol. 18(20), pages 1-19, October.
    9. Evangelos Danopoulos & Maureen Twiddy & Jeanette M Rotchell, 2020. "Microplastic contamination of drinking water: A systematic review," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-23, July.
    10. Claudia Menne-Lothmann & Wolfgang Viechtbauer & Petra Höhn & Zuzana Kasanova & Simone P Haller & Marjan Drukker & Jim van Os & Marieke Wichers & Jennifer Y F Lau, 2014. "How to Boost Positive Interpretations? A Meta-Analysis of the Effectiveness of Cognitive Bias Modification for Interpretation," PLOS ONE, Public Library of Science, vol. 9(6), pages 1-26, June.
    11. José Felipe Orzuna-Orzuna & Juan Eduardo Godina-Rodríguez & Jonathan Raúl Garay-Martínez & Guillermo Reséndiz-González & Santiago Joaquín-Cancino & Alejandro Lara-Bueno, 2024. "Milk Yield, Composition, and Fatty Acid Profile in Milk of Dairy Cows Supplemented with Microalgae Schizochytrium sp.: A Meta-Analysis," Agriculture, MDPI, vol. 14(7), pages 1-13, July.
    12. Wei-Cheng Chang & Chin Lin & Cho-Hao Lee & Tzu-Ling Sung & Tao-Hsin Tung & Jorn-Hon Liu, 2017. "Vitrectomy with or without internal limiting membrane peeling for idiopathic epiretinal membrane: A meta-analysis," PLOS ONE, Public Library of Science, vol. 12(6), pages 1-18, June.
    13. Christopher Hansen & Holger Steinmetz & Jörn Block, 2022. "How to conduct a meta-analysis in eight steps: a practical guide," Management Review Quarterly, Springer, vol. 72(1), pages 1-19, February.
    14. Shaylea Badovinac & Jodi Martin & Camille Guérin-Marion & Monica O’Neill & Rebecca Pillai Riddell & Jean-François Bureau & Rebecca Spiegel, 2018. "Associations between mother-preschooler attachment and maternal depression symptoms: A systematic review and meta-analysis," PLOS ONE, Public Library of Science, vol. 13(10), pages 1-27, October.
    15. Fazel, Seena & Burghart, Matthias & Fanshawe, Thomas & Gil, Sharon Danielle & Monahan, John & Yu, Rongqin, 2022. "The predictive performance of criminal risk assessment tools used at sentencing: Systematic review of validation studies," Journal of Criminal Justice, Elsevier, vol. 81(C).
    16. Amro Qaddoura & Payam Yazdan-Ashoori & Conrad Kabali & Lehana Thabane & R Brian Haynes & Stuart J Connolly & Harriette Gillian Christine Van Spall, 2015. "Efficacy of Hospital at Home in Patients with Heart Failure: A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-15, June.
    17. Boshra H. Namin & Torvald Øgaard & Jo Røislien, 2021. "Workplace Incivility and Turnover Intention in Organizations: A Meta-Analytic Review," IJERPH, MDPI, vol. 19(1), pages 1-19, December.
    18. Ruohuang Jiao & Wojtek Przepiorka & Vincent Buskens, 2022. "Moderators of reputation effects in peer-to-peer online markets: a meta-analytic model selection approach," Journal of Computational Social Science, Springer, vol. 5(1), pages 1041-1067, May.
    19. Pedro Silva Moreira & Pedro R Almeida & Hugo Leite-Almeida & Nuno Sousa & Patrício Costa, 2016. "Impact of Chronic Stress Protocols in Learning and Memory in Rodents: Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 11(9), pages 1-24, September.
    20. Romain Cadario & Pierre Chandon, 2020. "Which Healthy Eating Nudges Work Best? A Meta-Analysis of Field Experiments," Marketing Science, INFORMS, vol. 39(3), pages 465-486, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intell:v:90:y:2022:i:c:s016028962100088x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/intelligence .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.