IDEAS home Printed from https://ideas.repec.org/a/eee/intell/v90y2022ics016028962100088x.html
   My bibliography  Save this article

Process differences as a function of test modifications: Construct validity of Raven's advanced progressive matrices under standard, abbreviated and/or speeded conditions – A meta-analysis

Author

Listed:
  • Tatel, Corey E.
  • Tidler, Zachary R.
  • Ackerman, Phillip L.

Abstract

Historically, there has been substantial disagreement about the importance of speed vs. level in determining individual differences in intelligence – a disagreement that persists across various different modern assessment measures of intellectual abilities. The current investigation considers whether changes to the administration constraints (time limitations or speededness, and total test length) of the Raven's Advanced Progressive Matrices test – which has been identified as a measure highly saturated with general intelligence – results in differences to the underlying ability determinants of test performance. A review of empirical studies was conducted, where versions of Raven's Advanced Progressive Matrices Tests were administered under various time constraints and item lengths. Meta-analytic techniques were used to determine whether introducing speed constraints or shortening the length of the test changes the construct validity of the tests (as indicated by differences in convergent and discriminant correlations with other ability traits). The meta-analysis combined results from 142 studies composed of a total of 26,848 participants. Substantial differences were found for correlations of Raven's Advanced Progressive Matrices and Spatial Visualization (as large as ρ̂ = 0.26), Memory (as large as ρ̂ = 0.08), and Perceptual Speed (as large as ρ̂ = 0.34) abilities under speeded conditions and shorter test lengths. Examinees may draw on different strategies for test performance, that in turn, draw on different combinations of abilities, when the test is abbreviated or significant time constraints are introduced. Implications for using this test under different conditions are discussed.

Suggested Citation

  • Tatel, Corey E. & Tidler, Zachary R. & Ackerman, Phillip L., 2022. "Process differences as a function of test modifications: Construct validity of Raven's advanced progressive matrices under standard, abbreviated and/or speeded conditions – A meta-analysis," Intelligence, Elsevier, vol. 90(C).
    • Handle: RePEc:eee:intell:v:90:y:2022:i:c:s016028962100088x
    DOI: 10.1016/j.intell.2021.101604
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S016028962100088X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.intell.2021.101604?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. du Pont, Alta & Karbin, Zoe & Rhee, Soo Hyun & Corley, Robin P. & Hewitt, John K. & Friedman, Naomi P., 2020. "Differential associations between rumination and intelligence subtypes," Intelligence, Elsevier, vol. 78(C).
    2. Martinez, David, 2019. "Immediate and long-term memory and their relation to crystallized and fluid intelligence," Intelligence, Elsevier, vol. 76(C).
    3. Burgoyne, Alexander P. & Harris, Lauren Julius & Hambrick, David Z., 2019. "Predicting piano skill acquisition in beginners: The role of general intelligence, music aptitude, and mindset," Intelligence, Elsevier, vol. 76(C), pages 1-1.
    4. Burgoyne, Alexander P. & Harris, Lauren Julius & Hambrick, David Z., 2019. "Predicting piano skill acquisition in beginners: The role of general intelligence, music aptitude, and mindset," Intelligence, Elsevier, vol. 76(C).
    5. Miroshnik, Kirill G. & Shcherbakova, Olga V., 2019. "The proportion and creativity of “old” and “new” ideas: Are they related to fluid intelligence?," Intelligence, Elsevier, vol. 76(C).
    6. Ren, Xuezhu & Tong, Yan & Peng, Peng & Wang, Tengfei, 2020. "Critical thinking predicts academic performance beyond general cognitive ability: Evidence from adults and children," Intelligence, Elsevier, vol. 82(C).
    7. David Moher & Alessandro Liberati & Jennifer Tetzlaff & Douglas G Altman & The PRISMA Group, 2009. "Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement," PLOS Medicine, Public Library of Science, vol. 6(7), pages 1-6, July.
    8. Martinez, David, 2019. "Immediate and long-term memory and their relation to crystallized and fluid intelligence," Intelligence, Elsevier, vol. 76(C), pages 1-1.
    9. Viechtbauer, Wolfgang, 2010. "Conducting Meta-Analyses in R with the metafor Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 36(i03).
    10. Frederic Lord, 1956. "A study of speed factors in tests and academic grades," Psychometrika, Springer;The Psychometric Society, vol. 21(1), pages 31-50, March.
    11. Klaas Sijtsma, 2009. "On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha," Psychometrika, Springer;The Psychometric Society, vol. 74(1), pages 107-120, March.
    12. repec:cup:judgdm:v:7:y:2012:i:1:p:25-47 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Miroshnik, Kirill G. & Forthmann, Boris & Karwowski, Maciej & Benedek, Mathias, 2023. "The relationship of divergent thinking with broad retrieval ability and processing speed: A meta-analysis," Intelligence, Elsevier, vol. 98(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Piers Steel & Sjoerd Beugelsdijk & Herman Aguinis, 2021. "The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 52(1), pages 23-44, February.
    2. Kelly R Moran & Sara Y Del Valle, 2016. "A Meta-Analysis of the Association between Gender and Protective Behaviors in Response to Respiratory Epidemics and Pandemics," PLOS ONE, Public Library of Science, vol. 11(10), pages 1-25, October.
    3. Evangelos Danopoulos & Maureen Twiddy & Jeanette M Rotchell, 2020. "Microplastic contamination of drinking water: A systematic review," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-23, July.
    4. Boshra H. Namin & Torvald Øgaard & Jo Røislien, 2021. "Workplace Incivility and Turnover Intention in Organizations: A Meta-Analytic Review," IJERPH, MDPI, vol. 19(1), pages 1-19, December.
    5. Ruohuang Jiao & Wojtek Przepiorka & Vincent Buskens, 2022. "Moderators of reputation effects in peer-to-peer online markets: a meta-analytic model selection approach," Journal of Computational Social Science, Springer, vol. 5(1), pages 1041-1067, May.
    6. Pedro Silva Moreira & Pedro R Almeida & Hugo Leite-Almeida & Nuno Sousa & Patrício Costa, 2016. "Impact of Chronic Stress Protocols in Learning and Memory in Rodents: Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 11(9), pages 1-24, September.
    7. repec:cup:judgdm:v:17:y:2022:i:4:p:720-744 is not listed on IDEAS
    8. Woodley of Menie, Michael A. & Peñaherrera-Aguirre, Mateo & Sarraf, Matthew A., 2022. "Signs of a Flynn effect in rodents? Secular differentiation of the manifold of general cognitive ability in laboratory mice (Mus musculus) and Norwegian rats (Rattus norvegicus) over a century—Results," Intelligence, Elsevier, vol. 95(C).
    9. Chad K Bush & Dayaamayi Kurimella & Lee J S Cross & Katherine R Conner & Sheryl Martin-Schild & Jiang He & Changwei Li & Jing Chen & Tanika Kelly, 2016. "Endovascular Treatment with Stent-Retriever Devices for Acute Ischemic Stroke: A Meta-Analysis of Randomized Controlled Trials," PLOS ONE, Public Library of Science, vol. 11(1), pages 1-19, January.
    10. Laurens Manning & Moses Laman & Wendy A Davis & Timothy M E Davis, 2014. "Clinical Features and Outcome in Children with Severe Plasmodium falciparum Malaria: A Meta-Analysis," PLOS ONE, Public Library of Science, vol. 9(2), pages 1-11, February.
    11. Mikkel Wallentin, 2018. "Sex differences in post-stroke aphasia rates are caused by age. A meta-analysis and database query," PLOS ONE, Public Library of Science, vol. 13(12), pages 1-18, December.
    12. Ewa Zasadzka & Anna Pieczyńska & Tomasz Trzmiel & Paweł Kleka & Mariola Pawlaczyk, 2021. "Correlation between Handgrip Strength and Depression in Older Adults—A Systematic Review and a Meta-Analysis," IJERPH, MDPI, vol. 18(9), pages 1-15, April.
    13. Dominika Wilczyńska & Wen Qi & José Carlos Jaenes & David Alarcón & María José Arenilla & Mariusz Lipowski, 2022. "Burnout and Mental Interventions among Youth Athletes: A Systematic Review and Meta-Analysis of the Studies," IJERPH, MDPI, vol. 19(17), pages 1-17, August.
    14. van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2018. "P-uniform," MetaArXiv zqjr9, Center for Open Science.
    15. B G Pijls & J M T A Meessen & J W Schoones & M Fiocco & H J L van der Heide & A Sedrakyan & R G H H Nelissen, 2016. "Increased Mortality in Metal-on-Metal versus Non-Metal-on-Metal Primary Total Hip Arthroplasty at 10 Years and Longer Follow-Up: A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-17, June.
    16. Briana N. M. Hagen & Charlotte B. Winder & Jared Wootten & Carrie K. McMullen & Andria Jones-Bitton, 2020. "A Systematic Review and Meta-Analysis of Depression among Farming Populations Worldwide," IJERPH, MDPI, vol. 17(24), pages 1-15, December.
    17. Sergio Nolazco & Kaspar Delhey & Shinichi Nakagawa & Anne Peters, 2022. "Ornaments are equally informative in male and female birds," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    18. Schubert, Anna-Lena, 2019. "A meta-analysis of the worst performance rule," Intelligence, Elsevier, vol. 73(C), pages 88-100.
    19. Boris Forthmann & Karin Kaczykowski & Mathias Benedek & Heinz Holling, 2023. "The Manic Idea Creator? A Review and Meta-Analysis of the Relationship between Bipolar Disorder and Creative Cognitive Potential," IJERPH, MDPI, vol. 20(13), pages 1-39, June.
    20. Fernando Santini & Wagner Junior Ladeira & Diego Costa Pinto & Márcia Maurer Herter & Claudio Hoffmann Sampaio & Barry J. Babin, 2020. "Customer engagement in social media: a framework and meta-analysis," Journal of the Academy of Marketing Science, Springer, vol. 48(6), pages 1211-1228, November.
    21. Merga Bayssa & Sintayehu Yigrem & Simret Betsha & Adugna Tolera, 2021. "Production, reproduction and some adaptation characteristics of Boran cattle breed under changing climate: A systematic review and meta-analysis," PLOS ONE, Public Library of Science, vol. 16(5), pages 1-26, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intell:v:90:y:2022:i:c:s016028962100088x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/intelligence .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.