IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0262200.html
   My bibliography  Save this article

The Jack and Jill Adaptive Working Memory Task: Construction, Calibration and Validation

Author

Listed:
  • Elina Tsigeman
  • Sebastian Silas
  • Klaus Frieler
  • Maxim Likhanov
  • Rebecca Gelding
  • Yulia Kovas
  • Daniel Müllensiefen

Abstract

Visuospatial working memory (VSWM) is essential to human cognitive abilities and is associated with important life outcomes such as academic performance. Recently, a number of reliable measures of VSWM have been developed to help understand psychological processes and for practical use in education. We sought to extend this work using Item Response Theory (IRT) and Computerised Adaptive Testing (CAT) frameworks to construct, calibrate and validate a new adaptive, computerised, and open-source VSWM test. We aimed to overcome the limitations of previous instruments and provide researchers with a valid and freely available VSWM measurement tool. The Jack and Jill (JaJ) VSWM task was constructed using explanatory item response modelling of data from a sample of the general adult population (Study 1, N = 244) in the UK and US. Subsequently, a static version of the task was tested for validity and reliability using a sample of adults from the UK and Australia (Study 2, N = 148) and a sample of Russian adolescents (Study 3, N = 263). Finally, the adaptive version of the JaJ task was implemented on the basis of the underlying IRT model and evaluated with another sample of Russian adolescents (Study 4, N = 239). JaJ showed sufficient internal consistency and concurrent validity as indicated by significant and substantial correlations with established measures of working memory, spatial ability, non-verbal intelligence, and academic achievement. The findings suggest that JaJ is an efficient and reliable measure of VSWM from adolescent to adult age.

Suggested Citation

  • Elina Tsigeman & Sebastian Silas & Klaus Frieler & Maxim Likhanov & Rebecca Gelding & Yulia Kovas & Daniel Müllensiefen, 2022. "The Jack and Jill Adaptive Working Memory Task: Construction, Calibration and Validation," PLOS ONE, Public Library of Science, vol. 17(1), pages 1-29, January.
    • Handle: RePEc:plo:pone00:0262200
    DOI: 10.1371/journal.pone.0262200
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0262200
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0262200&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0262200?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Magis, David & Barrada, Juan Ramon, 2017. "Computerized Adaptive Testing with R: Recent Updates of the Package catR," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 76(c01).
    2. Chalmers, R. Philip, 2012. "mirt: A Multidimensional Item Response Theory Package for the R Environment," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i06).
    3. Thomas Warm, 1989. "Weighted likelihood estimation of ability in item response theory," Psychometrika, Springer;The Psychometric Society, vol. 54(3), pages 427-450, September.
    4. Magis, David & Raîche, Gilles, 2012. "Random Generation of Response Patterns under Computerized Adaptive Testing with the R Package catR," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i08).
    5. Douglas Bonett & Thomas Wright, 2000. "Sample size requirements for estimating pearson, kendall and spearman correlations," Psychometrika, Springer;The Psychometric Society, vol. 65(1), pages 23-28, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Klaas Sijtsma & Jules L. Ellis & Denny Borsboom, 2024. "Recognize the Value of the Sum Score, Psychometrics’ Greatest Accomplishment," Psychometrika, Springer;The Psychometric Society, vol. 89(1), pages 84-117, March.
    2. Alexander Robitzsch, 2021. "A Comprehensive Simulation Study of Estimation Methods for the Rasch Model," Stats, MDPI, vol. 4(4), pages 1-23, October.
    3. Wickelmaier, Florian & Strobl, Carolin & Zeileis, Achim, 2012. "Psychoco: Psychometric Computing in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i01).
    4. David Magis, 2016. "Efficient Standard Error Formulas of Ability Estimators with Dichotomous Item Response Models," Psychometrika, Springer;The Psychometric Society, vol. 81(1), pages 184-200, March.
    5. Peter W. Rijn & Usama S. Ali & Hyo Jeong Shin & Sean-Hwane Joo, 2024. "Adjusted Residuals for Evaluating Conditional Independence in IRT Models for Multistage Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 89(1), pages 317-346, March.
    6. Maxwell Hong & Lizhen Lin & Ying Cheng, 2021. "Asymptotically Corrected Person Fit Statistics for Multidimensional Constructs with Simple Structure and Mixed Item Types," Psychometrika, Springer;The Psychometric Society, vol. 86(2), pages 464-488, June.
    7. Chun Wang, 2015. "On Latent Trait Estimation in Multidimensional Compensatory Item Response Models," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 428-449, June.
    8. Kai Liu & Longfei Zhang & Dongbo Tu & Yan Cai, 2022. "Developing an Item Bank of Computerized Adaptive Testing for Eating Disorders in Chinese University Students," SAGE Open, , vol. 12(4), pages 21582440221, December.
    9. Carmen Köhler & Alexander Robitzsch & Johannes Hartig, 2020. "A Bias-Corrected RMSD Item Fit Statistic: An Evaluation and Comparison to Alternatives," Journal of Educational and Behavioral Statistics, , vol. 45(3), pages 251-273, June.
    10. J. R. Lockwood & D. McCaffrey, 2020. "Using hidden information and performance level boundaries to study student–teacher assignments: implications for estimating teacher causal effects," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(4), pages 1333-1362, October.
    11. Jeffrey M. Patton & Ying Cheng & Maxwell Hong & Qi Diao, 2019. "Detection and Treatment of Careless Responses to Improve Item Parameter Estimation," Journal of Educational and Behavioral Statistics, , vol. 44(3), pages 309-341, June.
    12. Torberg Falch & Justina AV Fischer, 2008. "Does a generous welfare state crowd out student achievement? Panel data evidence from international student tests," TWI Research Paper Series 31, Thurgauer Wirtschaftsinstitut, Universität Konstanz.
    13. Luo, Nanyu & Ji, Feng & Han, Yuting & He, Jinbo & Zhang, Xiaoya, 2024. "Fitting item response theory models using deep learning computational frameworks," OSF Preprints tjxab, Center for Open Science.
    14. Steger, Diana & Schroeders, Ulrich & Wilhelm, Oliver, 2019. "On the dimensionality of crystallized intelligence: A smartphone-based assessment," Intelligence, Elsevier, vol. 72(C), pages 76-85.
    15. Michela Battauz & Ruggero Bellio, 2011. "Structural Modeling of Measurement Error in Generalized Linear Models with Rasch Measures as Covariates," Psychometrika, Springer;The Psychometric Society, vol. 76(1), pages 40-56, January.
    16. Melissa Gladstone & Gillian Lancaster & Gareth McCray & Vanessa Cavallera & Claudia R. L. Alves & Limbika Maliwichi & Muneera A. Rasheed & Tarun Dua & Magdalena Janus & Patricia Kariger, 2021. "Validation of the Infant and Young Child Development (IYCD) Indicators in Three Countries: Brazil, Malawi and Pakistan," IJERPH, MDPI, vol. 18(11), pages 1-19, June.
    17. Xiang Liu & James Yang & Hui Soo Chae & Gary Natriello, 2020. "Power Divergence Family of Statistics for Person Parameters in IRT Models," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 502-525, June.
    18. Björn Andersson & Tao Xin, 2021. "Estimation of Latent Regression Item Response Theory Models Using a Second-Order Laplace Approximation," Journal of Educational and Behavioral Statistics, , vol. 46(2), pages 244-265, April.
    19. Victoria T. Tanaka & George Engelhard & Matthew P. Rabbitt, 2020. "Using a Bifactor Model to Measure Food Insecurity in Households with Children," Journal of Family and Economic Issues, Springer, vol. 41(3), pages 492-504, September.
    20. repec:zbw:rwidps:0023 is not listed on IDEAS
    21. Çetin Toraman & Güneş Korkmaz, 2023. "What is the “Meaning of School†to High School Students? A Scale Development and Implementation Study Based on IRT and CTT," SAGE Open, , vol. 13(3), pages 21582440231, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0262200. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.