IDEAS home Printed from https://ideas.repec.org/a/sae/sagope/v5y2015i2p2158244015581860.html
   My bibliography  Save this article

Selecting Testlet Features With Predictive Value for the Testlet Effect

Author

Listed:
  • Muirne C. S. Paap
  • Qiwei He
  • Bernard P. Veldkamp

Abstract

High-stakes tests often consist of sets of questions (i.e., items) grouped around a common stimulus. Such groupings of items are often called testlets . A basic assumption of item response theory (IRT), the mathematical model commonly used in the analysis of test data, is that individual items are independent of one another. The potential dependency among items within a testlet is often ignored in practice. In this study, a technique called tree-based regression (TBR) was applied to identify key features of stimuli that could properly predict the dependence structure of testlet data for the Analytical Reasoning section of a high-stakes test. Relevant features identified included Percentage of “If†Clauses, Number of Entities, Theme/Topic, and Predicate Propositional Density; the testlet effect was smallest for stimuli that contained 31% or fewer “if†clauses, contained 9.8% or fewer verbs, and had Media or Animals as the main theme. This study illustrates the merits of TBR in the analysis of test data.

Suggested Citation

  • Muirne C. S. Paap & Qiwei He & Bernard P. Veldkamp, 2015. "Selecting Testlet Features With Predictive Value for the Testlet Effect," SAGE Open, , vol. 5(2), pages 21582440155, April.
    • Handle: RePEc:sae:sagope:v:5:y:2015:i:2:p:2158244015581860
    DOI: 10.1177/2158244015581860
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/2158244015581860
    Download Restriction: no
    File URL: https://libkey.io/10.1177/2158244015581860?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Martijn G. de Jong & Jan-Benedict E. M. Steenkamp & Bernard P. Veldkamp, 2009. "A Model for the Construction of Country-Specific Yet Internationally Comparable Short-Form Marketing Scales," Marketing Science, INFORMS, vol. 28(4), pages 674-689, 07-08.
    2. Eric Bradlow & Howard Wainer & Xiaohui Wang, 1999. "A Bayesian random effects model for testlets," Psychometrika, Springer;The Psychometric Society, vol. 64(2), pages 153-168, June.
    3. Edward Ip, 2001. "Testing for local dependency in dichotomous and polytomous item response models," Psychometrika, Springer;The Psychometric Society, vol. 66(1), pages 109-132, March.
    4. Paul Rosenbaum, 1984. "Testing the conditional independence and monotonicity assumptions of item response theory," Psychometrika, Springer;The Psychometric Society, vol. 49(3), pages 425-435, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gülden Kaya Uyanik & Levent Ertuna, 2022. "Examination of Testlet Effect in Open-Ended Items," SAGE Open, , vol. 12(1), pages 21582440221, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yuri Goegebeur & Paul Boeck & James Wollack & Allan Cohen, 2008. "A Speeded Item Response Model with Gradual Process Change," Psychometrika, Springer;The Psychometric Society, vol. 73(1), pages 65-87, March.
    2. Alexander Robitzsch, 2024. "A Comparison of Limited Information Estimation Methods for the Two-Parameter Normal-Ogive Model with Locally Dependent Items," Stats, MDPI, vol. 7(3), pages 1-16, June.
    3. Francesco Bartolucci & Antonio Forcina, 2005. "Likelihood inference on the underlying structure of IRT models," Psychometrika, Springer;The Psychometric Society, vol. 70(1), pages 31-43, March.
    4. Wang, Luming & Finn, Adam, 2013. "Dual-faceted multidimensional IRT models with hierarchical structure," Australasian marketing journal, Elsevier, vol. 21(2), pages 111-118.
    5. Jules Ellis, 2014. "An Inequality for Correlations in Unidimensional Monotone Latent Variable Models for Binary Variables," Psychometrika, Springer;The Psychometric Society, vol. 79(2), pages 303-316, April.
    6. Stefano Noventa & Andrea Spoto & Jürgen Heller & Augustin Kelava, 2019. "On a Generalization of Local Independence in Item Response Theory Based on Knowledge Space Theory," Psychometrika, Springer;The Psychometric Society, vol. 84(2), pages 395-421, June.
    7. Quinn N. Lathrop & Ying Cheng, 2017. "Item Cloning Variation and the Impact on the Parameters of Response Models," Psychometrika, Springer;The Psychometric Society, vol. 82(1), pages 245-263, March.
    8. Nana Kim & Daniel M. Bolt & James Wollack, 2022. "Noncompensatory MIRT For Passage-Based Tests," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 992-1009, September.
    9. Csilla Horváth & Feray Adigüzel & Hester van Herk, 2013. "Cultural Aspects Of Compulsive Buying In Emerging And Developed Economies: A Cross Cultural Study In Compulsive Buying," Organizations and Markets in Emerging Economies, Faculty of Economics, Vilnius University, vol. 4(2).
    10. A. Béguin & C. Glas, 2001. "MCMC estimation and some model-fit analysis of multidimensional IRT models," Psychometrika, Springer;The Psychometric Society, vol. 66(4), pages 541-561, December.
    11. Jules Ellis & Arnold Wollenberg, 1993. "Local homogeneity in latent trait models. A characterization of the homogeneous monotone irt model," Psychometrika, Springer;The Psychometric Society, vol. 58(3), pages 417-429, September.
    12. Edward Ip & Yuchung Wang & Paul Boeck & Michel Meulders, 2004. "Locally dependent latent trait model for polytomous responses with application to inventory of hostility," Psychometrika, Springer;The Psychometric Society, vol. 69(2), pages 191-216, June.
    13. Wang, Luming & Finn, Adam, 2014. "A psychometric theory that measures up to marketing reality: An adapted Many Faceted IRT model," Australasian marketing journal, Elsevier, vol. 22(2), pages 93-102.
    14. Martijn G. de Jong & Jan-Benedict E. M. Steenkamp & Bernard P. Veldkamp, 2009. "A Model for the Construction of Country-Specific Yet Internationally Comparable Short-Form Marketing Scales," Marketing Science, INFORMS, vol. 28(4), pages 674-689, 07-08.
    15. Forcina, Antonio, 2008. "Identifiability of extended latent class models with individual covariates," Computational Statistics & Data Analysis, Elsevier, vol. 52(12), pages 5263-5268, August.
    16. Alexander Robitzsch, 2023. "Linking Error in the 2PL Model," J, MDPI, vol. 6(1), pages 1-27, January.
    17. Jean-Paul Fox & Cees Glas, 2001. "Bayesian estimation of a multilevel IRT model using gibbs sampling," Psychometrika, Springer;The Psychometric Society, vol. 66(2), pages 271-288, June.
    18. Thum, Anna-Elisabeth, 2013. "Psychology in econometric models: conceptual and methodological foundations," MPRA Paper 52293, University Library of Munich, Germany.
    19. Michael Edwards, 2010. "A Markov Chain Monte Carlo Approach to Confirmatory Item Factor Analysis," Psychometrika, Springer;The Psychometric Society, vol. 75(3), pages 474-497, September.
    20. Sun-Joo Cho & Paul Boeck & Susan Embretson & Sophia Rabe-Hesketh, 2014. "Additive Multilevel Item Structure Models with Random Residuals: Item Modeling for Explanation and Item Generation," Psychometrika, Springer;The Psychometric Society, vol. 79(1), pages 84-104, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:sagope:v:5:y:2015:i:2:p:2158244015581860. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.