IDEAS home Printed from https://ideas.repec.org/a/sae/sagope/v5y2015i2p2158244015581860.html
   My bibliography  Save this article

Selecting Testlet Features With Predictive Value for the Testlet Effect

Author

Listed:
  • Muirne C. S. Paap
  • Qiwei He
  • Bernard P. Veldkamp

Abstract

High-stakes tests often consist of sets of questions (i.e., items) grouped around a common stimulus. Such groupings of items are often called testlets . A basic assumption of item response theory (IRT), the mathematical model commonly used in the analysis of test data, is that individual items are independent of one another. The potential dependency among items within a testlet is often ignored in practice. In this study, a technique called tree-based regression (TBR) was applied to identify key features of stimuli that could properly predict the dependence structure of testlet data for the Analytical Reasoning section of a high-stakes test. Relevant features identified included Percentage of “If†Clauses, Number of Entities, Theme/Topic, and Predicate Propositional Density; the testlet effect was smallest for stimuli that contained 31% or fewer “if†clauses, contained 9.8% or fewer verbs, and had Media or Animals as the main theme. This study illustrates the merits of TBR in the analysis of test data.

Suggested Citation

  • Muirne C. S. Paap & Qiwei He & Bernard P. Veldkamp, 2015. "Selecting Testlet Features With Predictive Value for the Testlet Effect," SAGE Open, , vol. 5(2), pages 21582440155, April.
    • Handle: RePEc:sae:sagope:v:5:y:2015:i:2:p:2158244015581860
    DOI: 10.1177/2158244015581860
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/2158244015581860
    Download Restriction: no
    File URL: https://libkey.io/10.1177/2158244015581860?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Eric Bradlow & Howard Wainer & Xiaohui Wang, 1999. "A Bayesian random effects model for testlets," Psychometrika, Springer;The Psychometric Society, vol. 64(2), pages 153-168, June.
    2. Martijn G. de Jong & Jan-Benedict E. M. Steenkamp & Bernard P. Veldkamp, 2009. "A Model for the Construction of Country-Specific Yet Internationally Comparable Short-Form Marketing Scales," Marketing Science, INFORMS, vol. 28(4), pages 674-689, 07-08.
    3. Edward Ip, 2001. "Testing for local dependency in dichotomous and polytomous item response models," Psychometrika, Springer;The Psychometric Society, vol. 66(1), pages 109-132, March.
    4. Paul Rosenbaum, 1984. "Testing the conditional independence and monotonicity assumptions of item response theory," Psychometrika, Springer;The Psychometric Society, vol. 49(3), pages 425-435, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gülden Kaya Uyanik & Levent Ertuna, 2022. "Examination of Testlet Effect in Open-Ended Items," SAGE Open, , vol. 12(1), pages 21582440221, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yuri Goegebeur & Paul Boeck & James Wollack & Allan Cohen, 2008. "A Speeded Item Response Model with Gradual Process Change," Psychometrika, Springer;The Psychometric Society, vol. 73(1), pages 65-87, March.
    2. Francesco Bartolucci & Antonio Forcina, 2005. "Likelihood inference on the underlying structure of IRT models," Psychometrika, Springer;The Psychometric Society, vol. 70(1), pages 31-43, March.
    3. Wang, Luming & Finn, Adam, 2013. "Dual-faceted multidimensional IRT models with hierarchical structure," Australasian marketing journal, Elsevier, vol. 21(2), pages 111-118.
    4. Jules Ellis, 2014. "An Inequality for Correlations in Unidimensional Monotone Latent Variable Models for Binary Variables," Psychometrika, Springer;The Psychometric Society, vol. 79(2), pages 303-316, April.
    5. Stefano Noventa & Andrea Spoto & Jürgen Heller & Augustin Kelava, 2019. "On a Generalization of Local Independence in Item Response Theory Based on Knowledge Space Theory," Psychometrika, Springer;The Psychometric Society, vol. 84(2), pages 395-421, June.
    6. Quinn N. Lathrop & Ying Cheng, 2017. "Item Cloning Variation and the Impact on the Parameters of Response Models," Psychometrika, Springer;The Psychometric Society, vol. 82(1), pages 245-263, March.
    7. Jules Ellis & Arnold Wollenberg, 1993. "Local homogeneity in latent trait models. A characterization of the homogeneous monotone irt model," Psychometrika, Springer;The Psychometric Society, vol. 58(3), pages 417-429, September.
    8. Wang, Luming & Finn, Adam, 2014. "A psychometric theory that measures up to marketing reality: An adapted Many Faceted IRT model," Australasian marketing journal, Elsevier, vol. 22(2), pages 93-102.
    9. Martijn G. de Jong & Jan-Benedict E. M. Steenkamp & Bernard P. Veldkamp, 2009. "A Model for the Construction of Country-Specific Yet Internationally Comparable Short-Form Marketing Scales," Marketing Science, INFORMS, vol. 28(4), pages 674-689, 07-08.
    10. Thum, Anna-Elisabeth, 2013. "Psychology in econometric models: conceptual and methodological foundations," MPRA Paper 52293, University Library of Munich, Germany.
    11. Michael Edwards, 2010. "A Markov Chain Monte Carlo Approach to Confirmatory Item Factor Analysis," Psychometrika, Springer;The Psychometric Society, vol. 75(3), pages 474-497, September.
    12. Jesper Tijmstra & Herbert Hoijtink & Klaas Sijtsma, 2015. "Evaluating Manifest Monotonicity Using Bayes Factors," Psychometrika, Springer;The Psychometric Society, vol. 80(4), pages 880-896, December.
    13. Jesper Tijmstra & Maria Bolsinova, 2019. "Bayes Factors for Evaluating Latent Monotonicity in Polytomous Item Response Theory Models," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 846-869, September.
    14. Michela Gnaldi & Silvia Bacci & Thiemo Kunze & Samuel Greiff, 2020. "Students’ Complex Problem Solving Profiles," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 469-501, June.
    15. Li Cai, 2010. "A Two-Tier Full-Information Item Factor Analysis Model with Applications," Psychometrika, Springer;The Psychometric Society, vol. 75(4), pages 581-612, December.
    16. Peida Zhan & Hong Jiao & Dandan Liao & Feiming Li, 2019. "A Longitudinal Higher-Order Diagnostic Classification Model," Journal of Educational and Behavioral Statistics, , vol. 44(3), pages 251-281, June.
    17. Paul Rosenbaum, 1989. "Criterion-related construct validity," Psychometrika, Springer;The Psychometric Society, vol. 54(4), pages 625-633, September.
    18. repec:jss:jstsof:36:c01 is not listed on IDEAS
    19. Jesper Tijmstra & David Hessen & Peter Heijden & Klaas Sijtsma, 2013. "Testing Manifest Monotonicity Using Order-Constrained Statistical Inference," Psychometrika, Springer;The Psychometric Society, vol. 78(1), pages 83-97, January.
    20. Su, Liangjun & White, Halbert, 2014. "Testing conditional independence via empirical likelihood," Journal of Econometrics, Elsevier, vol. 182(1), pages 27-44.
    21. Yunxiao Chen & Xiaoou Li & Jingchen Liu & Zhiliang Ying, 2018. "Robust Measurement via A Fused Latent and Graphical Item Response Theory Model," Psychometrika, Springer;The Psychometric Society, vol. 83(3), pages 538-562, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:sagope:v:5:y:2015:i:2:p:2158244015581860. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.