IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v88y2023i3d10.1007_s11336-023-09912-x.html
   My bibliography  Save this article

Exploring the Effects of Item-Specific Factors in Sequential and IRTree Models

Author

Listed:
  • Weicong Lyu

    (University of Wisconsin-Madison)

  • Daniel M. Bolt

    (University of Wisconsin-Madison)

  • Samuel Westby

    (Northeastern University)

Abstract

Test items for which the item score reflects a sequential or IRTree modeling outcome are considered. For such items, we argue that item-specific factors, although not empirically measurable, will often be present across stages of the same item. In this paper, we present a conceptual model that incorporates such factors. We use the model to demonstrate how the varying conditional distributions of item-specific factors across stages become absorbed into the stage-specific item discrimination and difficulty parameters, creating ambiguity in the interpretations of item and person parameters beyond the first stage. We discuss implications in relation to various applications considered in the literature, including methodological studies of (1) repeated attempt items; (2) answer change/review, (3) on-demand item hints; (4) item skipping behavior; and (5) Likert scale items. Our own empirical applications, as well as several examples published in the literature, show patterns of violations of item parameter invariance across stages that are highly suggestive of item-specific factors. For applications using sequential or IRTree models as analytical models, or for which the resulting item score might be viewed as outcomes of such a process, we recommend (1) regular inspection of data or analytic results for empirical evidence (or theoretical expectations) of item-specific factors; and (2) sensitivity analyses to evaluate the implications of item-specific factors for the intended inferences or applications.

Suggested Citation

  • Weicong Lyu & Daniel M. Bolt & Samuel Westby, 2023. "Exploring the Effects of Item-Specific Factors in Sequential and IRTree Models," Psychometrika, Springer;The Psychometric Society, vol. 88(3), pages 745-775, September.
  • Handle: RePEc:spr:psycho:v:88:y:2023:i:3:d:10.1007_s11336-023-09912-x
    DOI: 10.1007/s11336-023-09912-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11336-023-09912-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11336-023-09912-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bas Hemker & L. Andries van der Ark & Klaas Sijtsma, 2001. "On measurement properties of continuation ratio models," Psychometrika, Springer;The Psychometric Society, vol. 66(4), pages 487-506, December.
    2. Timo Bechger & Wies Akkermans, 2001. "A note on the equivalence of the graded response model and the sequential model," Psychometrika, Springer;The Psychometric Society, vol. 66(3), pages 461-463, September.
    3. Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521766555, October.
    4. Fumiko Samejima, 1995. "Acceleration model in the heterogeneous case of the general graded response model," Psychometrika, Springer;The Psychometric Society, vol. 60(4), pages 549-572, December.
    5. Geoff Masters, 1982. "A rasch model for partial credit scoring," Psychometrika, Springer;The Psychometric Society, vol. 47(2), pages 149-174, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Thorsten Meiser & Fabiola Reiber, 2023. "Item-Specific Factors in IRTree Models: When They Matter and When They Don’t," Psychometrika, Springer;The Psychometric Society, vol. 88(3), pages 739-744, September.
    2. Weicong Lyu & Daniel M. Bolt, 2023. "Rejoinder to Commentaries on Lyu, Bolt and Westby’s “Exploring the Effects of Item Specific Factors in Sequential and IRTree Models”," Psychometrika, Springer;The Psychometric Society, vol. 88(3), pages 1026-1031, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stefano Noventa & Andrea Spoto & Jürgen Heller & Augustin Kelava, 2019. "On a Generalization of Local Independence in Item Response Theory Based on Knowledge Space Theory," Psychometrika, Springer;The Psychometric Society, vol. 84(2), pages 395-421, June.
    2. Fumiko Samejima, 1997. "Departure from normal assumptions: A promise for future psychometrics with substantive mathematical modeling," Psychometrika, Springer;The Psychometric Society, vol. 62(4), pages 471-493, December.
    3. Jesper Tijmstra & Maria Bolsinova, 2019. "Bayes Factors for Evaluating Latent Monotonicity in Polytomous Item Response Theory Models," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 846-869, September.
    4. Ligtvoet, R., 2015. "A test for using the sum score to obtain a stochastic ordering of subjects," Journal of Multivariate Analysis, Elsevier, vol. 133(C), pages 136-139.
    5. Anders Skrondal & Sophia Rabe‐Hesketh, 2007. "Latent Variable Modelling: A Survey," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 34(4), pages 712-745, December.
    6. L. Ark & Wicher Bergsma, 2010. "A Note on Stochastic Ordering of the Latent Trait Using the Sum of Polytomous Item Scores," Psychometrika, Springer;The Psychometric Society, vol. 75(2), pages 272-279, June.
    7. Bas Hemker & L. Andries van der Ark & Klaas Sijtsma, 2001. "On measurement properties of continuation ratio models," Psychometrika, Springer;The Psychometric Society, vol. 66(4), pages 487-506, December.
    8. van der Ark, L.A., 1999. "A reference card for the relationships between IRT models for ordered polytomous items and some relevant properties," WORC Paper 99.10.02, Tilburg University, Work and Organization Research Centre.
    9. Gerhard Tutz, 2021. "Hierarchical Models for the Analysis of Likert Scales in Regression and Item Response Analysis," International Statistical Review, International Statistical Institute, vol. 89(1), pages 18-35, April.
    10. Wenchao Ma & Jimmy de la Torre, 2019. "Category-Level Model Selection for the Sequential G-DINA Model," Journal of Educational and Behavioral Statistics, , vol. 44(1), pages 45-77, February.
    11. Gerhard Tutz, 2022. "Item Response Thresholds Models: A General Class of Models for Varying Types of Items," Psychometrika, Springer;The Psychometric Society, vol. 87(4), pages 1238-1269, December.
    12. Rudy Ligtvoet & L. Ark & Wicher Bergsma & Klaas Sijtsma, 2011. "Polytomous Latent Scales for the Investigation of the Ordering of Items," Psychometrika, Springer;The Psychometric Society, vol. 76(2), pages 200-216, April.
    13. Dan Yavorsky & Elisabeth Honka & Keith Chen, 2021. "Consumer search in the U.S. auto industry: The role of dealership visits," Quantitative Marketing and Economics (QME), Springer, vol. 19(1), pages 1-52, March.
    14. Zhifeng Gao & Ted C. Schroeder, 2009. "Consumer responses to new food quality information: are some consumers more sensitive than others?," Agricultural Economics, International Association of Agricultural Economists, vol. 40(3), pages 339-346, May.
    15. Cheng, Leilei & Yin, Changbin & Chien, Hsiaoping, 2015. "Demand for milk quantity and safety in urban China: evidence from Beijing and Harbin," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 59(2), April.
    16. Sebastian Heidenreich & Andrea Phillips-Beyer & Bruno Flamion & Melissa Ross & Jaein Seo & Kevin Marsh, 2021. "Benefit–Risk or Risk–Benefit Trade-Offs? Another Look at Attribute Ordering Effects in a Pilot Choice Experiment," The Patient: Patient-Centered Outcomes Research, Springer;International Academy of Health Preference Research, vol. 14(1), pages 65-74, January.
    17. Wen, Chieh-Hua & Huang, Chia-Jung & Fu, Chiang, 2020. "Incorporating continuous representation of preferences for flight departure times into stated itinerary choice modeling," Transport Policy, Elsevier, vol. 98(C), pages 10-20.
    18. Frith, Michael J., 2019. "Modelling taste heterogeneity regarding offence location choices," Journal of choice modelling, Elsevier, vol. 33(C).
    19. Johannes Buggle & Thierry Mayer & Seyhun Orcan Sakalli & Mathias Thoenig, 2023. "The Refugee’s Dilemma: Evidence from Jewish Migration out of Nazi Germany," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(2), pages 1273-1345.
    20. Boneva, Teodora & Golin, Marta & Kaufmann, Katja Maria & Rauh, Christopher, 2022. "Beliefs about Maternal Labor Supply," IZA Discussion Papers 15788, Institute of Labor Economics (IZA).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:88:y:2023:i:3:d:10.1007_s11336-023-09912-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.