IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v80y2015i2p317-340.html
   My bibliography  Save this article

A Statistical Test for Differential Item Pair Functioning

Author

Listed:
  • Timo Bechger
  • Gunter Maris

Abstract

This paper presents an IRT-based statistical test for differential item functioning (DIF). The test is developed for items conforming to the Rasch (Probabilistic models for some intelligence and attainment tests, The Danish Institute of Educational Research, Copenhagen, 1960 ) model but we will outline its extension to more complex IRT models. Its difference from the existing procedures is that DIF is defined in terms of the relative difficulties of pairs of items and not in terms of the difficulties of individual items. The argument is that the difficulty of an item is not identified from the observations, whereas the relative difficulties are. This leads to a test that is closely related to Lord’s (Applications of item response theory to practical testing problems, Erlbaum, Hillsdale, 1980 ) test for item DIF albeit with a different and more correct interpretation. Illustrations with real and simulated data are provided. Copyright The Psychometric Society 2015

Suggested Citation

  • Timo Bechger & Gunter Maris, 2015. "A Statistical Test for Differential Item Pair Functioning," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 317-340, June.
    • Handle: RePEc:spr:psycho:v:80:y:2015:i:2:p:317-340
    DOI: 10.1007/s11336-014-9408-y
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s11336-014-9408-y
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11336-014-9408-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mark Wilson & Raymond Adams, 1995. "Rasch models for item bundles," Psychometrika, Springer;The Psychometric Society, vol. 60(2), pages 181-198, June.
    2. Geoff Masters, 1982. "A rasch model for partial credit scoring," Psychometrika, Springer;The Psychometric Society, vol. 47(2), pages 149-174, June.
    3. William Stout, 2002. "Psychometrics: From practice to theory and back," Psychometrika, Springer;The Psychometric Society, vol. 67(4), pages 485-518, December.
    4. Ivo Ponocny, 2001. "Nonparametric goodness-of-fit tests for the rasch model," Psychometrika, Springer;The Psychometric Society, vol. 66(3), pages 437-459, September.
    5. Paul Boeck, 2008. "Random Item IRT Models," Psychometrika, Springer;The Psychometric Society, vol. 73(4), pages 533-559, December.
    6. David Hessen, 2005. "Constant latent odds-ratios models and the mantel-haenszel null hypothesis," Psychometrika, Springer;The Psychometric Society, vol. 70(3), pages 497-516, September.
    7. Nambury Raju, 1988. "The area between two item characteristic curves," Psychometrika, Springer;The Psychometric Society, vol. 53(4), pages 495-502, December.
    8. Gabrielsen, Arne, 1978. "Consistency and identifiability," Journal of Econometrics, Elsevier, vol. 8(2), pages 261-263, October.
    9. Norman Verhelst, 2008. "An Efficient MCMC Algorithm to Sample Binary Matrices with Fixed Marginals," Psychometrika, Springer;The Psychometric Society, vol. 73(4), pages 705-728, December.
    10. Verhelst, Norman & Hatzinger, Reinhold & Mair, Patrick, 2007. "The Rasch Sampler," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 20(i04).
    11. Erling Andersen, 1973. "A goodness of fit test for the rasch model," Psychometrika, Springer;The Psychometric Society, vol. 38(1), pages 123-140, March.
    12. Erling Andersen, 1977. "Sufficient statistics and latent trait models," Psychometrika, Springer;The Psychometric Society, vol. 42(1), pages 69-81, March.
    13. William Meredith & Roger Millsap, 1992. "On the misuse of manifest variables in the detection of measurement bias," Psychometrika, Springer;The Psychometric Society, vol. 57(2), pages 289-311, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ting Wang & Carolin Strobl & Achim Zeileis & Edgar C. Merkle, 2018. "Score-Based Tests of Differential Item Functioning via Pairwise Maximum Likelihood Estimation," Psychometrika, Springer;The Psychometric Society, vol. 83(1), pages 132-155, March.
    2. Robert J. Zwitser & S. Sjoerd F. Glaser & Gunter Maris, 2017. "Monitoring Countries in a Changing World: A New Look at DIF in International Surveys," Psychometrika, Springer;The Psychometric Society, vol. 82(1), pages 210-232, March.
    3. Peter F. Halpin, 2024. "Differential Item Functioning via Robust Scaling," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 796-821, September.
    4. Paul Boeck & Sun-Joo Cho, 2021. "Not all DIF is shaped similarly," Psychometrika, Springer;The Psychometric Society, vol. 86(3), pages 712-716, September.
    5. K. B. S. Huth & L. J. Waldorp & J. Luigjes & A. E. Goudriaan & R. J. Holst & M. Marsman, 2022. "A Note on the Structural Change Test in Highly Parameterized Psychometric Models," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 1064-1080, September.
    6. Alexander Robitzsch, 2020. "L p Loss Functions in Invariance Alignment and Haberman Linking with Few or Many Groups," Stats, MDPI, vol. 3(3), pages 1-38, August.
    7. Chen, Yunxiao & Li, Chengcheng & Ouyang, Jing & Xu, Gongjun, 2023. "DIF statistical inference without knowing anchoring items," LSE Research Online Documents on Economics 119923, London School of Economics and Political Science, LSE Library.
    8. Wallin, Gabriel & Chen, Yunxiao & Moustaki, Irini, 2024. "DIF analysis with unknown groups and anchor items," LSE Research Online Documents on Economics 121991, London School of Economics and Political Science, LSE Library.
    9. Alexander Robitzsch & Oliver Lüdtke, 2022. "Mean Comparisons of Many Groups in the Presence of DIF: An Evaluation of Linking and Concurrent Scaling Approaches," Journal of Educational and Behavioral Statistics, , vol. 47(1), pages 36-68, February.
    10. Ke-Hai Yuan & Hongyun Liu & Yuting Han, 2021. "Differential Item Functioning Analysis Without A Priori Information on Anchor Items: QQ Plots and Graphical Test," Psychometrika, Springer;The Psychometric Society, vol. 86(2), pages 345-377, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Clemens Draxler, 2010. "Sample Size Determination for Rasch Model Tests," Psychometrika, Springer;The Psychometric Society, vol. 75(4), pages 708-724, December.
    2. Clemens Draxler, 2018. "Bayesian conditional inference for Rasch models," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 102(2), pages 245-262, April.
    3. Gerhard Tutz & Gunther Schauberger, 2015. "A Penalty Approach to Differential Item Functioning in Rasch Models," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 21-43, March.
    4. Georg Gittler & Gerhard Fischer, 2011. "IRT-Based Measurement of Short-Term Changes of Ability, With an Application to Assessing the “Mozart Effectâ€," Journal of Educational and Behavioral Statistics, , vol. 36(1), pages 33-75, February.
    5. Clemens Draxler & Johannes Zessin, 2015. "The power function of conditional tests of the Rasch model," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 99(3), pages 367-378, July.
    6. Clemens Draxler & Andreas Kurz & Can Gürer & Jan Philipp Nolte, 2024. "An Improved Inferential Procedure to Evaluate Item Discriminations in a Conditional Maximum Likelihood Framework," Journal of Educational and Behavioral Statistics, , vol. 49(3), pages 403-430, June.
    7. Clemens Draxler & Rainer Alexandrowicz, 2015. "Sample Size Determination Within the Scope of Conditional Maximum Likelihood Estimation with Special Focus on Testing the Rasch Model," Psychometrika, Springer;The Psychometric Society, vol. 80(4), pages 897-919, December.
    8. Jürgen M. Pelikan & Thomas Link & Christa Straßmayr & Karin Waldherr & Tobias Alfers & Henrik Bøggild & Robert Griebler & Maria Lopatina & Dominika Mikšová & Marie Germund Nielsen & Sandra Peer & Mitj, 2022. "Measuring Comprehensive, General Health Literacy in the General Adult Population: The Development and Validation of the HLS 19 -Q12 Instrument in Seventeen Countries," IJERPH, MDPI, vol. 19(21), pages 1-31, October.
    9. Jean-Baptiste Gaborieau & Cristina Pronello, 2021. "Validation of a unidimensional and probabilistic measurement scale for pro-environmental behaviour by travellers," Transportation, Springer, vol. 48(2), pages 555-593, April.
    10. Rudy Ligtvoet, 2022. "Incomplete Tests of Conditional Association for the Assessment of Model Assumptions," Psychometrika, Springer;The Psychometric Society, vol. 87(4), pages 1214-1237, December.
    11. Roberto Burro & Riccardo Sartori & Giulio Vidotto, 2011. "The method of constant stimuli with three rating categories and the use of Rasch models," Quality & Quantity: International Journal of Methodology, Springer, vol. 45(1), pages 43-58, January.
    12. Antonio Caronni & Marina Ramella & Pietro Arcuri & Claudia Salatino & Lucia Pigini & Maurizio Saruggia & Chiara Folini & Stefano Scarano & Rosa Maria Converti, 2023. "The Rasch Analysis Shows Poor Construct Validity and Low Reliability of the Quebec User Evaluation of Satisfaction with Assistive Technology 2.0 (QUEST 2.0) Questionnaire," IJERPH, MDPI, vol. 20(2), pages 1-19, January.
    13. David J. Hessen, 2023. "Fitting and Testing Log-Linear Subpopulation Models with Known Support," Psychometrika, Springer;The Psychometric Society, vol. 88(3), pages 917-939, September.
    14. C. Glas & Anna Dagohoy, 2007. "A Person Fit Test For Irt Models For Polytomous Items," Psychometrika, Springer;The Psychometric Society, vol. 72(2), pages 159-180, June.
    15. Jerry Welkenhuysen-Gybels, 2004. "The Performance of Some Observed and Unobserved Conditional Invariance Techniques for the Detection of Differential Item Functioning," Quality & Quantity: International Journal of Methodology, Springer, vol. 38(6), pages 681-702, December.
    16. repec:jss:jstsof:39:i12 is not listed on IDEAS
    17. David Andrich, 2010. "Sufficiency and Conditional Estimation of Person Parameters in the Polytomous Rasch Model," Psychometrika, Springer;The Psychometric Society, vol. 75(2), pages 292-308, June.
    18. Luz Dary Upegui-Arango & Thomas Forkmann & Tine Nielsen & Nina Hallensleben & Heide Glaesmer & Lena Spangenberg & Tobias Teismann & Georg Juckel & Maren Boecker, 2020. "Psychometric evaluation of the Interpersonal Needs Questionnaire (INQ) using item analysis according to the Rasch model," PLOS ONE, Public Library of Science, vol. 15(8), pages 1-21, August.
    19. Minjeong Jeon & Frank Rijmen & Sophia Rabe-Hesketh, 2013. "Modeling Differential Item Functioning Using a Generalization of the Multiple-Group Bifactor Model," Journal of Educational and Behavioral Statistics, , vol. 38(1), pages 32-60, February.
    20. Robert Zwitser & Gunter Maris, 2015. "Conditional Statistical Inference with Multistage Testing Designs," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 65-84, March.
    21. Clemens Draxler & Andreas Kurz, 2021. "Conditional Inference in Small Sample Scenarios Using a Resampling Approach," Stats, MDPI, vol. 4(4), pages 1-13, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:80:y:2015:i:2:p:317-340. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.