IDEAS home Printed from https://ideas.repec.org/p/ehl/lserod/119923.html
   My bibliography  Save this paper

DIF statistical inference without knowing anchoring items

Author

Listed:
  • Chen, Yunxiao
  • Li, Chengcheng
  • Ouyang, Jing
  • Xu, Gongjun

Abstract

Establishing the invariance property of an instrument (e.g., a questionnaire or test) is a key step for establishing its measurement validity. Measurement invariance is typically assessed by differential item functioning (DIF) analysis, i.e., detecting DIF items whose response distribution depends not only on the latent trait measured by the instrument but also on the group membership. DIF analysis is confounded by the group difference in the latent trait distributions. Many DIF analyses require knowing several anchor items that are DIF-free in order to draw inferences on whether each of the rest is a DIF item, where the anchor items are used to identify the latent trait distributions. When no prior information on anchor items is available, or some anchor items are misspecified, item purification methods and regularized estimation methods can be used. The former iteratively purifies the anchor set by a stepwise model selection procedure, and the latter selects the DIF-free items by a LASSO-type regularization approach. Unfortunately, unlike the methods based on a correctly specified anchor set, these methods are not guaranteed to provide valid statistical inference (e.g., confidence intervals and p-values). In this paper, we propose a new method for DIF analysis under a multiple indicators and multiple causes (MIMIC) model for DIF. This method adopts a minimal L 1 norm condition for identifying the latent trait distributions. Without requiring prior knowledge about an anchor set, it can accurately estimate the DIF effects of individual items and further draw valid statistical inferences for quantifying the uncertainty. Specifically, the inference results allow us to control the type-I error for DIF detection, which may not be possible with item purification and regularized estimation methods. We conduct simulation studies to evaluate the performance of the proposed method and compare it with the anchor-set-based likelihood ratio test approach and the LASSO approach. The proposed method is applied to analysing the three personality scales of the Eysenck personality questionnaire-revised (EPQ-R).

Suggested Citation

  • Chen, Yunxiao & Li, Chengcheng & Ouyang, Jing & Xu, Gongjun, 2023. "DIF statistical inference without knowing anchoring items," LSE Research Online Documents on Economics 119923, London School of Economics and Political Science, LSE Library.
  • Handle: RePEc:ehl:lserod:119923
    as

    Download full text from publisher

    File URL: http://eprints.lse.ac.uk/119923/
    File Function: Open access version.
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Robin Shealy & William Stout, 1993. "A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF," Psychometrika, Springer;The Psychometric Society, vol. 58(2), pages 159-194, June.
    2. Alexander Robitzsch, 2020. "L p Loss Functions in Invariance Alignment and Haberman Linking with Few or Many Groups," Stats, MDPI, vol. 3(3), pages 1-38, August.
    3. Gerhard Tutz & Gunther Schauberger, 2015. "A Penalty Approach to Differential Item Functioning in Rasch Models," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 21-43, March.
    4. Nambury Raju, 1988. "The area between two item characteristic curves," Psychometrika, Springer;The Psychometric Society, vol. 53(4), pages 495-502, December.
    5. Carolin Strobl & Julia Kopf & Achim Zeileis, 2015. "Rasch Trees: A New Method for Detecting Differential Item Functioning in the Rasch Model," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 289-316, June.
    6. Timo Bechger & Gunter Maris, 2015. "A Statistical Test for Differential Item Pair Functioning," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 317-340, June.
    7. Gerhard Tutz & Moritz Berger, 2016. "Item-focussed Trees for the Identification of Items in Differential Item Functioning," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 727-750, September.
    8. Goldberger, Arthur S, 1972. "Structural Equation Methods in the Social Sciences," Econometrica, Econometric Society, vol. 40(6), pages 979-1001, November.
    9. Steenkamp, Jan-Benedict E M & Baumgartner, Hans, 1998. "Assessing Measurement Invariance in Cross-National Consumer Research," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 25(1), pages 78-90, June.
    10. Ke-Hai Yuan & Hongyun Liu & Yuting Han, 2021. "Differential Item Functioning Analysis Without A Priori Information on Anchor Items: QQ Plots and Graphical Test," Psychometrika, Springer;The Psychometric Society, vol. 86(2), pages 345-377, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wallin, Gabriel & Chen, Yunxiao & Moustaki, Irini, 2024. "DIF analysis with unknown groups and anchor items," LSE Research Online Documents on Economics 121991, London School of Economics and Political Science, LSE Library.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ke-Hai Yuan & Hongyun Liu & Yuting Han, 2021. "Differential Item Functioning Analysis Without A Priori Information on Anchor Items: QQ Plots and Graphical Test," Psychometrika, Springer;The Psychometric Society, vol. 86(2), pages 345-377, June.
    2. Wallin, Gabriel & Chen, Yunxiao & Moustaki, Irini, 2024. "DIF analysis with unknown groups and anchor items," LSE Research Online Documents on Economics 121991, London School of Economics and Political Science, LSE Library.
    3. Ting Wang & Carolin Strobl & Achim Zeileis & Edgar C. Merkle, 2018. "Score-Based Tests of Differential Item Functioning via Pairwise Maximum Likelihood Estimation," Psychometrika, Springer;The Psychometric Society, vol. 83(1), pages 132-155, March.
    4. Alexander Robitzsch, 2020. "L p Loss Functions in Invariance Alignment and Haberman Linking with Few or Many Groups," Stats, MDPI, vol. 3(3), pages 1-38, August.
    5. Gerhard Tutz & Moritz Berger, 2016. "Item-focussed Trees for the Identification of Items in Differential Item Functioning," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 727-750, September.
    6. Paul Boeck & Sun-Joo Cho, 2021. "Not all DIF is shaped similarly," Psychometrika, Springer;The Psychometric Society, vol. 86(3), pages 712-716, September.
    7. Jeanne A. Teresi & Chun Wang & Marjorie Kleinman & Richard N. Jones & David J. Weiss, 2021. "Differential Item Functioning Analyses of the Patient-Reported Outcomes Measurement Information System (PROMIS®) Measures: Methods, Challenges, Advances, and Future Directions," Psychometrika, Springer;The Psychometric Society, vol. 86(3), pages 674-711, September.
    8. Youmi Suk & Kyung T. Han, 2024. "A Psychometric Framework for Evaluating Fairness in Algorithmic Decision Making: Differential Algorithmic Functioning," Journal of Educational and Behavioral Statistics, , vol. 49(2), pages 151-172, April.
    9. Minjeong Jeon & Frank Rijmen & Sophia Rabe-Hesketh, 2013. "Modeling Differential Item Functioning Using a Generalization of the Multiple-Group Bifactor Model," Journal of Educational and Behavioral Statistics, , vol. 38(1), pages 32-60, February.
    10. Jorgensen, Bradley S. & Syme, Geoffrey J., 2000. "Protest responses and willingness to pay: attitude toward paying for stormwater pollution abatement," Ecological Economics, Elsevier, vol. 33(2), pages 251-265, May.
    11. David Magis & Francis Tuerlinckx & Paul De Boeck, 2015. "Detection of Differential Item Functioning Using the Lasso Approach," Journal of Educational and Behavioral Statistics, , vol. 40(2), pages 111-135, April.
    12. K. B. S. Huth & L. J. Waldorp & J. Luigjes & A. E. Goudriaan & R. J. Holst & M. Marsman, 2022. "A Note on the Structural Change Test in Highly Parameterized Psychometric Models," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 1064-1080, September.
    13. Peter F. Halpin, 2024. "Differential Item Functioning via Robust Scaling," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 796-821, September.
    14. Tsukasa Kato, 2021. "Measurement Invariance in the Center for Epidemiologic Studies-Depression (CES-D) Scale among English-Speaking Whites and Asians," IJERPH, MDPI, vol. 18(10), pages 1-10, May.
    15. Mao, Lu, 2022. "Identification of the outcome distribution and sensitivity analysis under weak confounder–instrument interaction," Statistics & Probability Letters, Elsevier, vol. 189(C).
    16. Janina Isabel Steinert & Lucie Dale Cluver & G. J. Melendez-Torres & Sebastian Vollmer, 2018. "One Size Fits All? The Validity of a Composite Poverty Index Across Urban and Rural Households in South Africa," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 136(1), pages 51-72, February.
    17. Yeung, Matthew C.H. & Ramasamy, Bala & Chen, Junsong & Paliwoda, Stan, 2013. "Customer satisfaction and consumer expenditure in selected European countries," International Journal of Research in Marketing, Elsevier, vol. 30(4), pages 406-416.
    18. Matanda, Margaret Jekanyika & Freeman, Susan, 2009. "Effect of perceived environmental uncertainty on exporter-importer inter-organisational relationships and export performance improvement," International Business Review, Elsevier, vol. 18(1), pages 89-107, February.
    19. Magnusson, Leandro M. & Tarverdi, Yashar, 2020. "Measuring governance: Why do errors matter?," World Development, Elsevier, vol. 136(C).
    20. Alexander Robitzsch, 2024. "Bias-Reduced Haebara and Stocking–Lord Linking," J, MDPI, vol. 7(3), pages 1-12, September.

    More about this item

    Keywords

    confidence interval; differential item functioning; item response theory; least absolute deviations; measurement invariance;
    All these keywords.

    JEL classification:

    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ehl:lserod:119923. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: LSERO Manager (email available below). General contact details of provider: https://edirc.repec.org/data/lsepsuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.