IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v45y2020i2p209-226.html
   My bibliography  Save this article

Scoring Tests With Contaminated Response Vectors

Author

Listed:
  • Arnond Sakworawich

    (National Institute of Development Administration)

  • Howard Wainer

    (Pennington, New Jersey)

Abstract

Test scoring models vary in their generality, some even adjust for examinees answering multiple-choice items correctly by accident (guessing), but no models, that we are aware of, automatically adjust an examinee’s score when there is internal evidence of cheating. In this study, we use a combination of jackknife technology with an adaptive robust estimator to reduce the bias in examinee scores due to contamination through events such as having access to some of the test items in advance of the test administration. We illustrate our methodology with a data set of test items we knew to have been divulged to a subset of the examinees.

Suggested Citation

  • Arnond Sakworawich & Howard Wainer, 2020. "Scoring Tests With Contaminated Response Vectors," Journal of Educational and Behavioral Statistics, , vol. 45(2), pages 209-226, April.
    • Handle: RePEc:sae:jedbes:v:45:y:2020:i:2:p:209-226
    DOI: 10.3102/1076998619882902
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998619882902
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998619882902?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Rizopoulos, Dimitris, 2006. "ltm: An R Package for Latent Variable Modeling and Item Response Analysis," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 17(i05).
    2. Howard Wainer & Benjamin Wright, 1980. "Robust estimation of ability in the Rasch model," Psychometrika, Springer;The Psychometric Society, vol. 45(3), pages 373-391, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang Yixin & Lü Xin & Ma Jian & Qiao Han, 2014. "A Robust Factor Analysis Model for Dichotomous Data," Journal of Systems Science and Information, De Gruyter, vol. 2(5), pages 437-450, October.
    2. Arulmani Thiyagarajan & Tyler G. James & Roy Rillera Marzo, 2022. "Psychometric properties of the 21-item Depression, Anxiety, and Stress Scale (DASS-21) among Malaysians during COVID-19: a methodological study," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-8, December.
    3. Cervantes, Víctor H., 2017. "DFIT: An R Package for Raju's Differential Functioning of Items and Tests Framework," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 76(i05).
    4. Andrew Karl & Randy Eubank & Jelena Milovanovic & Mark Reiser & Dennis Young, 2014. "Using RngStreams for parallel random number generation in C++ and R," Computational Statistics, Springer, vol. 29(5), pages 1301-1320, October.
    5. Sora Lee & Daniel M. Bolt, 2018. "Asymmetric Item Characteristic Curves and Item Complexity: Insights from Simulation and Real Data Analyses," Psychometrika, Springer;The Psychometric Society, vol. 83(2), pages 453-475, June.
    6. Nick Bailey & Anne-Catherine Guio, 2022. "Adaptive Deprivation Scales in a Multi-National Context: The European Child Deprivation Indicators," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 15(6), pages 2335-2362, December.
    7. Dawit G. Ayele & Temesgen Zewotir & Henry Mwambi, 2014. "Using Rasch Modeling to Re-Evaluate Rapid Malaria Diagnosis Test Analyses," IJERPH, MDPI, vol. 11(7), pages 1-11, June.
    8. Paula Fariña & Jorge González & Ernesto San Martín, 2019. "The Use of an Identifiability-Based Strategy for the Interpretation of Parameters in the 1PL-G and Rasch Models," Psychometrika, Springer;The Psychometric Society, vol. 84(2), pages 511-528, June.
    9. Mark Alfano & Kathryn Iurino & Paul Stey & Brian Robinson & Markus Christen & Feng Yu & Daniel Lapsley, 2017. "Development and validation of a multi-dimensional measure of intellectual humility," PLOS ONE, Public Library of Science, vol. 12(8), pages 1-28, August.
    10. Francisco José Eiroa-Orosa & Laura Limiñana-Bravo, 2019. "An Instrument to Measure Mental Health Professionals’ Beliefs and Attitudes towards Service Users’ Rights," IJERPH, MDPI, vol. 16(2), pages 1-16, January.
    11. Jinshu Cui & Heather Rosoff & Richard S. John, 2017. "A Polytomous Item Response Theory Model for Measuring Near-Miss Appraisal as a Psychological Trait," Decision Analysis, INFORMS, vol. 14(2), pages 75-86, June.
    12. Lindsey W. Vilca & Evelyn L. Chambi-Mamani & Emely D. Quispe-Kana & Mónica Hernández-López & Tomás Caycho-Rodríguez, 2022. "Functioning of the EROS-R Scale in a Clinical Sample of Psychiatric Patients: New Psychometric Evidence from the Classical Test Theory and the Item Response Theory," IJERPH, MDPI, vol. 19(16), pages 1-14, August.
    13. Fabrizio Maturo & Francesca Fortuna & Tonio Di Battista, 2019. "Testing Equality of Functions Across Multiple Experimental Conditions for Different Ability Levels in the IRT Context: The Case of the IPRASE TLT 2016 Survey," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 146(1), pages 19-39, November.
    14. Ellen Bernadette Maria Elsman & Gerardus Hermanus Maria Bartholomeus van Rens & Ruth Marie Antoinette van Nispen, 2018. "Psychometric properties of a new intake questionnaire for visually impaired young adults: The Participation and Activity Inventory for Young Adults (PAI-YA)," PLOS ONE, Public Library of Science, vol. 13(8), pages 1-24, August.
    15. Herbert Hoijtink, 1990. "A latent trait model for dichotomous choice data," Psychometrika, Springer;The Psychometric Society, vol. 55(4), pages 641-656, December.
    16. Pedro Mateu & Enrique Vásquez & Javier Zúñiga & Franklin Ibáñez, 2020. "Happiness and poverty in the very poor Peru: measurement improvements and a consistent relationship," Quality & Quantity: International Journal of Methodology, Springer, vol. 54(3), pages 1075-1094, June.
    17. Grand, James A. & Golubovich, Juliya & Ryan, Ann Marie & Schmitt, Neal, 2013. "The detection and influence of problematic item content in ability tests: An examination of sensitivity review practices for personnel selection test development," Organizational Behavior and Human Decision Processes, Elsevier, vol. 121(2), pages 158-173.
    18. David Magis, 2016. "Efficient Standard Error Formulas of Ability Estimators with Dichotomous Item Response Models," Psychometrika, Springer;The Psychometric Society, vol. 81(1), pages 184-200, March.
    19. Christof Schuster & Ke-Hai Yuan, 2011. "Robust Estimation of Latent Ability in Item Response Models," Journal of Educational and Behavioral Statistics, , vol. 36(6), pages 720-735, December.
    20. John Patrick Lalor & Pedro Rodriguez, 2023. "py-irt : A Scalable Item Response Theory Library for Python," INFORMS Journal on Computing, INFORMS, vol. 35(1), pages 5-13, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:45:y:2020:i:2:p:209-226. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.