IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v157y2021ics0167947321000116.html
   My bibliography  Save this article

An exchange algorithm for optimal calibration of items in computerized achievement tests

Author

Listed:
  • Ul Hassan, Mahmood
  • Miller, Frank

Abstract

The importance of large scale achievement tests, like national tests in school, eligibility tests for university, or international assessments for evaluation of students, is increasing. Pretesting of questions for the above mentioned tests is done to determine characteristic properties of the questions by adding them to an ordinary achievement test. If computerized tests are used, it has been shown using optimal experimental design methods that it is efficient to assign pretest questions to examinees based on their abilities. The specific distribution of abilities of the available examinees are considered and restricted optimal designs are applied. A new algorithm is developed which builds on an equivalence theorem. It discretizes the design space with the possibility to change the grid adaptively during the run, makes use of an exchange idea and filters computed designs. It is illustrated how the algorithm works through some examples as well as how convergence can be checked. The new algorithm is flexible and can be used even if different models are assumed for different questions.

Suggested Citation

  • Ul Hassan, Mahmood & Miller, Frank, 2021. "An exchange algorithm for optimal calibration of items in computerized achievement tests," Computational Statistics & Data Analysis, Elsevier, vol. 157(C).
    • Handle: RePEc:eee:csdana:v:157:y:2021:i:c:s0167947321000116
    DOI: 10.1016/j.csda.2021.107177
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947321000116
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2021.107177?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hung-Yi Lu, 2014. "Application of Optimal Designs to Item Calibration," PLOS ONE, Public Library of Science, vol. 9(9), pages 1-8, September.
    2. Mahmood Ul Hassan & Frank Miller, 2019. "Optimal Item Calibration for Computerized Achievement Tests," Psychometrika, Springer;The Psychometric Society, vol. 84(4), pages 1101-1128, December.
    3. Hao Ren & Wim J. van der Linden & Qi Diao, 2017. "Continuous Online Item Calibration: Parameter Recovery and Item Utilization," Psychometrika, Springer;The Psychometric Society, vol. 82(2), pages 498-522, June.
    4. Martijn Berger, 1992. "Sequential sampling designs for the two-parameter item response theory model," Psychometrika, Springer;The Psychometric Society, vol. 57(4), pages 521-538, December.
    5. Yuanzhi Huang & Steven G. Gilmour & Kalliopi Mylona & Peter Goos, 2019. "Optimal design of experiments for non‐linear response surface models," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 68(3), pages 623-640, April.
    6. Martijn Berger & C. Joy King & Weng Wong, 2000. "Minimax d-optimal designs for item response theory models," Psychometrika, Springer;The Psychometric Society, vol. 65(3), pages 377-390, September.
    7. Douglas Jones & Zhiying Jin, 1994. "Optimal sequential designs for on-line item estimation," Psychometrika, Springer;The Psychometric Society, vol. 59(1), pages 59-75, March.
    8. Wim Linden & Hao Ren, 2015. "Optimal Bayesian Adaptive Design for Test-Item Calibration," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 263-288, June.
    9. Yuan-chin Chang & Hung-Yi Lu, 2010. "Online Calibration Via Variable Length Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 75(1), pages 140-157, March.
    10. García-Ródenas, Ricardo & García-García, José Carlos & López-Fidalgo, Jesús & Martín-Baos, José Ángel & Wong, Weng Kee, 2020. "A comparison of general-purpose optimization algorithms for finding optimal approximate experimental designs," Computational Statistics & Data Analysis, Elsevier, vol. 144(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mahmood Ul Hassan & Frank Miller, 2019. "Optimal Item Calibration for Computerized Achievement Tests," Psychometrika, Springer;The Psychometric Society, vol. 84(4), pages 1101-1128, December.
    2. Yinhong He & Ping Chen, 2020. "Optimal Online Calibration Designs for Item Replenishment in Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 85(1), pages 35-55, March.
    3. Wim Linden & Hao Ren, 2015. "Optimal Bayesian Adaptive Design for Test-Item Calibration," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 263-288, June.
    4. Hyeon-Ah Kang & Yi Zheng & Hua-Hua Chang, 2020. "Online Calibration of a Joint Model of Item Responses and Response Times in Computerized Adaptive Testing," Journal of Educational and Behavioral Statistics, , vol. 45(2), pages 175-208, April.
    5. Yuan-chin Chang & Hung-Yi Lu, 2010. "Online Calibration Via Variable Length Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 75(1), pages 140-157, March.
    6. Hao Ren & Wim J. van der Linden & Qi Diao, 2017. "Continuous Online Item Calibration: Parameter Recovery and Item Utilization," Psychometrika, Springer;The Psychometric Society, vol. 82(2), pages 498-522, June.
    7. Wim J. van der Linden & Bingnan Jiang, 2020. "A Shadow-Test Approach to Adaptive Item Calibration," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 301-321, June.
    8. Ping Chen, 2017. "A Comparative Study of Online Item Calibration Methods in Multidimensional Computerized Adaptive Testing," Journal of Educational and Behavioral Statistics, , vol. 42(5), pages 559-590, October.
    9. Yuan-chin Chang, 2011. "Sequential estimation in generalized linear models when covariates are subject to errors," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 73(1), pages 93-120, January.
    10. Ping Chen & Chun Wang, 2016. "A New Online Calibration Method for Multidimensional Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 674-701, September.
    11. Chiara Tommasi & Juan M. Rodríguez-Díaz & Jesús F. López-Fidalgo, 2023. "An equivalence theorem for design optimality with respect to a multi-objective criterion," Statistical Papers, Springer, vol. 64(4), pages 1041-1056, August.
    12. Rios, Nicholas & Winker, Peter & Lin, Dennis K.J., 2022. "TA algorithms for D-optimal OofA Mixture designs," Computational Statistics & Data Analysis, Elsevier, vol. 168(C).
    13. Ping Chen & Tao Xin & Chun Wang & Hua-Hua Chang, 2012. "Online Calibration Methods for the DINA Model with Independent Attributes in CD-CAT," Psychometrika, Springer;The Psychometric Society, vol. 77(2), pages 201-222, April.
    14. Zimu Chen & Zhanfeng Wang & Yuan‐chin Ivan Chang, 2020. "Sequential adaptive variables and subject selection for GEE methods," Biometrics, The International Biometric Society, vol. 76(2), pages 496-507, June.
    15. Ulrike Graßhoff & Heinz Holling & Rainer Schwabe, 2012. "Optimal Designs for the Rasch Model," Psychometrika, Springer;The Psychometric Society, vol. 77(4), pages 710-723, October.
    16. Wim J. van der Linden & Hao Ren, 2020. "A Fast and Simple Algorithm for Bayesian Adaptive Testing," Journal of Educational and Behavioral Statistics, , vol. 45(1), pages 58-85, February.
    17. García-Ródenas, Ricardo & García-García, José Carlos & López-Fidalgo, Jesús & Martín-Baos, José Ángel & Wong, Weng Kee, 2020. "A comparison of general-purpose optimization algorithms for finding optimal approximate experimental designs," Computational Statistics & Data Analysis, Elsevier, vol. 144(C).
    18. Nedka Dechkova Nikiforova & Rossella Berni & Jesús Fernando López‐Fidalgo, 2022. "Optimal approximate choice designs for a two‐step coffee choice, taste and choice again experiment," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(5), pages 1895-1917, November.
    19. Wael Korani & Malek Mouhoub, 2021. "Review on Nature-Inspired Algorithms," SN Operations Research Forum, Springer, vol. 2(3), pages 1-26, September.
    20. Masoudi, Ehsan & Holling, Heinz & Wong, Weng Kee, 2017. "Application of imperialist competitive algorithm to find minimax and standardized maximin optimal designs," Computational Statistics & Data Analysis, Elsevier, vol. 113(C), pages 330-345.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:157:y:2021:i:c:s0167947321000116. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.