IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v80y2015i2p263-288.html
   My bibliography  Save this article

Optimal Bayesian Adaptive Design for Test-Item Calibration

Author

Listed:
  • Wim Linden
  • Hao Ren

Abstract

An optimal adaptive design for test-item calibration based on Bayesian optimality criteria is presented. The design adapts the choice of field-test items to the examinees taking an operational adaptive test using both the information in the posterior distributions of their ability parameters and the current posterior distributions of the field-test parameters. Different criteria of optimality based on the two types of posterior distributions are possible. The design can be implemented using an MCMC scheme with alternating stages of sampling from the posterior distributions of the test takers’ ability parameters and the parameters of the field-test items while reusing samples from earlier posterior distributions of the other parameters. Results from a simulation study demonstrated the feasibility of the proposed MCMC implementation for operational item calibration. A comparison of performances for different optimality criteria showed faster calibration of substantial numbers of items for the criterion of D-optimality relative to A-optimality, a special case of c-optimality, and random assignment of items to the test takers. Copyright The Psychometric Society 2015

Suggested Citation

  • Wim Linden & Hao Ren, 2015. "Optimal Bayesian Adaptive Design for Test-Item Calibration," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 263-288, June.
    • Handle: RePEc:spr:psycho:v:80:y:2015:i:2:p:263-288
    DOI: 10.1007/s11336-013-9391-8
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s11336-013-9391-8
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11336-013-9391-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Robert Mislevy & Hua-Hua Chang, 2000. "Does adaptive testing violate local independence?," Psychometrika, Springer;The Psychometric Society, vol. 65(2), pages 149-156, June.
    2. Martijn Berger, 1992. "Sequential sampling designs for the two-parameter item response theory model," Psychometrika, Springer;The Psychometric Society, vol. 57(4), pages 521-538, December.
    3. Martha Stocking, 1990. "Specifying optimum examinees for item parameter estimation in item response theory," Psychometrika, Springer;The Psychometric Society, vol. 55(3), pages 461-475, September.
    4. Martijn Berger & C. Joy King & Weng Wong, 2000. "Minimax d-optimal designs for item response theory models," Psychometrika, Springer;The Psychometric Society, vol. 65(3), pages 377-390, September.
    5. Douglas Jones & Zhiying Jin, 1994. "Optimal sequential designs for on-line item estimation," Psychometrika, Springer;The Psychometric Society, vol. 59(1), pages 59-75, March.
    6. Yuan-chin Chang & Hung-Yi Lu, 2010. "Online Calibration Via Variable Length Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 75(1), pages 140-157, March.
    7. Rosenthal, Jeffrey S., 2007. "AMCMC: An R interface for adaptive MCMC," Computational Statistics & Data Analysis, Elsevier, vol. 51(12), pages 5467-5470, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mahmood Ul Hassan & Frank Miller, 2019. "Optimal Item Calibration for Computerized Achievement Tests," Psychometrika, Springer;The Psychometric Society, vol. 84(4), pages 1101-1128, December.
    2. Ul Hassan, Mahmood & Miller, Frank, 2021. "An exchange algorithm for optimal calibration of items in computerized achievement tests," Computational Statistics & Data Analysis, Elsevier, vol. 157(C).
    3. Frank Miller & Ellinor Fackle-Fornius, 2024. "Parallel Optimal Calibration of Mixed-Format Items for Achievement Tests," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 903-928, September.
    4. Yinhong He & Ping Chen, 2020. "Optimal Online Calibration Designs for Item Replenishment in Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 85(1), pages 35-55, March.
    5. Wim J. van der Linden & Hao Ren, 2020. "A Fast and Simple Algorithm for Bayesian Adaptive Testing," Journal of Educational and Behavioral Statistics, , vol. 45(1), pages 58-85, February.
    6. Wim J. van der Linden & Bingnan Jiang, 2020. "A Shadow-Test Approach to Adaptive Item Calibration," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 301-321, June.
    7. Hao Ren & Wim J. van der Linden & Qi Diao, 2017. "Continuous Online Item Calibration: Parameter Recovery and Item Utilization," Psychometrika, Springer;The Psychometric Society, vol. 82(2), pages 498-522, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yinhong He & Ping Chen, 2020. "Optimal Online Calibration Designs for Item Replenishment in Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 85(1), pages 35-55, March.
    2. Ul Hassan, Mahmood & Miller, Frank, 2021. "An exchange algorithm for optimal calibration of items in computerized achievement tests," Computational Statistics & Data Analysis, Elsevier, vol. 157(C).
    3. Mahmood Ul Hassan & Frank Miller, 2019. "Optimal Item Calibration for Computerized Achievement Tests," Psychometrika, Springer;The Psychometric Society, vol. 84(4), pages 1101-1128, December.
    4. Yuan-chin Chang & Hung-Yi Lu, 2010. "Online Calibration Via Variable Length Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 75(1), pages 140-157, March.
    5. Hao Ren & Wim J. van der Linden & Qi Diao, 2017. "Continuous Online Item Calibration: Parameter Recovery and Item Utilization," Psychometrika, Springer;The Psychometric Society, vol. 82(2), pages 498-522, June.
    6. Hyeon-Ah Kang & Yi Zheng & Hua-Hua Chang, 2020. "Online Calibration of a Joint Model of Item Responses and Response Times in Computerized Adaptive Testing," Journal of Educational and Behavioral Statistics, , vol. 45(2), pages 175-208, April.
    7. Ping Chen & Chun Wang, 2016. "A New Online Calibration Method for Multidimensional Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 674-701, September.
    8. Ping Chen, 2017. "A Comparative Study of Online Item Calibration Methods in Multidimensional Computerized Adaptive Testing," Journal of Educational and Behavioral Statistics, , vol. 42(5), pages 559-590, October.
    9. Frank Miller & Ellinor Fackle-Fornius, 2024. "Parallel Optimal Calibration of Mixed-Format Items for Achievement Tests," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 903-928, September.
    10. Pasanisi, Alberto & Fu, Shuai & Bousquet, Nicolas, 2012. "Estimating discrete Markov models from various incomplete data schemes," Computational Statistics & Data Analysis, Elsevier, vol. 56(9), pages 2609-2625.
    11. Edison M. Choe & Jinming Zhang & Hua-Hua Chang, 2018. "Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 83(3), pages 650-673, September.
    12. Yuan-chin Chang, 2011. "Sequential estimation in generalized linear models when covariates are subject to errors," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 73(1), pages 93-120, January.
    13. Douglas Jones & Zhiying Jin, 1994. "Optimal sequential designs for on-line item estimation," Psychometrika, Springer;The Psychometric Society, vol. 59(1), pages 59-75, March.
    14. Sandip Sinharay, 2016. "Person Fit Analysis in Computerized Adaptive Testing Using Tests for a Change Point," Journal of Educational and Behavioral Statistics, , vol. 41(5), pages 521-549, October.
    15. Ping Chen & Tao Xin & Chun Wang & Hua-Hua Chang, 2012. "Online Calibration Methods for the DINA Model with Independent Attributes in CD-CAT," Psychometrika, Springer;The Psychometric Society, vol. 77(2), pages 201-222, April.
    16. Zimu Chen & Zhanfeng Wang & Yuan‐chin Ivan Chang, 2020. "Sequential adaptive variables and subject selection for GEE methods," Biometrics, The International Biometric Society, vol. 76(2), pages 496-507, June.
    17. Sandip Sinharay, 2017. "Detection of Item Preknowledge Using Likelihood Ratio Test and Score Test," Journal of Educational and Behavioral Statistics, , vol. 42(1), pages 46-68, February.
    18. Vihola, Matti, 2010. "Grapham: Graphical models with adaptive random walk Metropolis algorithms," Computational Statistics & Data Analysis, Elsevier, vol. 54(1), pages 49-54, January.
    19. Jasra, Ajay & Doucet, Arnaud & Stephens, David A. & Holmes, Christopher C., 2008. "Interacting sequential Monte Carlo samplers for trans-dimensional simulation," Computational Statistics & Data Analysis, Elsevier, vol. 52(4), pages 1765-1791, January.
    20. Wim Linden & Edith Krimpen-Stoop, 2003. "Using response times to detect aberrant responses in computerized adaptive testing," Psychometrika, Springer;The Psychometric Society, vol. 68(2), pages 251-265, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:80:y:2015:i:2:p:263-288. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.