IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v206y2013i1p595-61010.1007-s10479-012-1218-y.html
   My bibliography  Save this article

Application of robust optimization to automated test assembly

Author

Listed:
  • Bernard Veldkamp

Abstract

In automated test assembly (ATA), 0-1 linear programming (0-1 LP) methods are applied to select questions (items) from an item bank to assemble an optimal test. The objective in this 0-1 LP optimization problem is to assemble a test that measures, in as precise a way as possible, the ability of candidates. Item response theory (IRT) is commonly applied to model the relationship between the responses of candidates and their ability level. Parameters that describe the characteristics of each item, such as difficulty level and the extent to which an item differentiates between more and less able test takers (discrimination) are estimated in the application of the IRT model. Unfortunately, since all parameters in IRT models have to be estimated, they do have a level of uncertainty to them. Some of the other parameters in the test assembly model, such as average response times, have been estimated with uncertainty as well. General 0-1 LP methods do not take this uncertainty into account, and overestimate the predicted level of measurement precision. In this paper, alternative robust optimization methods are applied. It is demonstrated how the Bertsimas and Sim method can be applied to take this uncertainty into account in ATA. The impact of applying this method is illustrated in two numerical examples. Implications are discussed, and some directions for future research are presented. Copyright Springer Science+Business Media, LLC 2013

Suggested Citation

  • Bernard Veldkamp, 2013. "Application of robust optimization to automated test assembly," Annals of Operations Research, Springer, vol. 206(1), pages 595-610, July.
    • Handle: RePEc:spr:annopr:v:206:y:2013:i:1:p:595-610:10.1007/s10479-012-1218-y
    DOI: 10.1007/s10479-012-1218-y
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10479-012-1218-y
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10479-012-1218-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Adema, Jos J. & Boekkooi-Timminga, Ellen & van der Linden, Wim J., 1991. "Achievement test construction using 0-1 linear programming," European Journal of Operational Research, Elsevier, vol. 55(1), pages 103-111, November.
    2. Ronald Armstrong & Dmitry Belov & Alexander Weissman, 2005. "Developing and Assembling the Law School Admission Test," Interfaces, INFORMS, vol. 35(2), pages 140-151, April.
    3. Wim Linden & Ellen Boekkooi-Timminga, 1989. "A maximin model for IRT-based test design with practical constraints," Psychometrika, Springer;The Psychometric Society, vol. 54(2), pages 237-247, June.
    4. Martijn G. de Jong & Jan-Benedict E. M. Steenkamp & Bernard P. Veldkamp, 2009. "A Model for the Construction of Country-Specific Yet Internationally Comparable Short-Form Marketing Scales," Marketing Science, INFORMS, vol. 28(4), pages 674-689, 07-08.
    5. T. Theunissen, 1985. "Binary programming and test design," Psychometrika, Springer;The Psychometric Society, vol. 50(4), pages 411-420, December.
    6. A. L. Soyster, 1973. "Technical Note—Convex Programming with Set-Inclusive Constraints and Applications to Inexact Linear Programming," Operations Research, INFORMS, vol. 21(5), pages 1154-1157, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Giada Spaccapanico Proietti & Mariagiulia Matteucci & Stefania Mignani & Bernard P. Veldkamp, 2024. "Chance-Constrained Automated Test Assembly," Journal of Educational and Behavioral Statistics, , vol. 49(1), pages 92-120, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Armstrong, Ronald D. & Kung, Mabel T. & Roussos, Louis A., 2010. "Determining targets for multi-stage adaptive tests using integer programming," European Journal of Operational Research, Elsevier, vol. 205(3), pages 709-718, September.
    2. Edmonds, Jennifer & Armstrong, Ronald, 2009. "A mixed integer programming model for multiple stage adaptive testing," European Journal of Operational Research, Elsevier, vol. 193(2), pages 342-350, March.
    3. Wenqing Chen & Melvyn Sim & Jie Sun & Chung-Piaw Teo, 2010. "From CVaR to Uncertainty Set: Implications in Joint Chance-Constrained Optimization," Operations Research, INFORMS, vol. 58(2), pages 470-485, April.
    4. Stefan Mišković, 2017. "A VNS-LP algorithm for the robust dynamic maximal covering location problem," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 39(4), pages 1011-1033, October.
    5. Jeong, Jaehee & Premsankar, Gopika & Ghaddar, Bissan & Tarkoma, Sasu, 2024. "A robust optimization approach for placement of applications in edge computing considering latency uncertainty," Omega, Elsevier, vol. 126(C).
    6. Beck, Yasmine & Ljubić, Ivana & Schmidt, Martin, 2023. "A survey on bilevel optimization under uncertainty," European Journal of Operational Research, Elsevier, vol. 311(2), pages 401-426.
    7. Antonio G. Martín & Manuel Díaz-Madroñero & Josefa Mula, 2020. "Master production schedule using robust optimization approaches in an automobile second-tier supplier," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 28(1), pages 143-166, March.
    8. Wang, Luming & Finn, Adam, 2014. "A psychometric theory that measures up to marketing reality: An adapted Many Faceted IRT model," Australasian marketing journal, Elsevier, vol. 22(2), pages 93-102.
    9. Martijn G. de Jong & Jan-Benedict E. M. Steenkamp & Bernard P. Veldkamp, 2009. "A Model for the Construction of Country-Specific Yet Internationally Comparable Short-Form Marketing Scales," Marketing Science, INFORMS, vol. 28(4), pages 674-689, 07-08.
    10. Rashed Khanjani-Shiraz & Ali Babapour-Azar & Zohreh Hosseini-Noudeh & Panos M. Pardalos, 2022. "Distributionally robust maximum probability shortest path problem," Journal of Combinatorial Optimization, Springer, vol. 43(1), pages 140-167, January.
    11. Khoirunnisa Rohadatul Aisy Muslihin & Endang Rusyaman & Diah Chaerani, 2022. "Conic Duality for Multi-Objective Robust Optimization Problem," Mathematics, MDPI, vol. 10(21), pages 1-22, October.
    12. Walid Ben-Ameur & Adam Ouorou & Guanglei Wang & Mateusz Żotkiewicz, 2018. "Multipolar robust optimization," EURO Journal on Computational Optimization, Springer;EURO - The Association of European Operational Research Societies, vol. 6(4), pages 395-434, December.
    13. Roberto Gomes de Mattos & Fabricio Oliveira & Adriana Leiras & Abdon Baptista de Paula Filho & Paulo Gonçalves, 2019. "Robust optimization of the insecticide-treated bed nets procurement and distribution planning under uncertainty for malaria prevention and control," Annals of Operations Research, Springer, vol. 283(1), pages 1045-1078, December.
    14. F. Davarian & J. Behnamian, 2022. "Robust finite-horizon scheduling/rescheduling of operating rooms with elective and emergency surgeries under resource constraints," Journal of Scheduling, Springer, vol. 25(6), pages 625-641, December.
    15. Fakhar, Majid & Mahyarinia, Mohammad Reza & Zafarani, Jafar, 2018. "On nonsmooth robust multiobjective optimization under generalized convexity with applications to portfolio optimization," European Journal of Operational Research, Elsevier, vol. 265(1), pages 39-48.
    16. Pinar, Mehmet & Stengos, Thanasis & Topaloglou, Nikolas, 2020. "On the construction of a feasible range of multidimensional poverty under benchmark weight uncertainty," European Journal of Operational Research, Elsevier, vol. 281(2), pages 415-427.
    17. Antonio J. Conejo & Nicholas G. Hall & Daniel Zhuoyu Long & Runhao Zhang, 2021. "Robust Capacity Planning for Project Management," INFORMS Journal on Computing, INFORMS, vol. 33(4), pages 1533-1550, October.
    18. Bastian, Nathaniel D. & Lunday, Brian J. & Fisher, Christopher B. & Hall, Andrew O., 2020. "Models and methods for workforce planning under uncertainty: Optimizing U.S. Army cyber branch readiness and manning," Omega, Elsevier, vol. 92(C).
    19. Wang, Fan & Zhang, Chao & Zhang, Hui & Xu, Liang, 2021. "Short-term physician rescheduling model with feature-driven demand for mental disorders outpatients," Omega, Elsevier, vol. 105(C).
    20. Zhi Chen & Melvyn Sim & Peng Xiong, 2020. "Robust Stochastic Optimization Made Easy with RSOME," Management Science, INFORMS, vol. 66(8), pages 3329-3339, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:206:y:2013:i:1:p:595-610:10.1007/s10479-012-1218-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.