IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v342y2024i3d10.1007_s10479-022-05099-z.html
   My bibliography  Save this article

Outliers detection in assessment tests’ quality evaluation through the blended use of functional data analysis and item response theory

Author

Listed:
  • Fabrizio Maturo

    (University of Campania Luigi Vanvitelli)

  • Francesca Fortuna

    (Roma Tre University)

  • Tonio Di Battista

    (University of Chieti-Pescara G. D’Annunzio)

Abstract

The quality of assessment tests plays a fundamental role in decision-making problems in various fields such as education, psychology, and behavioural medicine. The first phase in the questionnaires’ validation process is outliers’ recognition. The latter can be identified at different levels, such as subject responses, individuals, and items. This paper focuses on item outliers and proposes a blended use of functional data analysis and item response theory for identifying outliers in assessment tests. The basic idea is that item characteristics curves derived from test responses can be treated as functions, and functional tools can be exploited to discover anomalies in item behaviour. For this purpose, this research suggests a multi-step strategy to catch magnitude and shape outliers employing a suitable transformation of item characteristics curves and their first derivatives. A simulation study emphasises the effectiveness of the proposed technique and exhibits exciting results in discovering outliers that classical functional methods do not detect. Moreover, the applicability of the method is shown with a real dataset. The final aim is to offer a methodology for improving the questionnaires’ quality.

Suggested Citation

  • Fabrizio Maturo & Francesca Fortuna & Tonio Di Battista, 2024. "Outliers detection in assessment tests’ quality evaluation through the blended use of functional data analysis and item response theory," Annals of Operations Research, Springer, vol. 342(3), pages 1547-1562, November.
    • Handle: RePEc:spr:annopr:v:342:y:2024:i:3:d:10.1007_s10479-022-05099-z
    DOI: 10.1007/s10479-022-05099-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10479-022-05099-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10479-022-05099-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Febrero-Bande, Manuel & de la Fuente, Manuel Oviedo, 2012. "Statistical Computing in Functional Data Analysis: The R Package fda.usc," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 51(i04).
    2. R. Bock & Murray Aitkin, 1981. "Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm," Psychometrika, Springer;The Psychometric Society, vol. 46(4), pages 443-459, December.
    3. Mia Hubert & Peter Rousseeuw & Pieter Segaert, 2015. "Multivariate functional outlier detection," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 24(2), pages 177-202, July.
    4. López-Pintado, Sara & Romo, Juan, 2009. "On the Concept of Depth for Functional Data," Journal of the American Statistical Association, American Statistical Association, vol. 104(486), pages 718-734.
    5. Cuevas, Antonio & Febrero, Manuel & Fraiman, Ricardo, 2006. "On the use of the bootstrap for estimating functions with functional data," Computational Statistics & Data Analysis, Elsevier, vol. 51(2), pages 1063-1074, November.
    6. Francesca Fortuna & Fabrizio Maturo, 2019. "K-means clustering of item characteristic curves and item information curves via functional principal component analysis," Quality & Quantity: International Journal of Methodology, Springer, vol. 53(5), pages 2291-2304, September.
    7. Fabrizio Maturo & Antonio Balzanella & Tonio Di Battista, 2019. "Building Statistical Indicators of Equitable and Sustainable Well-Being in a Functional Framework," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 146(3), pages 449-471, December.
    8. Mia Hubert & Peter Rousseeuw & Pieter Segaert, 2015. "Rejoinder to ‘multivariate functional outlier detection’," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 24(2), pages 269-277, July.
    9. Rizopoulos, Dimitris, 2006. "ltm: An R Package for Latent Variable Modeling and Item Response Analysis," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 17(i05).
    10. Antonio Cuevas & Manuel Febrero & Ricardo Fraiman, 2007. "Robust estimation and classification for functional data via projection-based depth notions," Computational Statistics, Springer, vol. 22(3), pages 481-496, September.
    11. Ricardo Fraiman & Graciela Muniz, 2001. "Trimmed means for functional data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 10(2), pages 419-440, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dai, Wenlin & Mrkvička, Tomáš & Sun, Ying & Genton, Marc G., 2020. "Functional outlier detection and taxonomy by sequential transformations," Computational Statistics & Data Analysis, Elsevier, vol. 149(C).
    2. Carlo Sguera & Pedro Galeano & Rosa Lillo, 2014. "Spatial depth-based classification for functional data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 23(4), pages 725-750, December.
    3. Cristian F. Jiménez‐Varón & Fouzi Harrou & Ying Sun, 2024. "Pointwise data depth for univariate and multivariate functional outlier detection," Environmetrics, John Wiley & Sons, Ltd., vol. 35(5), August.
    4. Carlo Sguera & Sara López-Pintado, 2021. "A notion of depth for sparse functional data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 30(3), pages 630-649, September.
    5. Kuhnt, Sonja & Rehage, André, 2016. "An angle-based multivariate functional pseudo-depth for shape outlier detection," Journal of Multivariate Analysis, Elsevier, vol. 146(C), pages 325-340.
    6. Dai, Wenlin & Genton, Marc G., 2019. "Directional outlyingness for multivariate functional data," Computational Statistics & Data Analysis, Elsevier, vol. 131(C), pages 50-65.
    7. Oluwasegun Taiwo Ojo & Antonio Fernández Anta & Rosa E. Lillo & Carlo Sguera, 2022. "Detecting and classifying outliers in big functional data," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 16(3), pages 725-760, September.
    8. Mia Hubert & Peter Rousseeuw & Pieter Segaert, 2015. "Multivariate functional outlier detection," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 24(2), pages 177-202, July.
    9. repec:cte:wsrepe:ws140101 is not listed on IDEAS
    10. Miguel Flores & Salvador Naya & Rubén Fernández-Casal & Sonia Zaragoza & Paula Raña & Javier Tarrío-Saavedra, 2020. "Constructing a Control Chart Using Functional Data," Mathematics, MDPI, vol. 8(1), pages 1-26, January.
    11. Nagy, Stanislav & Ferraty, Frédéric, 2019. "Data depth for measurable noisy random functions," Journal of Multivariate Analysis, Elsevier, vol. 170(C), pages 95-114.
    12. Alba M. Franco-Pereira & Rosa E. Lillo, 2020. "Rank tests for functional data based on the epigraph, the hypograph and associated graphical representations," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 14(3), pages 651-676, September.
    13. Fabrizio Maturo & Rosanna Verde, 2023. "Supervised classification of curves via a combined use of functional data analysis and tree-based methods," Computational Statistics, Springer, vol. 38(1), pages 419-459, March.
    14. Blanquero, R. & Carrizosa, E. & Jiménez-Cordero, A. & Martín-Barragán, B., 2019. "Functional-bandwidth kernel for Support Vector Machine with Functional Data: An alternating optimization algorithm," European Journal of Operational Research, Elsevier, vol. 275(1), pages 195-207.
    15. Boente, Graciela & Parada, Daniela, 2023. "Robust estimation for functional quadratic regression models," Computational Statistics & Data Analysis, Elsevier, vol. 187(C).
    16. López-Pintado, Sara & Romo, Juan, 2011. "A half-region depth for functional data," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1679-1695, April.
    17. Graciela Boente & Matías Salibián-Barrera, 2021. "Robust functional principal components for sparse longitudinal data," METRON, Springer;Sapienza Università di Roma, vol. 79(2), pages 159-188, August.
    18. J. A. Cuesta-Albertos & M. Febrero-Bande & M. Oviedo de la Fuente, 2017. "The $$\hbox {DD}^G$$ DD G -classifier in the functional setting," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 26(1), pages 119-142, March.
    19. Daniel Hlubinka & Irène Gijbels & Marek Omelka & Stanislav Nagy, 2015. "Integrated data depth for smooth functions and its application in supervised classification," Computational Statistics, Springer, vol. 30(4), pages 1011-1031, December.
    20. Amovin-Assagba, Martial & Gannaz, Irène & Jacques, Julien, 2022. "Outlier detection in multivariate functional data through a contaminated mixture model," Computational Statistics & Data Analysis, Elsevier, vol. 174(C).
    21. Francesca Ieva & Anna Paganoni, 2015. "Discussion of “multivariate functional outlier detection” by M. Hubert, P. Rousseeuw and P. Segaert," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 24(2), pages 217-221, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:342:y:2024:i:3:d:10.1007_s10479-022-05099-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.