IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v167y2004i2p249-274.html
   My bibliography  Save this article

New ways of specifying data edits

Author

Listed:
  • George Petrakos
  • Claudio Conversano
  • Gregory Farmakis
  • Francesco Mola
  • Roberta Siciliano
  • Photis Stavropoulos

Abstract

Summary. Data editing is the process by which data that are collected in some way (a statistical survey for example) are examined for errors and corrected with the help of software. Edits, the logical conditions that should be satisfied by the data, are specified by subject‐matter experts with a procedure which could be tedious and could lead to mistakes with practical implications. To render the process of edit specification more efficient we provide a new step—the definition of the so‐called abstract data model of a survey—which describes the structure of the phenomenon that is studied in a survey. The existence of this model enables experts to identify all combinations of variables which should be checked by edits and to avoid the definition of conflicting edits. Furthermore, we introduce an automatic data validation strategy—TREEVAL—that consists of fast tree growing to derive automatically the functional form of edits and of a statistical criterion to clean the incoming data. The TREEVAL strategy is cast within a total quality management framework. The application of the methodologies proposed is demonstrated with the help of a real life application.

Suggested Citation

  • George Petrakos & Claudio Conversano & Gregory Farmakis & Francesco Mola & Roberta Siciliano & Photis Stavropoulos, 2004. "New ways of specifying data edits," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 167(2), pages 249-274, May.
    • Handle: RePEc:bla:jorssa:v:167:y:2004:i:2:p:249-274
    DOI: 10.1046/j.1467-985X.2003.00745.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1046/j.1467-985X.2003.00745.x
    Download Restriction: no
    File URL: https://libkey.io/10.1046/j.1467-985X.2003.00745.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Cappelli, Carmela & Mola, Francesco & Siciliano, Roberta, 2002. "A statistical approach to growing a reliable honest tree," Computational Statistics & Data Analysis, Elsevier, vol. 38(3), pages 285-299, January.
    2. R. S. Garfinkel & A. S. Kunnathur & G. E. Liepins, 1986. "Optimal Imputation of Erroneous Data: Categorical Data, General Edits," Operations Research, INFORMS, vol. 34(5), pages 744-751, October.
    3. Siciliano, Roberta & Mola, Francesco, 2000. "Multivariate data analysis and modeling through classification and regression trees," Computational Statistics & Data Analysis, Elsevier, vol. 32(3-4), pages 285-301, January.
    4. Leopold Granquist, 1997. "The New View on Editing," International Statistical Review, International Statistical Institute, vol. 65(3), pages 381-387, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Riccardo Borgoni & Ann Berrington, 2013. "Evaluating a sequential tree-based procedure for multivariate imputation of complex missing data structures," Quality & Quantity: International Journal of Methodology, Springer, vol. 47(4), pages 1991-2008, June.
    2. Claudio Conversano & Roberta Siciliano, 2009. "Incremental Tree-Based Missing Data Imputation with Lexicographic Ordering," Journal of Classification, Springer;The Classification Society, vol. 26(3), pages 361-379, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Roberta Siciliano & Antonio D’Ambrosio & Massimo Aria & Sonia Amodio, 2017. "Analysis of Web Visit Histories, Part II: Predicting Navigation by Nested STUMP Regression Trees," Journal of Classification, Springer;The Classification Society, vol. 34(3), pages 473-493, October.
    2. Pier Perri & Peter Heijden, 2012. "A Property of the CHAID Partitioning Method for Dichotomous Randomized Response Data and Categorical Predictors," Journal of Classification, Springer;The Classification Society, vol. 29(1), pages 76-90, April.
    3. Lee, Tzu-Haw & Shih, Yu-Shan, 2006. "Unbiased variable selection for classification trees with multivariate responses," Computational Statistics & Data Analysis, Elsevier, vol. 51(2), pages 659-667, November.
    4. de Waal Ton, 2013. "Selective Editing: A Quest for Efficiency and Data Quality," Journal of Official Statistics, Sciendo, vol. 29(4), pages 473-488, December.
    5. Mariangela Sciandra & Antonella Plaia & Vincenza Capursi, 2017. "Classification trees for multivariate ordinal response: an application to Student Evaluation Teaching," Quality & Quantity: International Journal of Methodology, Springer, vol. 51(2), pages 641-655, March.
    6. Sergio Delgado-Quintero & Juan-José Salazar-González, 2008. "A new approach for data editing and imputation," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 68(3), pages 407-428, December.
    7. Massimo Aria & Antonio D’Ambrosio & Carmela Iorio & Roberta Siciliano & Valentina Cozza, 2020. "Dynamic recursive tree-based partitioning for malignant melanoma identification in skin lesion dermoscopic images," Statistical Papers, Springer, vol. 61(4), pages 1645-1661, August.
    8. Sam Asher & Denis Nekipelov & Paul Novosad & Stephen P. Ryan, 2016. "Classification Trees for Heterogeneous Moment-Based Models," NBER Working Papers 22976, National Bureau of Economic Research, Inc.
    9. Claudio Conversano & Francesco Mola & Roberta Siciliano, 2001. "Partitioning Algorithms and Combined Model Integration for Data Mining," Computational Statistics, Springer, vol. 16(3), pages 323-339, September.
    10. Di Zio Marco & Guarnera Ugo, 2013. "A Contamination Model for Selective Editing," Journal of Official Statistics, Sciendo, vol. 29(4), pages 539-555, December.
    11. Carmela Cappelli & Francesca Iorio & Angela Maddaloni & Pierpaolo D’Urso, 2021. "Atheoretical Regression Trees for classifying risky financial institutions," Annals of Operations Research, Springer, vol. 299(1), pages 1357-1377, April.
    12. Antonio D’Ambrosio & Willem J. Heiser, 2016. "A Recursive Partitioning Method for the Prediction of Preference Rankings Based Upon Kemeny Distances," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 774-794, September.
    13. Piccarreta, Raffaella, 2010. "Binary trees for dissimilarity data," Computational Statistics & Data Analysis, Elsevier, vol. 54(6), pages 1516-1524, June.
    14. Cappelli, Carmela & Mola, Francesco & Siciliano, Roberta, 2002. "A statistical approach to growing a reliable honest tree," Computational Statistics & Data Analysis, Elsevier, vol. 38(3), pages 285-299, January.
    15. Noh, Hyun Gon & Song, Moon Sup & Park, Sung Hyun, 2004. "An unbiased method for constructing multilabel classification trees," Computational Statistics & Data Analysis, Elsevier, vol. 47(1), pages 149-164, August.
    16. Dine, Abdessamad & Larocque, Denis & Bellavance, François, 2009. "Multivariate trees for mixed outcomes," Computational Statistics & Data Analysis, Elsevier, vol. 53(11), pages 3795-3804, September.
    17. Antonio D’Ambrosio & Massimo Aria & Roberta Siciliano, 2012. "Accurate Tree-based Missing Data Imputation and Data Fusion within the Statistical Learning Paradigm," Journal of Classification, Springer;The Classification Society, vol. 29(2), pages 227-258, July.
    18. Arbue´s Ignacio & Revilla Pedro & Salgado David, 2013. "An Optimization Approach to Selective Editing," Journal of Official Statistics, Sciendo, vol. 29(4), pages 489-510, December.
    19. Claudio Conversano & Roberta Siciliano, 2009. "Incremental Tree-Based Missing Data Imputation with Lexicographic Ordering," Journal of Classification, Springer;The Classification Society, vol. 26(3), pages 361-379, December.
    20. Amodio, S. & D’Ambrosio, A. & Siciliano, R., 2016. "Accurate algorithms for identifying the median ranking when dealing with weak and partial rankings under the Kemeny axiomatic approach," European Journal of Operational Research, Elsevier, vol. 249(2), pages 667-676.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:167:y:2004:i:2:p:249-274. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.