IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0238145.html
   My bibliography  Save this article

On the effects of hard and soft equality constraints in the iterative outlier elimination procedure

Author

Listed:
  • Vinicius Francisco Rofatto
  • Marcelo Tomio Matsuoka
  • Ivandro Klein
  • Maurício Roberto Veronez
  • Luiz Gonzaga da Silveira Junior

Abstract

Reliability analysis allows for the estimation of a system’s probability of detecting and identifying outliers. Failure to identify an outlier can jeopardize the reliability level of a system. Due to its importance, outliers must be appropriately treated to ensure the normal operation of a system. System models are usually developed from certain constraints. Constraints play a central role in model precision and validity. In this work, we present a detailed investigation of the effects of the hard and soft constraints on the reliability of a measurement system model. Hard constraints represent a case in which there exist known functional relations between the unknown model parameters, whereas the soft constraints are employed where such functional relations can be slightly violated depending on their uncertainty. The results highlighted that the success rate of identifying an outlier for the case of hard constraints is larger than soft constraints. This suggested that hard constraints be used in the stage of pre-processing data for the purpose of identifying and removing possible outlying measurements. After identifying and removing possible outliers, one should set up the soft constraints to propagate their uncertainties to the model parameters during the data processing.

Suggested Citation

  • Vinicius Francisco Rofatto & Marcelo Tomio Matsuoka & Ivandro Klein & Maurício Roberto Veronez & Luiz Gonzaga da Silveira Junior, 2020. "On the effects of hard and soft equality constraints in the iterative outlier elimination procedure," PLOS ONE, Public Library of Science, vol. 15(8), pages 1-29, August.
    • Handle: RePEc:plo:pone00:0238145
    DOI: 10.1371/journal.pone.0238145
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0238145
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0238145&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0238145?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Randa Aljably & Yuan Tian & Mznah Al-Rodhaan & Abdullah Al-Dhelaan, 2019. "Anomaly detection over differential preserved privacy in online social networks," PLOS ONE, Public Library of Science, vol. 14(4), pages 1-20, April.
    2. Yanguang Chen, 2016. "Spatial Autocorrelation Approaches to Testing Residuals from Least Squares Regression," PLOS ONE, Public Library of Science, vol. 11(1), pages 1-19, January.
    3. Nysia I George & John F Bowyer & Nathaniel M Crabtree & Ching-Wei Chang, 2015. "An Iterative Leave-One-Out Approach to Outlier Detection in RNA-Seq Data," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-10, June.
    4. Mathieu Gautier & Toby Dylan Hocking & Jean-Louis Foulley, 2010. "A Bayesian Outlier Criterion to Detect SNPs under Selection in Large Data Sets," PLOS ONE, Public Library of Science, vol. 5(8), pages 1-16, August.
    5. Markus Goldstein & Seiichi Uchida, 2016. "A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-31, April.
    6. Dan Weaving & Ben Jones & Matt Ireton & Sarah Whitehead & Kevin Till & Clive B Beggs, 2019. "Overcoming the problem of multicollinearity in sports performance data: A novel application of partial least squares correlation analysis," PLOS ONE, Public Library of Science, vol. 14(2), pages 1-16, February.
    7. Bruno Faria & Fernao Vistulo de Abreu, 2019. "Cellular frustration algorithms for anomaly detection applications," PLOS ONE, Public Library of Science, vol. 14(7), pages 1-31, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yin, Sihua & Yang, Haidong & Xu, Kangkang & Zhu, Chengjiu & Zhang, Shaqing & Liu, Guosheng, 2022. "Dynamic real–time abnormal energy consumption detection and energy efficiency optimization analysis considering uncertainty," Applied Energy, Elsevier, vol. 307(C).
    2. Adele Ravagnani & Fabrizio Lillo & Paola Deriu & Piero Mazzarisi & Francesca Medda & Antonio Russo, 2024. "Dimensionality reduction techniques to support insider trading detection," Papers 2403.00707, arXiv.org, revised May 2024.
    3. Davide Nicola Continanza & Andrea del Monaco & Marco di Lucido & Daniele Figoli & Pasquale Maddaloni & Filippo Quarta & Giuseppe Turturiello, 2023. "Stacking machine learning models for anomaly detection: comparing AnaCredit to other banking data sets," IFC Bulletins chapters, in: Bank for International Settlements (ed.), Data science in central banking: applications and tools, volume 59, Bank for International Settlements.
    4. Priyanga Dilini Talagala & Rob J Hyndman & Kate Smith-Miles, 2019. "Anomaly Detection in High Dimensional Data," Monash Econometrics and Business Statistics Working Papers 20/19, Monash University, Department of Econometrics and Business Statistics.
    5. Sevvandi Kandanaarachchi & Mario A Munoz & Rob J Hyndman & Kate Smith-Miles, 2018. "On normalization and algorithm selection for unsupervised outlier detection," Monash Econometrics and Business Statistics Working Papers 16/18, Monash University, Department of Econometrics and Business Statistics.
    6. Tae‐Hyoung T. Gim, 2021. "Partial least squares regression and importance–satisfaction analyses of the strategic drivers of happiness: A quality of life survey in Seoul, Korea," Growth and Change, Wiley Blackwell, vol. 52(1), pages 567-599, March.
    7. María Gabriela Pizarro Inostroza & Francisco Javier Navas González & Vincenzo Landi & José Manuel León Jurado & Juan Vicente Delgado Bermejo & Javier Fernández Álvarez & María del Amparo Martínez Mart, 2020. "Software-Automatized Individual Lactation Model Fitting, Peak and Persistence and Bayesian Criteria Comparison for Milk Yield Genetic Studies in Murciano-Granadina Goats," Mathematics, MDPI, vol. 8(9), pages 1-21, September.
    8. Priyanga Dilini Talagala & Rob J Hyndman & Catherine Leigh & Kerrie Mengersen & Kate Smith-Miles, 2019. "A Feature-Based Framework for Detecting Technical Outliers in Water-Quality Data from In Situ Sensors," Monash Econometrics and Business Statistics Working Papers 1/19, Monash University, Department of Econometrics and Business Statistics.
    9. Piero Mazzarisi & Adele Ravagnani & Paola Deriu & Fabrizio Lillo & Francesca Medda & Antonio Russo, 2022. "A machine learning approach to support decision in insider trading detection," Papers 2212.05912, arXiv.org.
    10. Cian Ryan & Finbarr Murphy & Martin Mullins, 2019. "Semiautonomous Vehicle Risk Analysis: A Telematics‐Based Anomaly Detection Approach," Risk Analysis, John Wiley & Sons, vol. 39(5), pages 1125-1140, May.
    11. Elmira Asadi-Fard & Samereh Falahatkar & Mahdi Tanha Ziyarati & Xiaodong Zhang & Mariapia Faruolo, 2023. "Assessment of RXD Algorithm Capability for Gas Flaring Detection through OLI-SWIR Channels," Sustainability, MDPI, vol. 15(6), pages 1-20, March.
    12. Kenichiro Nagata & Toshikazu Tsuji & Kimitaka Suetsugu & Kayoko Muraoka & Hiroyuki Watanabe & Akiko Kanaya & Nobuaki Egashira & Ichiro Ieiri, 2021. "Detection of overdose and underdose prescriptions—An unsupervised machine learning approach," PLOS ONE, Public Library of Science, vol. 16(11), pages 1-14, November.
    13. Ruhi Kiran Bajaj & Rebecca Mary Meiring & Fernando Beltran, 2023. "Co-Design, Development, and Evaluation of a Health Monitoring Tool Using Smartwatch Data: A Proof-of-Concept Study," Future Internet, MDPI, vol. 15(3), pages 1-15, March.
    14. Haosu Zhao & Bart Julien Dewancker & Feng Hua & Junping He & Weijun Gao, 2020. "Restrictions of Historical Tissues on Urban Growth, Self-Sustaining Agglomeration in Walled Cities of Chinese Origin," Sustainability, MDPI, vol. 12(14), pages 1-29, July.
    15. Chatterjee, Joyjit & Dethlefs, Nina, 2021. "Scientometric review of artificial intelligence for operations & maintenance of wind turbines: The past, present and future," Renewable and Sustainable Energy Reviews, Elsevier, vol. 144(C).
    16. Fan, Cheng & Xiao, Fu & Zhao, Yang & Wang, Jiayuan, 2018. "Analytical investigation of autoencoder-based methods for unsupervised anomaly detection in building energy data," Applied Energy, Elsevier, vol. 211(C), pages 1123-1135.
    17. Nancy Cherotich Sitienei & Meshack Misoi & Chidozie Ibeneme, 2023. "Supply Chain Management Practices on Organizational Performance: A Case Study of Tea Industries in North Rift Valley, Kenya," International Journal of Research and Innovation in Social Science, International Journal of Research and Innovation in Social Science (IJRISS), vol. 7(7), pages 846-865, July.
    18. Alejandro Colomo Magaña & Ernesto Colomo Magaña & Francisco D. Guillén-Gámez & Andrea Cívico Ariza, 2022. "Analysis of Prospective Teachers’ Perceptions of the Flipped Classroom as a Classroom Methodology," Societies, MDPI, vol. 12(4), pages 1-15, June.
    19. Shuo Xu & Liyuan Hao & Xin An & Dongsheng Zhai & Hongshen Pang, 2019. "Types of DOI errors of cited references in Web of Science with a cleaning method," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(3), pages 1427-1437, September.
    20. Milan Miric & Hakan Ozalp & Erdem Dogukan Yilmaz, 2023. "Trade‐offs to using standardized tools: Innovation enablers or creativity constraints?," Strategic Management Journal, Wiley Blackwell, vol. 44(4), pages 909-942, April.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0238145. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.