IDEAS home Printed from https://ideas.repec.org/p/msh/ebswps/2021-2.html
   My bibliography  Save this paper

Leave-one-out Kernel Density Estimates for Outlier Detection

Author

Listed:
  • Sevvandi Kandanaarachchi
  • Rob J Hyndman

Abstract

This paper introduces lookout, a new approach to detect outliers using leave-one-out kernel density estimates and extreme value theory. Outlier detection methods that use kernel density estimates generally employ a user defined parameter to determine the bandwidth. Lookout uses persistent homology to construct a bandwidth suitable for outlier detection without any user input. We demonstrate the effectiveness of lookout on an extensive data repository by comparing its performance with other outlier detection methods based on extreme value theory. Furthermore, we introduce outlier persistence, a useful concept that explores the birth and the cessation of outliers with changing bandwidth and significance levels. The R package lookout implements this algorithm.

Suggested Citation

  • Sevvandi Kandanaarachchi & Rob J Hyndman, 2021. "Leave-one-out Kernel Density Estimates for Outlier Detection," Monash Econometrics and Business Statistics Working Papers 2/21, Monash University, Department of Econometrics and Business Statistics.
    • Handle: RePEc:msh:ebswps:2021-2
    as

    Download full text from publisher

    File URL: https://www.monash.edu/business/ebs/research/publications/ebs/wp02-2021.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Peter Burridge & A. M. Robert Taylor, 2006. "Additive Outlier Detection Via Extreme‐Value Theory," Journal of Time Series Analysis, Wiley Blackwell, vol. 27(5), pages 685-701, September.
    2. Chad M Topaz & Lori Ziegelmeier & Tom Halverson, 2015. "Topological Data Analysis of Biological Aggregation Models," PLOS ONE, Public Library of Science, vol. 10(5), pages 1-26, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Olivier Darné & Amélie Charles, 2012. "A note on the uncertain trend in US real GNP: Evidence from robust unit root tests," Economics Bulletin, AccessEcon, vol. 32(3), pages 2399-2406.
    2. Priyanga Dilini Talagala & Rob J Hyndman & Catherine Leigh & Kerrie Mengersen & Kate Smith-Miles, 2019. "A Feature-Based Framework for Detecting Technical Outliers in Water-Quality Data from In Situ Sensors," Monash Econometrics and Business Statistics Working Papers 1/19, Monash University, Department of Econometrics and Business Statistics.
    3. Priyanga Dilini Talagala & Rob J Hyndman & Kate Smith-Miles & Sevvandi Kandanaarachchi & Mario A Munoz, 2018. "Anomaly detection in streaming nonstationary temporal data," Monash Econometrics and Business Statistics Working Papers 4/18, Monash University, Department of Econometrics and Business Statistics.
    4. Bent Nielsen & Andrew Whitby, 2015. "A Joint Chow Test for Structural Instability," Econometrics, MDPI, vol. 3(1), pages 1-31, March.
    5. Sam Astill & David I. Harvey & A. M. Robert Taylor, 2013. "A bootstrap test for additive outliers in non-stationary time series," Journal of Time Series Analysis, Wiley Blackwell, vol. 34(4), pages 454-465, July.
    6. Chini, Emilio Zanetti, 2023. "Can we estimate macroforecasters’ mis-behavior?," Journal of Economic Dynamics and Control, Elsevier, vol. 149(C).
    7. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    8. David I. Harvey & Stephen J. Leybourne & A. M. Robert Taylor, 2009. "Robust methods for detecting multiple level breaks in autocorrelated time series [Revised to become No. 10/01 above]," Discussion Papers 09/01, University of Nottingham, Granger Centre for Time Series Econometrics.
    9. Gabriel Rodriguez & Dionisio Ramirez, 2014. "A Note on the Size of the ADF Test with Additive Outliers and Fractional Errors. A Reappraisal about the (Non)Stationarity of the Latin-American Inflation Series," Revista Economía, Fondo Editorial - Pontificia Universidad Católica del Perú, vol. 37(73), pages 113-132.
    10. Priyanga Dilini Talagala & Rob J Hyndman & Kate Smith-Miles, 2019. "Anomaly Detection in High Dimensional Data," Monash Econometrics and Business Statistics Working Papers 20/19, Monash University, Department of Econometrics and Business Statistics.
    11. Harvey, David I. & Leybourne, Stephen J. & Taylor, A.M. Robert, 2010. "Robust methods for detecting multiple level breaks in autocorrelated time series," Journal of Econometrics, Elsevier, vol. 157(2), pages 342-358, August.
    12. Chen, Yi-Hsuan & Tu, Anthony H., 2013. "Estimating hedged portfolio value-at-risk using the conditional copula: An illustration of model risk," International Review of Economics & Finance, Elsevier, vol. 27(C), pages 514-528.
    13. M Ulmer & Lori Ziegelmeier & Chad M Topaz, 2019. "A topological approach to selecting models of biological experiments," PLOS ONE, Public Library of Science, vol. 14(3), pages 1-18, March.
    14. Kulkarni, Saumitra & Pharasi, Hirdesh K. & Vijayaraghavan, Sudharsan & Kumar, Sunil & Chakraborti, Anirban & Samal, Areejit, 2024. "Investigation of Indian stock markets using topological data analysis and geometry-inspired network measures," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 643(C).
    15. Roberto Baragona & Francesco Battaglia & Domenico Cucina, 2016. "Empirical Likelihood for Outlier Detection and Estimation in Autoregressive Time Series," Journal of Time Series Analysis, Wiley Blackwell, vol. 37(3), pages 315-336, May.
    16. Gabriel Rodriguez & Dionisio Ramirez, 2013. "A comparison between Tau-d and the procedure TRAMO-SEATS is also included," Documentos de Trabajo / Working Papers 2013-355, Departamento de Economía - Pontificia Universidad Católica del Perú.
    17. Josep Lluís Carrion-I-Silvestre & María Dolores Gadea, 2016. "Bounds, Breaks and Unit Root Tests," Journal of Time Series Analysis, Wiley Blackwell, vol. 37(2), pages 165-181, March.
    18. Alanya-Beltran, Willy, 2022. "Unit roots in lower-bounded series with outliers," Economic Modelling, Elsevier, vol. 115(C).

    More about this item

    Keywords

    anomaly detection; topological data analysis; persistent homology; extreme value theory; peak over thresholds; generalized Pareto distribution;
    All these keywords.

    JEL classification:

    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • C65 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Miscellaneous Mathematical Tools
    • C87 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Econometric Software

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:msh:ebswps:2021-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Professor Xibin Zhang (email available below). General contact details of provider: https://edirc.repec.org/data/dxmonau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.