IDEAS home Printed from https://ideas.repec.org/p/msh/ebswps/2019-17.html
   My bibliography  Save this paper

Dimension Reduction For Outlier Detection Using DOBIN

Author

Listed:
  • Sevvandi Kandanaarachchi
  • Rob J Hyndman

Abstract

This paper introduces DOBIN, a new approach to select a set of basis vectors tailored for outlier detection. DOBIN has a solid mathematical foundation and can be used as a dimension reduction tool for outlier detection tasks. We demonstrate the effectiveness of DOBIN on an extensive data repository, by comparing the performance of outlier detection methods using DOBIN and other bases. We further illustrate the utility of DOBIN as an outlier visualization tool. The R package dobin implements this basis construction.

Suggested Citation

  • Sevvandi Kandanaarachchi & Rob J Hyndman, 2019. "Dimension Reduction For Outlier Detection Using DOBIN," Monash Econometrics and Business Statistics Working Papers 17/19, Monash University, Department of Econometrics and Business Statistics.
    • Handle: RePEc:msh:ebswps:2019-17
    as

    Download full text from publisher

    File URL: https://www.monash.edu/business/ebs/research/publications/ebs/wp17-2019.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Vakili, Kaveh & Schmitt, Eric, 2014. "Finding multivariate outliers with FastPCS," Computational Statistics & Data Analysis, Elsevier, vol. 69(C), pages 54-66.
    2. Billor, Nedret & Hadi, Ali S. & Velleman, Paul F., 2000. "BACON: blocked adaptive computationally efficient outlier nominators," Computational Statistics & Data Analysis, Elsevier, vol. 34(3), pages 279-298, September.
    3. Sevvandi Kandanaarachchi & Mario A Munoz & Rob J Hyndman & Kate Smith-Miles, 2018. "On normalization and algorithm selection for unsupervised outlier detection," Monash Econometrics and Business Statistics Working Papers 16/18, Monash University, Department of Econometrics and Business Statistics.
    4. Markus Goldstein & Seiichi Uchida, 2016. "A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-31, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sevvandi Kandanaarachchi & Mario A Munoz & Rob J Hyndman & Kate Smith-Miles, 2018. "On normalization and algorithm selection for unsupervised outlier detection," Monash Econometrics and Business Statistics Working Papers 16/18, Monash University, Department of Econometrics and Business Statistics.
    2. Catherine Fuss & Angelos Theodorakopoulos, 2018. "Compositional Changes in Aggregate Productivity in an Era of Globalisation and Financial Crisis," Working Papers of VIVES - Research Centre for Regional Economics 627696, KU Leuven, Faculty of Economics and Business (FEB), VIVES - Research Centre for Regional Economics.
    3. L. Pitsoulis & G. Zioutas, 2010. "A fast algorithm for robust regression with penalised trimmed squares," Computational Statistics, Springer, vol. 25(4), pages 663-689, December.
    4. Yin, Sihua & Yang, Haidong & Xu, Kangkang & Zhu, Chengjiu & Zhang, Shaqing & Liu, Guosheng, 2022. "Dynamic real–time abnormal energy consumption detection and energy efficiency optimization analysis considering uncertainty," Applied Energy, Elsevier, vol. 307(C).
    5. Cristian BARRA & Roberto ZOTTI, 2019. "Bank Performance, Financial Stability And Market Concentration: Evidence From Cooperative And Non‐Cooperative Banks," Annals of Public and Cooperative Economics, Wiley Blackwell, vol. 90(1), pages 103-139, March.
    6. Charlotte Fabri & Sam Vermeulen & Steven Van Passel & Sergei Schaub, 2024. "Crop diversification and the effect of weather shocks on Italian farmers' income and income risk," Journal of Agricultural Economics, Wiley Blackwell, vol. 75(3), pages 955-980, September.
    7. Hong Choon Ong & Ekele Alih, 2015. "A Control Chart Based on Cluster-Regression Adjustment for Retrospective Monitoring of Individual Characteristics," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-30, April.
    8. Adele Ravagnani & Fabrizio Lillo & Paola Deriu & Piero Mazzarisi & Francesca Medda & Antonio Russo, 2024. "Dimensionality reduction techniques to support insider trading detection," Papers 2403.00707, arXiv.org, revised May 2024.
    9. Davide Nicola Continanza & Andrea del Monaco & Marco di Lucido & Daniele Figoli & Pasquale Maddaloni & Filippo Quarta & Giuseppe Turturiello, 2023. "Stacking machine learning models for anomaly detection: comparing AnaCredit to other banking data sets," IFC Bulletins chapters, in: Bank for International Settlements (ed.), Data science in central banking: applications and tools, volume 59, Bank for International Settlements.
    10. Stefani, Gianluca & Gadanakis, Yiorgos & Lombardi, Ginevra Virginia & Tiberti, Marco, 2017. "The impact of financial leverage on farms capacity to react in market shocks," 2017 International Congress, August 28-September 1, 2017, Parma, Italy 261156, European Association of Agricultural Economists.
    11. Schmitt, Eric & Öllerer, Viktoria & Vakili, Kaveh, 2014. "The finite sample breakdown point of PCS," Statistics & Probability Letters, Elsevier, vol. 94(C), pages 214-220.
    12. Peter Nemec, 2024. "Contesting the public works domain: examining the factors affecting presence and success of SMES in public procurement," Empirical Economics, Springer, vol. 67(5), pages 2135-2173, November.
    13. Batalla-Bejerano, Joan & Costa-Campi, Maria Teresa & Trujillo-Baute, Elisa, 2016. "Collateral effects of liberalisation: Metering, losses, load profiles and cost settlement in Spain’s electricity system," Energy Policy, Elsevier, vol. 94(C), pages 421-431.
    14. A.A.M. Nurunnabi & M. Nasser & A.H.M.R. Imon, 2016. "Identification and classification of multiple outliers, high leverage points and influential observations in linear regression," Journal of Applied Statistics, Taylor & Francis Journals, vol. 43(3), pages 509-525, March.
    15. Jiří Schwarz & Martin Pospíšil, 2018. "Bankruptcy, Investment, and Financial Constraints: Evidence from the Czech Republic," Eastern European Economics, Taylor & Francis Journals, vol. 56(2), pages 99-121, March.
    16. AMENDOLA, Adalgiso & BARRA, Cristian & BOCCIA, Marinella & PAPACCIO, Anna, 2018. "Market Structure and Financial Stability: Theory and Evidence," CELPE Discussion Papers 156, CELPE - CEnter for Labor and Political Economics, University of Salerno, Italy.
    17. Bedri Kamil Onur Tas, 2024. "A machine learning approach to detect collusion in public procurement with limited information," Journal of Computational Social Science, Springer, vol. 7(2), pages 1913-1935, October.
    18. Priyanga Dilini Talagala & Rob J Hyndman & Kate Smith-Miles, 2019. "Anomaly Detection in High Dimensional Data," Monash Econometrics and Business Statistics Working Papers 20/19, Monash University, Department of Econometrics and Business Statistics.
    19. Fratianni, Michele & Marchionne, Francesco, 2017. "Bank asset reallocation and sovereign debt," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 47(C), pages 15-32.
    20. Kosiorowski Daniel & Mielczarek Dominik & Rydlewski Jerzy P. & Snarska Małgorzata, 2018. "Generalized Exponential Smoothing In Prediction Of Hierarchical Time Series," Statistics in Transition New Series, Statistics Poland, vol. 19(2), pages 331-350, June.

    More about this item

    Keywords

    outlier detection; dimension reduction; outlier visualization; basis vectors;
    All these keywords.

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C38 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Classification Methdos; Cluster Analysis; Principal Components; Factor Analysis
    • C88 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Other Computer Software

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:msh:ebswps:2019-17. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Professor Xibin Zhang (email available below). General contact details of provider: https://edirc.repec.org/data/dxmonau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.