IDEAS home Printed from https://ideas.repec.org/p/msh/ebswps/2019-17.html
   My bibliography  Save this paper

Dimension Reduction For Outlier Detection Using DOBIN

Author

Listed:
  • Sevvandi Kandanaarachchi
  • Rob J Hyndman

Abstract

This paper introduces DOBIN, a new approach to select a set of basis vectors tailored for outlier detection. DOBIN has a solid mathematical foundation and can be used as a dimension reduction tool for outlier detection tasks. We demonstrate the effectiveness of DOBIN on an extensive data repository, by comparing the performance of outlier detection methods using DOBIN and other bases. We further illustrate the utility of DOBIN as an outlier visualization tool. The R package dobin implements this basis construction.

Suggested Citation

  • Sevvandi Kandanaarachchi & Rob J Hyndman, 2019. "Dimension Reduction For Outlier Detection Using DOBIN," Monash Econometrics and Business Statistics Working Papers 17/19, Monash University, Department of Econometrics and Business Statistics.
    • Handle: RePEc:msh:ebswps:2019-17
    as

    Download full text from publisher

    File URL: https://www.monash.edu/business/ebs/research/publications/ebs/wp17-2019.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Vakili, Kaveh & Schmitt, Eric, 2014. "Finding multivariate outliers with FastPCS," Computational Statistics & Data Analysis, Elsevier, vol. 69(C), pages 54-66.
    2. Sevvandi Kandanaarachchi & Mario A Munoz & Rob J Hyndman & Kate Smith-Miles, 2018. "On normalization and algorithm selection for unsupervised outlier detection," Monash Econometrics and Business Statistics Working Papers 16/18, Monash University, Department of Econometrics and Business Statistics.
    3. Markus Goldstein & Seiichi Uchida, 2016. "A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-31, April.
    4. Billor, Nedret & Hadi, Ali S. & Velleman, Paul F., 2000. "BACON: blocked adaptive computationally efficient outlier nominators," Computational Statistics & Data Analysis, Elsevier, vol. 34(3), pages 279-298, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sevvandi Kandanaarachchi & Mario A Munoz & Rob J Hyndman & Kate Smith-Miles, 2018. "On normalization and algorithm selection for unsupervised outlier detection," Monash Econometrics and Business Statistics Working Papers 16/18, Monash University, Department of Econometrics and Business Statistics.
    2. Catherine Fuss & Angelos Theodorakopoulos, 2018. "Compositional Changes in Aggregate Productivity in an Era of Globalisation and Financial Crisis," Working Papers of VIVES - Research Centre for Regional Economics 627696, KU Leuven, Faculty of Economics and Business (FEB), VIVES - Research Centre for Regional Economics.
    3. Yin, Sihua & Yang, Haidong & Xu, Kangkang & Zhu, Chengjiu & Zhang, Shaqing & Liu, Guosheng, 2022. "Dynamic real–time abnormal energy consumption detection and energy efficiency optimization analysis considering uncertainty," Applied Energy, Elsevier, vol. 307(C).
    4. Charlotte Fabri & Sam Vermeulen & Steven Van Passel & Sergei Schaub, 2024. "Crop diversification and the effect of weather shocks on Italian farmers' income and income risk," Journal of Agricultural Economics, Wiley Blackwell, vol. 75(3), pages 955-980, September.
    5. Hong Choon Ong & Ekele Alih, 2015. "A Control Chart Based on Cluster-Regression Adjustment for Retrospective Monitoring of Individual Characteristics," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-30, April.
    6. Adele Ravagnani & Fabrizio Lillo & Paola Deriu & Piero Mazzarisi & Francesca Medda & Antonio Russo, 2024. "Dimensionality reduction techniques to support insider trading detection," Papers 2403.00707, arXiv.org, revised May 2024.
    7. Davide Nicola Continanza & Andrea del Monaco & Marco di Lucido & Daniele Figoli & Pasquale Maddaloni & Filippo Quarta & Giuseppe Turturiello, 2023. "Stacking machine learning models for anomaly detection: comparing AnaCredit to other banking data sets," IFC Bulletins chapters, in: Bank for International Settlements (ed.), Data science in central banking: applications and tools, volume 59, Bank for International Settlements.
    8. Stefani, Gianluca & Gadanakis, Yiorgos & Lombardi, Ginevra Virginia & Tiberti, Marco, 2017. "The impact of financial leverage on farms capacity to react in market shocks," 2017 International Congress, August 28-September 1, 2017, Parma, Italy 261156, European Association of Agricultural Economists.
    9. Peter Nemec, 2024. "Contesting the public works domain: examining the factors affecting presence and success of SMES in public procurement," Empirical Economics, Springer, vol. 67(5), pages 2135-2173, November.
    10. Batalla-Bejerano, Joan & Costa-Campi, Maria Teresa & Trujillo-Baute, Elisa, 2016. "Collateral effects of liberalisation: Metering, losses, load profiles and cost settlement in Spain’s electricity system," Energy Policy, Elsevier, vol. 94(C), pages 421-431.
    11. A.A.M. Nurunnabi & M. Nasser & A.H.M.R. Imon, 2016. "Identification and classification of multiple outliers, high leverage points and influential observations in linear regression," Journal of Applied Statistics, Taylor & Francis Journals, vol. 43(3), pages 509-525, March.
    12. Jiří Schwarz & Martin Pospíšil, 2018. "Bankruptcy, Investment, and Financial Constraints: Evidence from the Czech Republic," Eastern European Economics, Taylor & Francis Journals, vol. 56(2), pages 99-121, March.
    13. Kosiorowski Daniel & Mielczarek Dominik & Rydlewski Jerzy P. & Snarska Małgorzata, 2018. "Generalized Exponential Smoothing In Prediction Of Hierarchical Time Series," Statistics in Transition New Series, Statistics Poland, vol. 19(2), pages 331-350, June.
    14. José Ignacio Giménez-Nadal & José Alberto Molina & Jorge Velilla, 2024. "Intermediate activities while commuting," Review of Economics of the Household, Springer, vol. 22(3), pages 1185-1220, September.
    15. Balcazar Salazar,Carlos Felipe, 2015. "Long-run effects of democracy on income inequality : evidence from repeated cross-sections," Policy Research Working Paper Series 7153, The World Bank.
    16. Claude Diebolt & Audrey-Rose Menard & Faustine Perrin, 2017. "Behind the fertility–education nexus: what triggered the French development process?," European Review of Economic History, European Historical Economics Society, vol. 21(4), pages 357-392.
    17. Basher, Syed Abul & Raboy, David G. & Kaitibie, Simeon & Hossain, Ishrat, 2012. "The economics of food security in Arab micro states: preliminary evidence from micro data," MPRA Paper 39357, University Library of Munich, Germany.
    18. M. Hubert & P. Rousseeuw & K. Vakili, 2014. "Shape bias of robust covariance estimators: an empirical study," Statistical Papers, Springer, vol. 55(1), pages 15-28, February.
    19. Oyvat, Cem, 2016. "Agrarian Structures, Urbanization, and Inequality," World Development, Elsevier, vol. 83(C), pages 207-230.
    20. Barra, Cristian & Zotti, Roberto, 2017. "On the relationship between bank market concentration and stability of financial institutions: Evidence from the Italian banking sector," MPRA Paper 79900, University Library of Munich, Germany.

    More about this item

    Keywords

    outlier detection; dimension reduction; outlier visualization; basis vectors;
    All these keywords.

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C38 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Classification Methdos; Cluster Analysis; Principal Components; Factor Analysis
    • C88 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Other Computer Software

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:msh:ebswps:2019-17. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Professor Xibin Zhang (email available below). General contact details of provider: https://edirc.repec.org/data/dxmonau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.