Optimizing distance-based methods for large data sets

Distance-based methods for measuring spatial concentration of industries have received an increasing popularity in the spatial econometrics community. However, a limiting factor for using these methods is their computational complexity since both their memory requirements and running times are in $${\mathcal {O}}(n^2)$$ O ( n 2 ) . In this paper, we present an algorithm with constant memory requirements and shorter running time, enabling distance-based methods to deal with large data sets. We discuss three recent distance-based methods in spatial econometrics: the D&O-Index by Duranton and Overman (Rev Econ Stud 72(4):1077–1106, 2005 ), the M-function by Marcon and Puech (J Econ Geogr 10(5):745–762, 2010 ) and the Cluster-Index by Scholl and Brenner (Reg Stud (ahead-of-print):1–15, 2014 ). Finally, we present an alternative calculation for the latter index that allows the use of data sets with millions of firms. Copyright Springer-Verlag Berlin Heidelberg 2015