Sequential data assimilation for 1D self-exciting processes with application to urban crime data

A number of models – such as the Hawkes process and log Gaussian Cox process – have been used to understand how crime rates evolve in time and/or space. Within the context of these models and actual crime data, parameters are often estimated using maximum likelihood estimation (MLE) on batch data, but this approach has several limitations such as limited tracking in real-time and uncertainty quantification. For practical purposes, it would be desirable to move beyond batch data estimation to sequential data assimilation. A novel and general Bayesian sequential data assimilation algorithm is developed for joint state-parameter estimation for an inhomogeneous Poisson process by deriving an approximating Poisson-Gamma ‘Kalman’ filter that allows for uncertainty quantification. The ensemble-based implementation of the filter is developed in a similar approach to the ensemble Kalman filter, making the filter applicable to large-scale real world applications unlike nonlinear filters such as the particle filter. The filter has the advantage that it is independent of the underlying model for the process intensity, and can therefore be used for many different crime models, as well as other application domains. The performance of the filter is demonstrated on synthetic data and real Los Angeles gang crime data and compared against a very large sample-size particle filter, showing its effectiveness in practice. In addition the forecast skill of the Hawkes model is investigated for a forecast system using the Receiver Operating Characteristic (ROC) to provide a useful indicator for when predictive policing software for a crime type is likely to be useful. The ROC and Brier scores are used to compare and analyze the forecast skill of sequential data assimilation and MLE. It is found that sequential data assimilation produces improved probabilistic forecasts over the MLE.