Novelty detection based on learning entropy

The Approximate Individual Sample Learning Entropy is based on incremental learning of a predictor x˜(k+h)=ϕ(x(k),w), where x(k) is an input vector of a given size at time k, w is a vector of weights (adaptive parameters), and h is a prediction horizon. The basic assumption is that, after the underlying process x changes its behavior, the incrementally learning system will adapt the weights w to improve the predictor x˜. Our goal is to detect a change in the behavior of the weight increment process. The main idea of this paper is based on the fact that weight increments △w(k), where △w(k) = w(k + 1) − w(k), create a weakly stationary process until a change occurs. Once a novelty behavior of the underlying process x(k) occurs, the process △w(k) changes its characteristics (eg, the mean or variation). We suggest using convenient characteristics of △w(k) in a multivariate detection scheme (eg, the Hotelling's T2 control chart).