Characterizing time-resolved stochasticity in non-stationary time series

Time series often exhibit a combination of long-range drift and short-term stochastic fluctuations. Traditional methods for analyzing such series involve fitting regression models to capture the drift component and using the residuals to estimate the random component. We demonstrate, however, that estimating the drift in a real-time (time-resolved) manner poses significant challenges. We find, surprisingly, that contrary to conventional expectations, estimation of the drift is less accurate than evaluating short-term fluctuations in data with a given number of data points. Two factors contribute to this unexpected complexity: measurement noise, and the slower convergence rate of the drift estimation. As a result, real-time estimation of stochastic fluctuations can be more accurate. We introduce the term stochasticity, as the square of the estimated short-term fluctuations within a time window of length dt, which can be estimated in real-time (time-resolved) for given non-stationary time series and those exhibiting unique trajectories. To demonstrate the practical applications of the concept of real-time stochasticity, we calculate it for synthetic time series generated by both linear and nonlinear dynamical equations, which generate stationary and non-stationary trajectories for which we have access to the ground truth. We have also analyzed various real-world datasets: global temperature anomalies in 12 distinct geographical regions, keystroke time series from Parkinson’s disease patients, fluctuations in gold prices, atmospheric CO2 concentration, wind velocity data, and earthquake occurrences. Our method exclusively provides the time dependency, rather than both state and time dependencies, of the stochasticity.