DETONATE: Nonlinear Dynamic Evolution Modeling of Time-dependent 3-dimensional Point Cloud Profiles

Modeling the evolution of a 3D profile over time as a function of heterogeneous input data and the previous time steps’ 3D shape is a challenging, yet fundamental problem in many applications. We introduce a novel methodology for the nonlinear modeling of dynamically evolving 3D shape profiles. Our model integrates heterogeneous, multimodal inputs that may affect the evolvement of the 3D shape profiles. We leverage the forward and backward temporal dynamics to preserve the underlying temporal physical structures. Our approach is based on the Koopman operator theory for high-dimensional nonlinear dynamical systems. We leverage the theoretical Koopman framework to develop a deep learning-based framework for nonlinear, dynamic 3D modeling with consistent temporal dynamics. We evaluate our method on multiple high-dimensional and short-term dependent problems, and it achieves accurate estimates, while also being robust to noise.