Conceptually, Neural Ordinary Differential Equations (NeuralODEs) pose an
attractive way to extract dynamical laws from time series data, as they are
natural extensions of the traditional differential equation-based modeling
paradigm of the physical sciences. In practice, NeuralODEs display long
training times and suboptimal results, especially for longer duration data
where they may fail to fit the data altogether. While methods have been
proposed to stabilize NeuralODE training, many of these involve placing a
strong constraint on the functional form the trained NeuralODE can take that
the actual underlying governing equation does not guarantee satisfaction. In
this work, we present a novel NeuralODE training algorithm that leverages tools
from the chaos and mathematical optimization communities - synchronization and
homotopy optimization - for a breakthrough in tackling the NeuralODE training
obstacle. We demonstrate architectural changes are unnecessary for effective
NeuralODE training. Compared to the conventional training methods, our
algorithm achieves drastically lower loss values without any changes to the
model architectures. Experiments on both simulated and real systems with
complex temporal behaviors demonstrate NeuralODEs trained with our algorithm
are able to accurately capture true long term behaviors and correctly
extrapolate into the future.Comment: 12 pages, 6 figures, submitted to ICLR202