The proliferation of multi-unit cortical recordings over the last two
decades, especially in macaques and during motor-control tasks, has generated
interest in neural "population dynamics": the time evolution of neural activity
across a group of neurons working together. A good model of these dynamics
should be able to infer the activity of unobserved neurons within the same
population and of the observed neurons at future times. Accordingly,
Pandarinath and colleagues have introduced a benchmark to evaluate models on
these two (and related) criteria: four data sets, each consisting of firing
rates from a population of neurons, recorded from macaque cortex during
movement-related tasks. Here we show that simple, general-purpose architectures
based on recurrent neural networks (RNNs) outperform more "bespoke" models, and
indeed outperform all published models on all four data sets in the benchmark.
Performance can be improved further still with a novel, hybrid architecture
that augments the RNN with self-attention, as in transformer networks. But pure
transformer models fail to achieve this level of performance, either in our
work or that of other groups. We argue that the autoregressive bias imposed by
RNNs is critical for achieving the highest levels of performance. We conclude,
however, by proposing that the benchmark be augmented with an alternative
evaluation of latent dynamics that favors generative over discriminative models
like the ones we propose in this report.Comment: 23 pages, 10 figures, 4 table