Advances in optical and electrophysiological recording technologies have made
it possible to record the dynamics of thousands of neurons, opening up new
possibilities for interpreting and controlling large neural populations in
behaving animals. A promising way to extract computational principles from
these large datasets is to train data-constrained recurrent neural networks
(dRNNs). Performing this training in real-time could open doors for research
techniques and medical applications to model and control interventions at
single-cell resolution and drive desired forms of animal behavior. However,
existing training algorithms for dRNNs are inefficient and have limited
scalability, making it a challenge to analyze large neural recordings even in
offline scenarios. To address these issues, we introduce a training method
termed Convex Optimization of Recurrent Neural Networks (CORNN). In studies of
simulated recordings, CORNN attained training speeds ~100-fold faster than
traditional optimization approaches while maintaining or enhancing modeling
accuracy. We further validated CORNN on simulations with thousands of cells
that performed simple computations such as those of a 3-bit flip-flop or the
execution of a timed response. Finally, we showed that CORNN can robustly
reproduce network dynamics and underlying attractor structures despite
mismatches between generator and inference models, severe subsampling of
observed neurons, or mismatches in neural time-scales. Overall, by training
dRNNs with millions of parameters in subminute processing times on a standard
computer, CORNN constitutes a first step towards real-time network reproduction
constrained on large-scale neural recordings and a powerful computational tool
for advancing the understanding of neural computation.Comment: Accepted at NeurIPS 202