We initiate the study of biological neural networks from the perspective of
streaming algorithms. Like computers, human brains suffer from memory
limitations which pose a significant obstacle when processing large scale and
dynamically changing data. In computer science, these challenges are captured
by the well-known streaming model, which can be traced back to Munro and
Paterson `78 and has had significant impact in theory and beyond. In the
classical streaming setting, one must compute some function f of a stream of
updates S={u1,…,um}, given restricted single-pass access
to the stream. The primary complexity measure is the space used by the
algorithm.
We take the first steps towards understanding the connection between
streaming and neural algorithms. On the upper bound side, we design neural
algorithms based on known streaming algorithms for fundamental tasks, including
distinct elements, approximate median, heavy hitters, and more. The number of
neurons in our neural solutions almost matches the space bounds of the
corresponding streaming algorithms. As a general algorithmic primitive, we show
how to implement the important streaming technique of linear sketching
efficient in spiking neural networks. On the lower bound side, we give a
generic reduction, showing that any space-efficient spiking neural network can
be simulated by a space-efficiently streaming algorithm. This reduction lets us
translate streaming-space lower bounds into nearly matching neural-space lower
bounds, establishing a close connection between these two models.Comment: To appear in DISC'20, shorten abstrac