Dynamical Systems in Spiking Neuromorphic Hardware

Dynamical systems are universal computers. They can perceive stimuli, remember, learn from feedback, plan sequences of actions, and coordinate complex behavioural responses. The Neural Engineering Framework (NEF) provides a general recipe to formulate models of such systems as coupled sets of nonlinear differential equations and compile them onto recurrently connected spiking neural networks – akin to a programming language for spiking models of computation. The Nengo software ecosystem supports the NEF and compiles such models onto neuromorphic hardware. In this thesis, we analyze the theory driving the success of the NEF, and expose several core principles underpinning its correctness, scalability, completeness, robustness, and extensibility. We also derive novel theoretical extensions to the framework that enable it to far more effectively leverage a wide variety of dynamics in digital hardware, and to exploit the device-level physics in analog hardware. At the same time, we propose a novel set of spiking algorithms that recruit an optimal nonlinear encoding of time, which we call the Delay Network (DN). Backpropagation across stacked layers of DNs dramatically outperforms stacked Long Short-Term Memory (LSTM) networks—a state-of-the-art deep recurrent architecture—in accuracy and training time, on a continuous-time memory task, and a chaotic time-series prediction benchmark. The basic component of this network is shown to function on state-of-the-art spiking neuromorphic hardware including Braindrop and Loihi. This implementation approaches the energy-efficiency of the human brain in the former case, and the precision of conventional computation in the latter case

Benchmarks and Controls for Optimization with Quantum Annealing

Quantum annealing (QA) is a metaheuristic specialized for solving optimization problems which uses principles of adiabatic quantum computing, namely the adiabatic theorem. Some devices implement QA using quantum mechanical phenomena. These QA devices do not perfectly adhere to the adiabatic theorem because they are subject to thermal and magnetic noise. Thus, QA devices return statistical solutions with some probability of success where this probability is affected by the level of noise of the system. As these devices improve, it is believed that they will become less noisy and more accurate. However, some tuning strategies may further improve that probability of finding the correct solution and reduce the effects of noise on solution outcome. In this dissertation, these tuning strategies are explored in depth to determine the effect of preprocessing, annealing, and post-processing controls on performance. In particular, these tuning strategies were applied to a real-world NP (nondeterministic polynomial time)-hard optimization problem and portfolio optimization. Although the performance improved very little from tuning the spin reversal transforms, anneal time, and embedding, the results revealed that reverse annealing controls improved the probability of success by an order of magnitude over forward annealing alone. The chain strength experiments revealed that increasing the strength of the intra-chain coupling improves the probability of success until the intra-chain coupling strengths begin to overpower the inter-chain couplings. By taking a closer look at each physical qubit in the embedded chains, the probability for each qubit to be faulty was visualized and was used to develop a post-processing strategy that outperformed the standard, which chooses a logical qubit value from a broken chain. The results of these findings provide a guide for researchers to find the optimal set of controls for their unique real-world optimization problem to determine whether QA provides some benefit over classical computing, lay the groundwork for developing new tuning strategies that could further improve performance, and characterize the current hardware for benchmarking future generations of QA hardware