Driving Scene Understanding using Spiking Neural Networks

One of the applications of AI lies in developing intelligent systems for safe on-road driving, other than building and perfecting self-driving vehicles, and many others. Driving Scene Understanding (DSU) is one such area where AI algorithms can be used to infer the current actions of driver, pedestrians, nearby vehicles, etc. to improve the on-road decision making capability of the in-vehicle driver. Another related front of technological advancement in transportation is the production and development of electric vehicles. A future with battery electric vehicle and safe driving necessitates the creation of AI algorithms which not only assist in increasing the on-road safety but are also energy efficient. This thesis is an attempt towards developing such an energy efficient AI model for DSU using Spiking Neural Networks (SNNs). Low power neuromorphic hardware (e.g. Intel’s Loihi) can be leveraged for the deployment of such SNNs which offer low inference latency and energy efficiency. Out of a number of ways to build SNNs, an established method is to first train an Artificial Neural Network (ANN) with traditional neurons (e.g. ReLU) and then replace those neurons with spiking neurons (e.g. Integrate & Fire neurons) along with some other network modifications. Therefore, Chapter 4 first presents a 3D-CNN based ANN model, and identifies the appropriate spatial resolution and temporal depth of the incoming video frames for DSU. Through extensive experiments, it was found that MaxPooling performs better than AveragePooling in such a 3D-CNNs based model; however there exists no method to convert a network with MaxPooling layers into an SNN which can be entirely deployed on a specialized neuromorphic hardware. Chapter 6 presents two novel approaches to implement MaxPooling on a neuromorphic hardware; thus facilitating the conversion of networks with MaxPooling layers to fully neuromorphic-hardware compatible SNNs. These approaches have been tested with 2D-CNNs based SNNs for image recognition, and can be extended to the 3D-CNNs based SNNs as well; thus, theoretically realizing an energy efficient SNN for DSU

Exploring the potential of brain-inspired computing

The gap between brains and computers regarding both their cognitive capability and power efficiency is remarkably huge. Brains process information massively in parallel and its constituents are intrinsically self-organizing, while in digital computers the execution of instructions is deterministic and rather serial. The recent progress in the development of dedicated hardware systems implementing physical models of neurons and synapses enables to efficiently emulate spiking neural networks. In this work, we verify the design and explore the potential for brain-inspired computing of such an analog neuromorphic system, called Spikey. We demonstrate the versatility of this highly configurable substrate by the implementation of a rich repertoire of network models, including models for signal propagation and enhancement, general purpose classifiers, cortical models and decorrelating feedback systems. Network emulations on Spikey are highly accelerated and consume less than 1 nJ per synaptic transmission. The Spikey system, hence, outperforms modern desktop computers in terms of fast and efficient network simulations closing the gap to brains. During this thesis the stability, performance and user-friendliness of the Spikey system was improved integrating it into the neuroscientific tool chain and making it available for the community. The implementation of networks suitable to solve everyday tasks, like object or speech recognition, qualifies this technology to be an alternative to conventional computers. Considering the compactness, computational capability and power efficiency, neuromorphic systems may qualify as a valuable complement to classical computation