Bio-inspired learning and hardware acceleration with emerging memories

Machine Learning has permeated many aspects of engineering, ranging from the Internet of Things (IoT) applications to big data analytics. While computing resources available to implement these algorithms have become more powerful, both in terms of the complexity of problems that can be solved and the overall computing speed, the huge energy costs involved remains a significant challenge. The human brain, which has evolved over millions of years, is widely accepted as the most efficient control and cognitive processing platform. Neuro-biological studies have established that information processing in the human brain relies on impulse like signals emitted by neurons called action potentials. Motivated by these facts, the Spiking Neural Networks (SNNs), which are a bio-plausible version of neural networks have been proposed as an alternative computing paradigm where the timing of spikes generated by artificial neurons is central to its learning and inference capabilities. This dissertation demonstrates the computational power of the SNNs using conventional CMOS and emerging nanoscale hardware platforms. The first half of this dissertation presents an SNN architecture which is trained using a supervised spike-based learning algorithm for the handwritten digit classification problem. This network achieves an accuracy of 98.17% on the MNIST test data-set, with about 4X fewer parameters compared to the state-of-the-art neural networks achieving over 99% accuracy. In addition, a scheme for parallelizing and speeding up the SNN simulation on a GPU platform is presented. The second half of this dissertation presents an optimal hardware design for accelerating SNN inference and training with SRAM (Static Random Access Memory) and nanoscale non-volatile memory (NVM) crossbar arrays. Three prominent NVM devices are studied for realizing hardware accelerators for SNNs: Phase Change Memory (PCM), Spin Transfer Torque RAM (STT-RAM) and Resistive RAM (RRAM). The analysis shows that a spike-based inference engine with crossbar arrays of STT-RAM bit-cells is 2X and 5X more efficient compared to PCM and RRAM memories, respectively. Furthermore, the STT-RAM design has nearly 6X higher throughput per unit Watt per unit area than that of an equivalent SRAM-based (Static Random Access Memory) design. A hardware accelerator with on-chip learning on an STT-RAM memory array is also designed, requiring $16$ bits of floating-point synaptic weight precision to reach the baseline SNN algorithmic performance on the MNIST dataset. The complete design with STT-RAM crossbar array achieves nearly 20X higher throughput per unit Watt per unit mm^2 than an equivalent design with SRAM memory. In summary, this work demonstrates the potential of spike-based neuromorphic computing algorithms and its efficient realization in hardware based on conventional CMOS as well as emerging technologies. The schemes presented here can be further extended to design spike-based systems that can be ubiquitously deployed for energy and memory constrained edge computing applications

SpiNNaker - A Spiking Neural Network Architecture

20 years in conception and 15 in construction, the SpiNNaker project has delivered the world’s largest neuromorphic computing platform incorporating over a million ARM mobile phone processors and capable of modelling spiking neural networks of the scale of a mouse brain in biological real time. This machine, hosted at the University of Manchester in the UK, is freely available under the auspices of the EU Flagship Human Brain Project. This book tells the story of the origins of the machine, its development and its deployment, and the immense software development effort that has gone into making it openly available and accessible to researchers and students the world over. It also presents exemplar applications from ‘Talk’, a SpiNNaker-controlled robotic exhibit at the Manchester Art Gallery as part of ‘The Imitation Game’, a set of works commissioned in 2016 in honour of Alan Turing, through to a way to solve hard computing problems using stochastic neural networks. The book concludes with a look to the future, and the SpiNNaker-2 machine which is yet to come

SpiNNaker - A Spiking Neural Network Architecture

20 years in conception and 15 in construction, the SpiNNaker project has delivered the world’s largest neuromorphic computing platform incorporating over a million ARM mobile phone processors and capable of modelling spiking neural networks of the scale of a mouse brain in biological real time. This machine, hosted at the University of Manchester in the UK, is freely available under the auspices of the EU Flagship Human Brain Project. This book tells the story of the origins of the machine, its development and its deployment, and the immense software development effort that has gone into making it openly available and accessible to researchers and students the world over. It also presents exemplar applications from ‘Talk’, a SpiNNaker-controlled robotic exhibit at the Manchester Art Gallery as part of ‘The Imitation Game’, a set of works commissioned in 2016 in honour of Alan Turing, through to a way to solve hard computing problems using stochastic neural networks. The book concludes with a look to the future, and the SpiNNaker-2 machine which is yet to come

Brain-Inspired Computing

This open access book constitutes revised selected papers from the 4th International Workshop on Brain-Inspired Computing, BrainComp 2019, held in Cetraro, Italy, in July 2019. The 11 papers presented in this volume were carefully reviewed and selected for inclusion in this book. They deal with research on brain atlasing, multi-scale models and simulation, HPC and data infra-structures for neuroscience as well as artificial and natural neural architectures

Real-time implementation of 3D LiDAR point cloud semantic segmentation in an FPGA

Dissertação de mestrado em Informatics EngineeringIn the last few years, the automotive industry has relied heavily on deep learning applications for perception solutions. With data-heavy sensors, such as LiDAR, becoming a standard, the task of developing low-power and real-time applications has become increasingly more challenging. To obtain the maximum computational efficiency, no longer can one focus solely on the software aspect of such applications, while disregarding the underlying hardware. In this thesis, a hardware-software co-design approach is used to implement an inference application leveraging the SqueezeSegV3, a LiDAR-based convolutional neural network, on the Versal ACAP VCK190 FPGA. Automotive requirements carefully drive the development of the proposed solution, with real-time performance and low power consumption being the target metrics. A first experiment validates the suitability of Xilinx’s Vitis-AI tool for the deployment of deep convolutional neural networks on FPGAs. Both the ResNet-18 and SqueezeNet neural networks are deployed to the Zynq UltraScale+ MPSoC ZCU104 and Versal ACAP VCK190 FPGAs. The results show that both networks achieve far more than the real-time requirements while consuming low power. Compared to an NVIDIA RTX 3090 GPU, the performance per watt during both network’s inference is 12x and 47.8x higher and 15.1x and 26.6x higher respectively for the Zynq UltraScale+ MPSoC ZCU104 and the Versal ACAP VCK190 FPGA. These results are obtained with no drop in accuracy in the quantization step. A second experiment builds upon the results of the first by deploying a real-time application containing the SqueezeSegV3 model using the Semantic-KITTI dataset. A framerate of 11 Hz is achieved with a peak power consumption of 78 Watts. The quantization step results in a minimal accuracy and IoU degradation of 0.7 and 1.5 points respectively. A smaller version of the same model is also deployed achieving a framerate of 19 Hz and a peak power consumption of 76 Watts. The application performs semantic segmentation over all the point cloud with a field of view of 360°.Nos últimos anos a indústria automóvel tem cada vez mais aplicado deep learning para solucionar problemas de perceção. Dado que os sensores que produzem grandes quantidades de dados, como o LiDAR, se têm tornado standard, a tarefa de desenvolver aplicações de baixo consumo energético e com capacidades de reagir em tempo real tem-se tornado cada vez mais desafiante. Para obter a máxima eficiência computacional, deixou de ser possível focar-se apenas no software aquando do desenvolvimento de uma aplicação deixando de lado o hardware subjacente. Nesta tese, uma abordagem de desenvolvimento simultâneo de hardware e software é usada para implementar uma aplicação de inferência usando o SqueezeSegV3, uma rede neuronal convolucional profunda, na FPGA Versal ACAP VCK190. São os requisitos automotive que guiam o desenvolvimento da solução proposta, sendo a performance em tempo real e o baixo consumo energético, as métricas alvo principais. Uma primeira experiência valida a aptidão da ferramenta Vitis-AI para a implantação de redes neuronais convolucionais profundas em FPGAs. As redes ResNet-18 e SqueezeNet são ambas implantadas nas FPGAs Zynq UltraScale+ MPSoC ZCU104 e Versal ACAP VCK190. Os resultados mostram que ambas as redes ultrapassam os requisitos de tempo real consumindo pouca energia. Comparado com a GPU NVIDIA RTX 3090, a performance por Watt durante a inferência de ambas as redes é superior em 12x e 47.8x e 15.1x e 26.6x respetivamente na Zynq UltraScale+ MPSoC ZCU104 e na Versal ACAP VCK190. Estes resultados foram obtidos sem qualquer perda de accuracy na etapa de quantização. Uma segunda experiência é feita no seguimento dos resultados da primeira, implantando uma aplicação de inferência em tempo real contendo o modelo SqueezeSegV3 e usando o conjunto de dados Semantic-KITTI. Um framerate de 11 Hz é atingido com um pico de consumo energético de 78 Watts. O processo de quantização resulta numa perda mínima de accuracy e IoU com valores de 0.7 e 1.5 pontos respetivamente. Uma versão mais pequena do mesmo modelo é também implantada, atingindo uma framerate de 19 Hz e um pico de consumo energético de 76 Watts. A aplicação desenvolvida executa segmentação semântica sobre a totalidade das nuvens de pontos LiDAR, com um campo de visão de 360°

Applications in Electronics Pervading Industry, Environment and Society

This book features the manuscripts accepted for the Special Issue “Applications in Electronics Pervading Industry, Environment and Society—Sensing Systems and Pervasive Intelligence” of the MDPI journal Sensors. Most of the papers come from a selection of the best papers of the 2019 edition of the “Applications in Electronics Pervading Industry, Environment and Society” (APPLEPIES) Conference, which was held in November 2019. All these papers have been significantly enhanced with novel experimental results. The papers give an overview of the trends in research and development activities concerning the pervasive application of electronics in industry, the environment, and society. The focus of these papers is on cyber physical systems (CPS), with research proposals for new sensor acquisition and ADC (analog to digital converter) methods, high-speed communication systems, cybersecurity, big data management, and data processing including emerging machine learning techniques. Physical implementation aspects are discussed as well as the trade-off found between functional performance and hardware/system costs

A Steering Wheel Mounted Grip Sensor: Design, Development and Evaluation

Department of Human Factors EngineeringDriving is a commonplace but safety critical daily activity for billions of people. It remains one of the leading causes of death worldwide, particularly in younger adults. In the last decades, a wide range of technologies, such as intelligent braking or speed regulating systems, have been integrated into vehicles to improve safetyannually decreasing death rates testify to their success. A recent research focus in this area has been in the development of systems that sense human states or activities during driving. This is valuable because human error remains a key reason underlying many vehicle accidents and incidents. Technologies that can intervene in response to information sensed about a driver may be able to detect, predict and ultimately prevent problems before they progress into accidents, thus avoiding the occurrence of critical situations rather than just mitigating their consequences. Commercial examples of this kind of technology include systems that monitor driver alertness or lane holding and prompt drivers who are sleepy or drifting off-lane. More exploratory research in this area has sought to capture emotional state or stress/workload levels via physiological measurements of Heart Rate Variability (HRV), Electrocardiogram (ECG) and Electroencephalogram (EEG), or behavioral measurements of eye gaze or face pose. Other research has monitored explicitly user actions, such as head pose or foot movements to infer intended actions (such as overtaking or lane change) and provide automatic assessments of the safety of these future behaviors ??? for example, providing a timely warning to a driver who is planning to overtake about a vehicle in his or her blind spot. Researchers have also explored how sensing hands on the wheel can be used to infer a driver???s presence, identity or emotional state. This thesis extends this body of work through the design, development and evaluation of a steering wheel sensor platform that can directly detect a driver???s hand pose all around a steering wheel. This thesis argues that full steering hand pose is a potentially rich source of information about a driver???s intended actions. For example, it proposes a link between hand posture on the wheel and subsequent turning or lane change behavior. To explore this idea, this thesis describes the construction of a touch sensor in the form of a steering wheel cover. This cover integrates 32 equidistantly spread touch sensing electrodes (11.250 inter-sensor spacing) in the form of conductive ribbons (0.2" wide and 0.03" thick). Data from each ribbons is captured separately via a set of capacitive touch sensor microcontrollers every 64 ms. We connected this hardware platform to an OpenDS, an open source driving simulator and ran two studies capturing hand pose during a sequential lane change task and a slalom task. We analyzed the data to determine whether hand pose is a useful predictor of future turning behavior. For this we classified a 5-lane road into 4 turn sizes and used machine-learning recognizers to predict the future turn size from the change in hand posture in terms of hand movement properties from the early driving data. Driving task scenario of the first experiment was not appropriately matched with the real life turning task therefore we modified the scenario with more appropriate task in the second experiments. Class-wise prediction of the turn sizes for both experiments didn???t show good accuracy, however prediction accuracy was improved when the classes were reduced into two classes from four classes. In the experiment 2 turn sizes were overlapped between themselves, which made it very difficult to distinguish them. Therefore, we did continuous prediction as well and the prediction accuracy was better than the class-wise prediction system for the both experiments. In summary, this thesis designed, developed and evaluated a combined hardware and software system that senses the steering behavior of a driver by capturing grip pose. We assessed the value of this information via two studies that explored the relationship between wheel grip and future turning behaviors. The ultimate outcome of this study can inform the development of in car sensing systems to support safer driving.ope

Digitalisation and COVID-19: The Perfect Storm

\u201cA ship in the harbour is safe, but that is not what ships are built for,\u201d observed that sage 19th century philosopher William Shedd. In other words, technology of high potential is of little value if the potential is not exploited. As the shape of 2020 is increasingly defined by the coronavirus pandemic, digitalisation is like a ship loaded with technology that has a huge capacity for transforming mankind\u2019s combat against infectious disease. But it is still moored safely in harbour. Instead of sailing bravely into battle, it remains at the dockside, cowering from the storm beyond the breakwaters. Engineers and fitters constantly fine-tune it, and its officers and deckhands perfect their operating procedures, but that promise is unfulfilled, restrained by the hesitancy and indecision of officialdom. Out there, the seas of the pandemic are turbulent and uncharted, and it is impossible to know in advance everything of the other dangers that may lurk beyond those cloudy horizons. However, the more noble course is for orders to be given to complete the preparations, to cast off and set sail, and to join other vessels crewed by valiant healthcare workers and tireless researchers, already deeply engaged in a rescue mission for the whole of the human race. It is the destiny of digitalisation to navigate those oceans alongside other members of that task force, and the hour of destiny has arrived. This article focuses on the potential enablers and recommendation to maximise learnings during the era of COVID-19