Biomimetic Based EEG Learning for Robotics Complex Grasping and Dexterous Manipulation

There have been tremendous efforts to understand the biological nature of human grasping, in such a way that it can be learned and copied to prosthesis–robotics and dextrous grasping applications. Several biomimetic methods and techniques have been adopted, hence applied to analytically comprehend ways human performs grasping to duplicate human knowledge. A major topic for further study, is related to decoding the resulting EEG brainwaves during motorizing of fingers and moving parts. To accomplish this, there are a number of phases that are performed, including recording, pre-processing, filtration, and understanding of the waves. However, there are two important phases that have received substantial research attentions. The classification and decoding, of such massive and complex brain waves, as they are two important steps towards understanding patterns during grasping. In this respect, the fundamental objective of this research is to demonstrate how to employ advanced pattern recognition methods, like fuzzy c-mean clustering for understanding resulting EEG brain waves, in such a way to control a prosthesis or robotic hand, while relying sets of detected EEG brainwaves. There are a number of decoding and classification methods and techniques, however we shall look into fuzzy based clustering blended with principle component analysis (PAC) technique to help for the decoding mechanism. EEG brainwaves during a grasping and manipulation have been used for this analysis. This involves, movement of almost five fingers during a grasping defined task. The study has found that, it is not a straight forward task to decode all human fingers motions, as due to the complexity of grasping tasks. However, the adopted analysis was able to classify and identify the different narrowly performed and related fundamental events during a simple grasping task

Data-Driven Transducer Design and Identification for Internally-Paced Motor Brain Computer Interfaces: A Review

Brain-Computer Interfaces (BCIs) are systems that establish a direct communication pathway between the users' brain activity and external effectors. They offer the potential to improve the quality of life of motor-impaired patients. Motor BCIs aim to permit severely motor-impaired users to regain limb mobility by controlling orthoses or prostheses. In particular, motor BCI systems benefit patients if the decoded actions reflect the users' intentions with an accuracy that enables them to efficiently interact with their environment. One of the main challenges of BCI systems is to adapt the BCI's signal translation blocks to the user to reach a high decoding accuracy. This paper will review the literature of data-driven and user-specific transducer design and identification approaches and it focuses on internally-paced motor BCIs. In particular, continuous kinematic biomimetic and mental-task decoders are reviewed. Furthermore, static and dynamic decoding approaches, linear and non-linear decoding, offline and real-time identification algorithms are considered. The current progress and challenges related to the design of clinical-compatible motor BCI transducers are additionally discussed

Enhancing Upper Limb Prostheses Through Neuromorphic Sensory Feedback

Upper limb prostheses are rapidly improving in terms of both control and sensory feedback, giving rise to lifelike robotic devices that aim to restore function to amputees. Recent progress in forward control has enabled prosthesis users to make complicated grip patterns with a prosthetic hand and nerve stimulation has enabled sensations of touch in the missing hand of an amputee. A brief overview of the motivation behind the work in this thesis is given in Chapter 1, which is followed by a general overview of the field and state of the art research (Chapter 2). Chapters 3 and 4 look at the use of closed loop tactile feedback for improving prosthesis grasping functionality. This entails development of two algorithms for improving object manipulation (Chapter 3) and the first real-time implementation of neuromorphic tactile signals being used as feedback to a prosthesis controller for improved grasping (Chapter 4). The second half of the thesis (Chatpers 5 - 7) details how sensory information can be conveyed back to an amputee and how the tactile sensations can be utilized for creating a more lifelike prosthesis. Noninvasive electrical nerve stimulation was shown to provide sensations in multiple regions of the phantom hand of amputees both with and without targeted sensory reinnervation surgery (Chapter 5). A multilayered electronic dermis (e-dermis) was developed to mimic the behavior of receptors in the skin to provide, for the first time, sensations of both touch and pain back to an amputee and the prosthesis (Chapter 6). Finally, the first demonstration of sensory feedback as a key component of phantom hand movement for myoelectric pattern recognition shows that enhanced perceptions of the phantom hand can lead to improved prosthesis control (Chapter 7). This work provides the first demonstration of how amputees can perceive multiple tactile sensations through a neuromorphic stimulation paradigm. Furthermore, it describes the unique role that nerve stimulation and phantom hand activation play in the sensorimotor loop of upper limb amputees

VALIDATION OF A MODEL OF SENSORIMOTOR INTEGRATION WITH CLINICAL BENEFITS

Healthy sensorimotor integration – or how our touch influences our movements – is critical to efficiently interact with our environment. Yet, many aspects of this process are still poorly understood. Importantly, several movement disorders are often considered as originating from purely motor impairments, while a sensory origin could also lead to a similar set of symptoms. To alleviate these issues, we hereby propose a novel biologically-based model of the sensorimotor loop, known as the SMILE model. After describing both the functional, and the corresponding neuroanatomical versions of the SMILE, we tested several aspects of its motor component through functional magnetic resonance imaging (fMRI) and transcranial magnetic stimulation (TMS). Both experimental studies resulted in coherent outcomes with respect to the SMILE predictions, but they also provided novel scientific outcomes about such broad topics as the sub-phases of motor imagery, the neural processing of bodily representations, or the extend of the role of the extrastriate body area. In the final sections of this manuscript, we describe some potential clinical application of the SMILE. The first one presents the identification of plausible neuroanatomical origins for focal hand dystonia, a yet poorly understood sensorimotor disorder. The last chapter then covers possible improvements on brain-machine interfaces, driven by a better understanding of the sensorimotor system. -- La façon dont votre sens du toucher et vos mouvements interagissent est connue sous le nom d’intégration sensorimotrice. Ce procédé est essentiel pour une interaction normale avec tout ce qui nous entoure. Cependant, plusieurs aspects de ce processus sont encore méconnus. Plus important encore, l’origine de certaines déficiences motrices encore trop peu comprises sont parfois considérées comme purement motrice, alors qu’une origine sensorielle pourrait mener à un même ensemble de symptômes. Afin d’améliorer cette situation, nous proposons ici un nouveau modèle d’intégration sensorimotrice, dénommé « SMILE », basé sur les connaissances de neurobiologie actuelles. Dans ce manuscrit, nous commençons par décrire les caractéristiques fonctionnelles et neuroanatomiques du SMILE. Plusieurs expériences sont ensuite effectuées, via l’imagerie par résonance magnétique fonctionnelle (IRMf), et la stimulation magnétique transcranienne (SMT), afin de tester différents aspects de la composante motrice du SMILE. Si les résultats de ces expériences corroborent les prédictions du SMILE, elles ont aussi mis en évidences d’autres résultats scientifiques intéressants et novateurs, dans des domaines aussi divers que les sous-phases de l’imagination motrice, les processus cérébraux liés aux représentations corporelles, ou encore l’extension du rôle de l’extrastriate body area. Dans les dernières parties de ce manuscrit, nous dévoilons quelques applications cliniques potentielles de notre modèle. Nous utilisons le SMILE afin de proposer deux origines cérébrales plausibles de la dystonie focale de la main. Le dernier chapitre présente comment certaines technologies existantes, telles que les interfaces cerveaux-machines, pourraient bénéficier d’une meilleure compréhension du système sensorimoteur

A Survey of Spiking Neural Network Accelerator on FPGA

Due to the ability to implement customized topology, FPGA is increasingly used to deploy SNNs in both embedded and high-performance applications. In this paper, we survey state-of-the-art SNN implementations and their applications on FPGA. We collect the recent widely-used spiking neuron models, network structures, and signal encoding formats, followed by the enumeration of related hardware design schemes for FPGA-based SNN implementations. Compared with the previous surveys, this manuscript enumerates the application instances that applied the above-mentioned technical schemes in recent research. Based on that, we discuss the actual acceleration potential of implementing SNN on FPGA. According to our above discussion, the upcoming trends are discussed in this paper and give a guideline for further advancement in related subjects

Pattern Recognition

Pattern recognition is a very wide research field. It involves factors as diverse as sensors, feature extraction, pattern classification, decision fusion, applications and others. The signals processed are commonly one, two or three dimensional, the processing is done in real- time or takes hours and days, some systems look for one narrow object class, others search huge databases for entries with at least a small amount of similarity. No single person can claim expertise across the whole field, which develops rapidly, updates its paradigms and comprehends several philosophical approaches. This book reflects this diversity by presenting a selection of recent developments within the area of pattern recognition and related fields. It covers theoretical advances in classification and feature extraction as well as application-oriented works. Authors of these 25 works present and advocate recent achievements of their research related to the field of pattern recognition

SOUNDS OF SALIENCE: GENERALIZING AUDIO EVENT DETECTION

Audio Event detection (AED) is a technology aimed at detecting and classifying sound events within an audio signal. AED plays a critical role in enabling machines to understand audio content in various contexts and has direct applications in content retrieval, audio analytics, and surveillance systems, among others. In this dissertation, we revise the classic paradigm of AED to introduce a nuanced, real-valued "degree of presence" for individual audio objects within complex auditory soundscapes. This transformative approach offers a richer and more accurate portrayal that closely mirrors human perception of sound with significant implications for diverse applications. To lay the groundwork for this approach, we tackle the crucial concept of auditory salience, which measures an object's capacity to command attention. While visual salience is well understood, the auditory counterpart has remained unexplored due to challenges in capturing attention in free-listening conditions. We surmount this hurdle by employing an inventive crowd-sourcing-based dichotic salience paradigm. By rigorously validating the reliability of crowd-sourced data through comparison with controlled laboratory settings, we prove the efficacy of this novel methodology and pave the way for collecting large-scale, diverse auditory salience datasets. Moreover, we expand the frontier of auditory salience research by exploring the often-overlooked semantic dimensions. Through carefully designed experimental studies that manipulate the direction of audio scenes, we reveal insights about how semantic cues significantly influence auditory salience in addition to acoustic attributes. Using data from the crowd-sourced dichotic paradigm and predictive models of salience and salient events, we establish that perceptual salience balances low-level and high-level attributes in guiding what stands out in a natural scene. In addition to the salience models, we developed robust methodologies to detect audio events with various dynamics. Mimicking how the human auditory cortex performs a rate-specific analysis that can selectively track objects over time, we developed deep-learning models whose latent spaces are constrained to follow specific dynamics. We leveraged large-scale unlabeled data to train rate-specific audio encoders using the constraints as priors in a variational framework. These rate-specific encoders provide performance gains when fine-tuned using a semi-supervised AED framework. We also developed a coherence-based regularization that enforces smoothness constraints on the latent space, which can lead to further gains in AED performance

2022 roadmap on neuromorphic computing and engineering

Modern computation based on von Neumann architecture is now a mature cutting-edge science. In the von Neumann architecture, processing and memory units are implemented as separate blocks interchanging data intensively and continuously. This data transfer is responsible for a large part of the power consumption. The next generation computer technology is expected to solve problems at the exascale with 10$^{18}$ calculations each second. Even though these future computers will be incredibly powerful, if they are based on von Neumann type architectures, they will consume between 20 and 30 megawatts of power and will not have intrinsic physically built-in capabilities to learn or deal with complex data as our brain does. These needs can be addressed by neuromorphic computing systems which are inspired by the biological concepts of the human brain. This new generation of computers has the potential to be used for the storage and processing of large amounts of digital information with much lower power consumption than conventional processors. Among their potential future applications, an important niche is moving the control from data centers to edge devices. The aim of this roadmap is to present a snapshot of the present state of neuromorphic technology and provide an opinion on the challenges and opportunities that the future holds in the major areas of neuromorphic technology, namely materials, devices, neuromorphic circuits, neuromorphic algorithms, applications, and ethics. The roadmap is a collection of perspectives where leading researchers in the neuromorphic community provide their own view about the current state and the future challenges for each research area. We hope that this roadmap will be a useful resource by providing a concise yet comprehensive introduction to readers outside this field, for those who are just entering the field, as well as providing future perspectives for those who are well established in the neuromorphic computing community