Towards Next Generation Neural Interfaces: Optimizing Power, Bandwidth and Data Quality

In this paper, we review the state-of-the-art in neural interface recording architectures. Through this we identify schemes which show the trade-off between data information quality (lossiness), computation (i.e. power and area requirements) and the number of channels. These trade-offs are then extended by considering the front-end amplifier bandwidth to also be a variable. We therefore explore the possibility of band-limiting the spectral content of recorded neural signals (to save power) and investigate the effect this has on subsequent processing (spike detection accuracy). We identify the spike detection method most robust to such signals, optimize the threshold levels and modify this to exploit such a strategy.Accepted versio

Exploiting All-Programmable System on Chips for Closed-Loop Real-Time Neural Interfaces

High-density microelectrode arrays (HDMEAs) feature thousands of recording electrodes in a single chip with an area of few square millimeters. The obtained electrode density is comparable and even higher than the typical density of neuronal cells in cortical cultures. Commercially available HDMEA-based acquisition systems are able to record the neural activity from the whole array at the same time with submillisecond resolution. These devices are a very promising tool and are increasingly used in neuroscience to tackle fundamental questions regarding the complex dynamics of neural networks. Even if electrical or optical stimulation is generally an available feature of such systems, they lack the capability of creating a closed-loop between the biological neural activity and the artificial system. Stimuli are usually sent in an open-loop manner, thus violating the inherent working basis of neural circuits that in nature are constantly reacting to the external environment. This forbids to unravel the real mechanisms behind the behavior of neural networks. The primary objective of this PhD work is to overcome such limitation by creating a fullyreconfigurable processing system capable of providing real-time feedback to the ongoing neural activity recorded with HDMEA platforms. The potentiality of modern heterogeneous FPGAs has been exploited to realize the system. In particular, the Xilinx Zynq All Programmable System on Chip (APSoC) has been used. The device features reconfigurable logic, specialized hardwired blocks, and a dual-core ARM-based processor; the synergy of these components allows to achieve high elaboration performances while maintaining a high level of flexibility and adaptivity. The developed system has been embedded in an acquisition and stimulation setup featuring the following platforms: \u2022 3\ub7Brain BioCam X, a state-of-the-art HDMEA-based acquisition platform capable of recording in parallel from 4096 electrodes at 18 kHz per electrode. \u2022 PlexStim\u2122 Electrical Stimulator System, able to generate electrical stimuli with custom waveforms to 16 different output channels. \u2022 Texas Instruments DLP\uae LightCrafter\u2122 Evaluation Module, capable of projecting 608x684 pixels images with a refresh rate of 60 Hz; it holds the function of optical stimulation. All the features of the system, such as band-pass filtering and spike detection of all the recorded channels, have been validated by means of ex vivo experiments. Very low-latency has been achieved while processing the whole input data stream in real-time. In the case of electrical stimulation the total latency is below 2 ms; when optical stimuli are needed, instead, the total latency is a little higher, being 21 ms in the worst case. The final setup is ready to be used to infer cellular properties by means of closed-loop experiments. As a proof of this concept, it has been successfully used for the clustering and classification of retinal ganglion cells (RGCs) in mice retina. For this experiment, the light-evoked spikes from thousands of RGCs have been correctly recorded and analyzed in real-time. Around 90% of the total clusters have been classified as ON- or OFF-type cells. In addition to the closed-loop system, a denoising prototype has been developed. The main idea is to exploit oversampling techniques to reduce the thermal noise recorded by HDMEAbased acquisition systems. The prototype is capable of processing in real-time all the input signals from the BioCam X, and it is currently being tested to evaluate the performance in terms of signal-to-noise-ratio improvement

NeuroBench: Advancing Neuromorphic Computing through Collaborative, Fair and Representative Benchmarking

The field of neuromorphic computing holds great promise in terms of advancing computing efficiency and capabilities by following brain-inspired principles. However, the rich diversity of techniques employed in neuromorphic research has resulted in a lack of clear standards for benchmarking, hindering effective evaluation of the advantages and strengths of neuromorphic methods compared to traditional deep-learning-based methods. This paper presents a collaborative effort, bringing together members from academia and the industry, to define benchmarks for neuromorphic computing: NeuroBench. The goals of NeuroBench are to be a collaborative, fair, and representative benchmark suite developed by the community, for the community. In this paper, we discuss the challenges associated with benchmarking neuromorphic solutions, and outline the key features of NeuroBench. We believe that NeuroBench will be a significant step towards defining standards that can unify the goals of neuromorphic computing and drive its technological progress. Please visit neurobench.ai for the latest updates on the benchmark tasks and metrics

Resource efficient on-node spike sorting

Current implantable brain-machine interfaces are recording multi-neuron activity by utilising multi-channel, multi-electrode micro-electrodes. With the rapid increase in recording capability has come more stringent constraints on implantable system power consumption and size. This is even more so with the increasing demand for wireless systems to increase the number of channels being monitored whilst overcoming the communication bottleneck (in transmitting raw data) via transcutaneous bio-telemetries. For systems observing unit activity, real-time spike sorting within an implantable device offers a unique solution to this problem. However, achieving such data compression prior to transmission via an on-node spike sorting system has several challenges. The inherent complexity of the spike sorting problem arising from various factors (such as signal variability, local field potentials, background and multi-unit activity) have required computationally intensive algorithms (e.g. PCA, wavelet transform, superparamagnetic clustering). Hence spike sorting systems have traditionally been implemented off-line, usually run on work-stations. Owing to their complexity and not-so-well scalability, these algorithms cannot be simply transformed into a resource efficient hardware. On the contrary, although there have been several attempts in implantable hardware, an implementation to match comparable accuracy to off-line within the required power and area requirements for future BMIs have yet to be proposed. Within this context, this research aims to fill in the gaps in the design towards a resource efficient implantable real-time spike sorter which achieves performance comparable to off-line methods. The research covered in this thesis target: 1) Identifying and quantifying the trade-offs on subsequent signal processing performance and hardware resource utilisation of the parameters associated with analogue-front-end. Following the development of a behavioural model of the analogue-front-end and an optimisation tool, the sensitivity of the spike sorting accuracy to different front-end parameters are quantified. 2) Identifying and quantifying the trade-offs associated with a two-stage hybrid solution to realising real-time on-node spike sorting. Initial part of the work focuses from the perspective of template matching only, while the second part of the work considers these parameters from the point of whole system including detection, sorting, and off-line training (template building). A set of minimum requirements are established which ensure robust, accurate and resource efficient operation. 3) Developing new feature extraction and spike sorting algorithms towards highly scalable systems. Based on waveform dynamics of the observed action potentials, a derivative based feature extraction and a spike sorting algorithm are proposed. These are compared with most commonly used methods of spike sorting under varying noise levels using realistic datasets to confirm their merits. The latter is implemented and demonstrated in real-time through an MCU based platform.Open Acces

NeuroBench:Advancing Neuromorphic Computing through Collaborative, Fair and Representative Benchmarking

The field of neuromorphic computing holds great promise in terms of advancing computing efficiency and capabilities by following brain-inspired principles. However, the rich diversity of techniques employed in neuromorphic research has resulted in a lack of clear standards for benchmarking, hindering effective evaluation of the advantages and strengths of neuromorphic methods compared to traditional deep-learning-based methods. This paper presents a collaborative effort, bringing together members from academia and the industry, to define benchmarks for neuromorphic computing: NeuroBench. The goals of NeuroBench are to be a collaborative, fair, and representative benchmark suite developed by the community, for the community. In this paper, we discuss the challenges associated with benchmarking neuromorphic solutions, and outline the key features of NeuroBench. We believe that NeuroBench will be a significant step towards defining standards that can unify the goals of neuromorphic computing and drive its technological progress. Please visit neurobench.ai for the latest updates on the benchmark tasks and metrics

Detection of interictal discharges with convolutional neural networks using discrete ordered multichannel intracranial EEG

Detection algorithms for electroencephalography (EEG) data, especially in the field of interictal epileptiform discharge (IED) detection, have traditionally employed handcrafted features which utilised specific characteristics of neural responses. Although these algorithms achieve high accuracy, mere detection of an IED holds little clinical significance. In this work, we consider deep learning for epileptic subjects to accommodate automatic feature generation from intracranial EEG data, while also providing clinical insight. Convolutional neural networks are trained in a subject independent fashion to demonstrate how meaningful features are automatically learned in a hierarchical process. We illustrate how the convolved filters in the deepest layers provide insight towards the different types of IEDs within the group, as confirmed by our expert clinicians. The morphology of the IEDs found in filters can help evaluate the treatment of a patient. To improve the learning of the deep model, moderately different score classes are utilised as opposed to binary IED and non-IED labels. The resulting model achieves state of the art classification performance and is also invariant to time differences between the IEDs. This study suggests that deep learning is suitable for automatic feature generation from intracranial EEG data, while also providing insight into the dat

Neuromorphic-Based Neuroprostheses for Brain Rewiring: State-of-the-Art and Perspectives in Neuroengineering.

Neuroprostheses are neuroengineering devices that have an interface with the nervous system and supplement or substitute functionality in people with disabilities. In the collective imagination, neuroprostheses are mostly used to restore sensory or motor capabilities, but in recent years, new devices directly acting at the brain level have been proposed. In order to design the next-generation of neuroprosthetic devices for brain repair, we foresee the increasing exploitation of closed-loop systems enabled with neuromorphic elements due to their intrinsic energy efficiency, their capability to perform real-time data processing, and of mimicking neurobiological computation for an improved synergy between the technological and biological counterparts. In this manuscript, after providing definitions of key concepts, we reviewed the first exploitation of a real-time hardware neuromorphic prosthesis to restore the bidirectional communication between two neuronal populations in vitro. Starting from that 'case-study', we provide perspectives on the technological improvements for real-time interfacing and processing of neural signals and their potential usage for novel in vitro and in vivo experimental designs. The development of innovative neuroprosthetics for translational purposes is also presented and discussed. In our understanding, the pursuit of neuromorphic-based closed-loop neuroprostheses may spur the development of novel powerful technologies, such as 'brain-prostheses', capable of rewiring and/or substituting the injured nervous system

Data-driven multivariate and multiscale methods for brain computer interface

This thesis focuses on the development of data-driven multivariate and multiscale methods for brain computer interface (BCI) systems. The electroencephalogram (EEG), the most convenient means to measure neurophysiological activity due to its noninvasive nature, is mainly considered. The nonlinearity and nonstationarity inherent in EEG and its multichannel recording nature require a new set of data-driven multivariate techniques to estimate more accurately features for enhanced BCI operation. Also, a long term goal is to enable an alternative EEG recording strategy for achieving long-term and portable monitoring. Empirical mode decomposition (EMD) and local mean decomposition (LMD), fully data-driven adaptive tools, are considered to decompose the nonlinear and nonstationary EEG signal into a set of components which are highly localised in time and frequency. It is shown that the complex and multivariate extensions of EMD, which can exploit common oscillatory modes within multivariate (multichannel) data, can be used to accurately estimate and compare the amplitude and phase information among multiple sources, a key for the feature extraction of BCI system. A complex extension of local mean decomposition is also introduced and its operation is illustrated on two channel neuronal spike streams. Common spatial pattern (CSP), a standard feature extraction technique for BCI application, is also extended to complex domain using the augmented complex statistics. Depending on the circularity/noncircularity of a complex signal, one of the complex CSP algorithms can be chosen to produce the best classification performance between two different EEG classes. Using these complex and multivariate algorithms, two cognitive brain studies are investigated for more natural and intuitive design of advanced BCI systems. Firstly, a Yarbus-style auditory selective attention experiment is introduced to measure the user attention to a sound source among a mixture of sound stimuli, which is aimed at improving the usefulness of hearing instruments such as hearing aid. Secondly, emotion experiments elicited by taste and taste recall are examined to determine the pleasure and displeasure of a food for the implementation of affective computing. The separation between two emotional responses is examined using real and complex-valued common spatial pattern methods. Finally, we introduce a novel approach to brain monitoring based on EEG recordings from within the ear canal, embedded on a custom made hearing aid earplug. The new platform promises the possibility of both short- and long-term continuous use for standard brain monitoring and interfacing applications

Development of an analysis pipeline for human microelectrode recordings in Parkinson’s disease

Thesis to obtain the master degree in Biomedical EngineeringBackground: Deep brain stimulation is a common treatment for advanced Parkinson’s Disease (PD). Intraoperative microelectrode recordings (MER) along preplanned trajectories are often used for accurate identification of subthalamic nucleus (STN), a common target for deep brain stimulation (DBS) in PD. However, this identification is performed manually and can be difficult in regions of transition. Misidentification may lead to suboptimal location of the DBS lead and inadequate clinical outcomes. Methods: A tool for unsupervised analysis and spike-sorting of human MER signals with feature extraction was developed. We also trained and tested a hybrid unsupervised/supervised machine learning approach that uses extracted MER time, frequency and noise properties for high-accuracy identification of STN. Lastly, we compared neurophysiological characteristics of different STN functional segments. Results: We obtained a classification accuracy of 96:28 3:15 % (30 trajectories, 5 patients) for individual STN-DBS surgery MER using an approach of "leave one subject out" validation with support vector machine classifier, all features based on time and frequency domain and human expert labels. The unsupervised sorting approach allowed us to sort a total of 357 STN neurons in 5 subjects. Dividing the STN in a dorsal, probably motor region, and a ventral, probably non-motor portion, we’ve found a higher burst rate (median (interquartile range) of 1.8 (1.5) vs 1.15 (0.05) bursts/s, p=0.001) and firing rate (median (interquartile range) of 21.4 (16.85) vs. 15.3 (14.33), p=0.013) of dorsal STN neurons among other features. Ongoing work will refine these results using anatomical gold standard through lead trajectory reconstruction, fused with an STN functional subdivision atlas. Conclusions: We’ve developed a tool for human MER analysis and extraction of related features, that provided good preliminary results in STN classification. In line with the literature, we were able to find preliminary activity differences in functionally segregated STN segments. This tool is fast and generalizable for other brain regions. Ongoing work using patient’s anatomy can further validate its’ usefulness in optimizing electrode placement and research purposes.A Estimulação Cerebral Profunda é um técnica utilizada para o tratamento dos sintomas motores da Doença de Parkinson com recurso a estimulação eléctrica intracraniana. Um dos alvos mais frequentemente utilizados para o tratamento é o núcleo subtalâmico e uma colocação precisa dos eléctrodos na região correcta é fulcral para o sucesso terapêutico e minimização de efeitos secundários. Esta cirurgia faz-se com recurso a estereotaxia e monitorização intra-operatória. Previamente à cirurgia, as coordenadas do alvo são estabelecidas e é definido o núcleo subtalâmico utilizando imagens pré-operatórias de ressonância magnética e de tomografia computadorizada do doente. Intra-operatoriamente são utilizados registos de microeléctrodos a diferentes profundidades em trajetórias pré-definidas para verificar as coordenadas estabelecidas e identificar -pela actividade eléctrica- a localização dos eléctrodos no cérebro. No entanto, a identificação do núcleo subtalâmico com registos de microeléctrodos é frequentemente realizada por inspeção visual, e pode ser difícil em regiões de transição. Erros na identificação podem levar a um posicionamento subóptimo dos elétrodos e a um mau resultado clínico. O correcto posicionamento do eléctrodo final de estimulação na porção sensoriomotora do núcleo subtalâmico está relacionado com os melhores resultados clínicos de acordo com a literatura. No entanto não é possível identificar claramente esta subdivisão com base a inspecção visual dos registos. O objetivo do presente estudo é o desenvolvimento de ferramentas para análise de registos de microeléctrodos em diferentes áreas do cérebro e identificação do núcleo subtalâmico, e da porção sensoriomotora, em doentes com Doença de Parkinson submetidos a estimulação cerebral profunda. Métodos Foram desenvolvidas ferramentas para análise do sinal, spike sorting e extração de características por processamento não supervisionado de registos de microeléctrodos humanos. As características relativas ao domínio do tempo e frequência foram obtidas para cada sinal através de medidas estatísticas (média, mediana e desvio padrão) dos registos fracionados em segmentos sobrepostos, para minimizar os efeitos do ruído. As características relacionadas com a atividade neuronal foram também extraídas após deteção e identificação dos diferentes neurónios em cada registo de forma não supervisionada. A localização dos diferentes registos ao longo do trajecto foi realizada por um especialista através de inspeção visual aleatória de todos os sinais. As características extraídas nas regiões do núcleo subtalâmico foram comparadas com as registadas fora de esta área. Posteriormente, uma abordagem híbrida de classificação utilizando métodos de aprendizagem automática -machine learning- foi treinada e testada para identificação dos registos localizados no núcleo subtalâmico utilizando as características extraídas baseadas no domínio do tempo e frequência. Para comparar as sub-regiões funcionais do núcleo subtalâmico, utilizaram-se os sinais identificados em esta região e dividiram-se estas profundidades etiquetadas pelo especialista como núcleo subtalâmico na porção mais dorsal (que terá maior probabilidade de ser motora) e outra ventral (que terá maior probabilidade de ser não motora). Utilizando esta subdivisão, compararamse as características neurofisiológicas relativas às diferentes sub-regiões funcionais com testes estatísticos. Conhecendo as limitações da classificação por inspecção visual, desenvolveu-se uma metodologia para refinamento das localizações dos diferentes registos baseada na reconstrução da trajetória dos eléctrodos implantados, utilizando imagens pre- e pos- operatórias. Esta reconstrução foi sobreposta a um atlas contendo as subdivisões funcionais do núcleo subtalâmico. Os resultados preliminares desta aplicação num sujeito, permitiram mostrar uma classificação mais realista destas sub-regiões na identificação da área motora vs. não motora (límbica e associativa). Resultados Foram identificadas várias características significativamente diferentes no domínio do tempo e frequência, entre os sinais classificados como pertencentes ao Núcleo Subtalámico e os sinais não-pertencentes. Estas diferenças permitiram o desenvolvimento de classificadores do tipo de máquinas de vectores de suporte, utilizando as localizações dos diferentes registos tendo como base a classificação baseada em inspecção visual por um especialista. Utilizando os sinais de maior certeza de classificação, obteve-se uma precisão de classificação do núcleo subtalâmico de 96,18 3,15 % (para 30 trajetórias, 5 pacientes) usando uma abordagem de validação cruzada de “deixar um sujeito fora"com máquina de vectores de suporte linear. Este modelo de validação garante que os dados relativos aos doentes utilizados para treinar o modelo não foram usados posteriormente para testar o mesmo. Utilizando todos os sinais considerados como núcleo subtalâmico independentemente do nível de certeza, obteve-se uma classificação de 84,32 5,75 %. A ferramenta não supervisionada para spike sorting permitiu a identificação de um total de 357 neurónios do núcleo subtalâmico identificado com a máxima certeza em 5 doentes (155 registos de microeléctrodos de 10 segundos de duração) e 521 neurónios foram extraídos de 420 sinais localizados fora do núcleo subtalâmico. Utilizando segmentos de 10 segundos livres de artefactos, identificamos 258 neurónios em 116 sinais do núcleo subtalâmico e 329 neurónios em 228 registos de não-Subtalámico. Foram encontradas diferenças significativas em em 13 das 15 características analisadas comparando STN com não-STN. As subregiões funcionais do núcleo subtalâmico (dorsal vs. ventral) foram comparadas quanto às caracteristicas de tempo, frequência e actividade neuronal. Encontraram-se diferenças significativas em 6 características extraídas dos sinais sem artefactos, consistentes com a literatura e também em uma característica de frequência. Em linha com o descrito previamente, encontramos um burts rate mais elevado na subdivisão dorsal em relação à ventral (mediana (amplitude interquartil) of 1.8(1.5) vs 1.15(0.05) bursts/s, p=0.001) e um firing rate mais alto (mediana (amplitude interquartil) of 21.4(16.85) vs. 15.3(14.33), p=0.013) entre outras características. Os resultados preliminares da análise de sinal refinado por imagem num sujeito, mostraram uma alta accuracy do expert na classificação de sinais STN/não-STN comparados com imagem (>85%). Apesar de neste sujeito não se obterem diferenças significativas entre regiões nas características extraídas, esta analise mostrou que divisão heurística dorsal/ventral era insuficiente para proceder à analise apropriada. Conclusões Foi desenvolvida uma ferramenta para análise de registo de microeléctrodos em humanos e para extração de características no domínio do tempo e frequência, que forneceu bons resultados na classificação do núcleo subtalâmico utilizando a identificação da localização de cada registo por um especialista. A análise e extração de características relativas à atividade neuronal foi realizada por uma ferramenta não supervisionada, que foi desenvolvida combinando algoritmos existentes e que pode ser generalizável a registos em outras regiões ou outro tipo de registos. Esta ferramenta permitiu a identificação de características que permitem diferenciar os sinais colhidos no núcleo subtalâmico dos identificados fora de esta região. De acordo com a literatura, fomos capazes de identificar diferenças em segmentos funcionalmente segregados do núcleo subtalâmico. O trabalho em curso, com refinação anatómica, vai-nos permitir avaliar a sua utilidade em optimizar o posicionamento dos elétrodos, podendo também ser utilizado para fins de investigação.N/

Neuromorphic Neuromodulation: Towards the next generation of on-device AI-revolution in electroceuticals

Neuromodulation techniques have emerged as promising approaches for treating a wide range of neurological disorders, precisely delivering electrical stimulation to modulate abnormal neuronal activity. While leveraging the unique capabilities of artificial intelligence (AI) holds immense potential for responsive neurostimulation, it appears as an extremely challenging proposition where real-time (low-latency) processing, low power consumption, and heat constraints are limiting factors. The use of sophisticated AI-driven models for personalized neurostimulation depends on back-telemetry of data to external systems (e.g. cloud-based medical mesosystems and ecosystems). While this can be a solution, integrating continuous learning within implantable neuromodulation devices for several applications, such as seizure prediction in epilepsy, is an open question. We believe neuromorphic architectures hold an outstanding potential to open new avenues for sophisticated on-chip analysis of neural signals and AI-driven personalized treatments. With more than three orders of magnitude reduction in the total data required for data processing and feature extraction, the high power- and memory-efficiency of neuromorphic computing to hardware-firmware co-design can be considered as the solution-in-the-making to resource-constraint implantable neuromodulation systems. This could lead to a new breed of closed-loop responsive and personalised feedback, which we describe as Neuromorphic Neuromodulation. This can empower precise and adaptive modulation strategies by integrating neuromorphic AI as tightly as possible to the site of the sensors and stimulators. This paper presents a perspective on the potential of Neuromorphic Neuromodulation, emphasizing its capacity to revolutionize implantable brain-machine microsystems and significantly improve patient-specificity.Comment: 17 page