75 research outputs found

    Measuring spectrally resolved information processing in neural data

    Get PDF
    Background: The human brain, an incredibly complex biological system comprising billions of neurons and trillions of synapses, possesses remarkable capabilities for information processing and distributed computations. Neurons, the fundamental building blocks, perform elementary operations on their inputs and collaborate extensively to execute intricate computations, giving rise to cognitive functions and behavior. Notably, distributed information processing in the brain heavily relies on rhythmic neural activity characterized by synchronized oscillations at specific frequencies. These oscillations play a crucial role in coordinating brain activity and facilitating communication between different neural circuits [1], effectively acting as temporal windows that enable efficient information exchange within specific frequency ranges. To understand distributed information processing in neural systems, breaking down its components, i.e., —information transfer, storage, and modification can be helpful, but requires precise mathematical definitions for each respective component. Thankfully, these definitions have recently become available [2]. Information theory is a natural choice for measuring information processing, as it offers a mathematically complete description of the concept of information and communication. The fundamental information-processing operations, are considered essential prerequisites for achieving universal information processing in any system [3]. By quantifying and analyzing these operations, we gain valuable insights into the brain’s complex computation and cognitive abilities. As information processing in the brain is intricately tied to rhythmic behavior, there is a need to establish a connection between information theoretic measures and frequency components. Previous attempts to achieve frequency-resolved information theoretic measures have mostly relied on narrowband filtering [4], which comes with several known issues of phase shifting and high false positive rate results [5], or simplifying the computation to few variables [6], that might result in missing important information in the analysed brain signals. Therefore, the current work aims to establish a frequency-resolved measure of two crucial components of information processing: information transfer and information storage. By proposing methodological advancements, this research seeks to shed light on the role of neural oscillations in information processing within the brain. Furthermore, a more comprehensive investigation was carried out on the communication between two critical brain regions responsible for motor inhibition in the frontal cortex (right Inferior Frontal gyrus (rIFG) and pre-Supplementary motor cortex (pre-SMA)). Here, neural oscillations in the beta band (12 − 30 Hz) have been proposed to have a pivotal role in response inhibition. A long-standing question in the field was to disentangle which of these two brain areas first signals the stopping process and drives the other [7]. Furthermore, it was hypothesized that beta oscillations carry the information transfer between these regions. The present work addresses the methodological problems and investigates spectral information processing in neural data, in three studies. Study 1 focuses on the critical role of information transfer, measured by transfer entropy, in distributed computation. Understanding the patterns of information transfer is essential for unraveling the computational algorithms in complex systems, such as the brain. As many natural systems rely on rhythmic processes for distributed computations, a frequency-resolved measure of information transfer becomes highly valuable. To address this, a novel algorithm is presented, efficiently identifying frequencies responsible for sending and receiving information in a network. The approach utilizes the invertible maximum overlap discrete wavelet transform (MODWT) to create surrogate data for computing transfer entropy, eliminating issues associated with phase shifts and filtering. However, measuring frequency-resolved information transfer poses a Partial information decomposition problem [8] that is yet to be fully resolved. The algorithm’s performance is validated using simulated data and applied to human magnetoencephalography (MEG) and ferret local field potential recordings (LFP). In human MEG, the study unveils a complex spectral configuration of cortical information transmission, showing top-down information flow from very high frequencies (above 100Hz) to both similarly high frequencies and frequencies around 20Hz in the temporal cortex. Contrary to the current assumption, the findings suggest that low frequencies do not solely send information to high frequencies. In the ferret LFP, the prefrontal cortex demonstrates the transmission of information at low frequencies, specifically within the range of 4-8 Hz. On the receiving end, V1 exhibits a preference for operating at very high frequency > 125 Hz. The spectrally resolved transfer entropy promises to deepen our understanding of rhythmic information exchange in natural systems, shedding light on the computational properties of oscillations on cognitive functions. In study 2, the primary focus lay on the second fundamental aspect of information processing: the active information storage (AIS). The AIS estimates how much information in the next measurements of the process can be predicted by examining its paste state. In processes that either produce little information (low entropy) or that are highly unpredictable, the AIS is low, whereas processes that are predictable but visit many different states with equal probabilities, exhibit high AIS [9]. Within this context, we introduced a novel spectrally-resolved AIS. Utilizing intracortical recordings of neural activity in anesthetized ferrets before and after loss of consciousness (LOC), the study reveals that the modulation of AIS by anesthesia is highly specific to different frequency bands, cortical layers, and brain regions. The findings reveal that the effects of anesthesia on AIS are prominent in the supragranular layers for the high/low gamma band, while the alpha/beta band exhibits the strongest decrease in AIS at infragranular layers, in accordance with the predictive coding theory. Additionally, the isoflurane impacts local information processing in a frequency-specific manner. For instance, increases in isoflurane concentration lead to a decrease in AIS in the alpha frequency but to an increase in AIS in the delta frequency range (<2Hz). In sum, analyzing spectrally-resolved AIS provides valuable insights into changes in cortical information processing under anesthesia. With rhythmic neural activity playing a significant role in biological neural systems, the introduction of frequency-specific components in active information storage allows a deeper understanding of local information processing in different brain areas and under various conditions. In study 3, to further verify the pivotal role of neural oscillations in information processing, we investigated the neural network mechanisms underlying response inhibition. A long-standing debate has centered around identifying the cortical initiator of response inhibition in the beta band, with two main regions proposed: the right rIFG and the pre-SMA. This third study aimed to determine which of these regions is activated first and exerts a potential information exchange on the other. Using high temporal resolution magnetoencephalography (MEG) and a relatively large cohort of subjects. A significant breakthrough is achieved by demonstrating that the rIFG is activated significantly earlier than the pre-SMA. The onset of beta band activity in the rIFG occurred at around 140 ms after the STOP signal. Further analyses showed that the beta-band activity in the rIFG was crucial for successful stopping, as evidenced by its predictive value for stopping performance. Connectivity analysis revealed that the rIFG sends information in the beta band to the pre-SMA but not vice versa, emphasizing the rIFG’s dominance in the response inhibition process. The results provide strong support for the hypothesis that the rIFG initiates stopping and utilizes beta-band oscillations for this purpose. These findings have significant implications, suggesting the possibility of spatially localized oscillation based interventions for response inhibition. Conclusion: In conclusion, the present work proposes a novel algorithm for uncovering the frequencies at which information is transferred between sources and targets in the brain, providing valuable insights into the computational dynamics of neural processes. The spectrally resolved transfer entropy was successfully applied to experimental neural data of intracranial recordings in ferrets and MEG recordings of humans. Furthermore, the study on active information storage (AIS) analysis under anesthesia revealed that the spectrally resolved AIS offers unique additional insights beyond traditional spectral power analysis. By examining changes in neural information processing, the study demonstrates how AIS analysis can deepen the understanding of anesthesia’s effects on cortical information processing. Moreover, the third study’s findings provide strong evidence supporting the critical role of beta oscillations in information processing, particularly in response inhibition. The research successfully demonstrates that beta oscillations in the rIFG functions as the key initiator of the response inhibition process, acting as a top-down control mechanism. The identification of beta oscillations as a crucial factor in information processing opens possibilities for further research and targeted interventions in neurological disorders. Taken together, the current work highlights the role of spectrally-resolved information processing in neural systems by not only introducing novel algorithms, but also successfully applying them to experimental oscillatory neural activity in relation to low-level cortical information processing (anesthesia) as well as high-level processes (cognitive response inhibition)

    A Physics-Informed, Deep Double Reservoir Network for Forecasting Boundary Layer Velocity

    Full text link
    When a fluid flows over a solid surface, it creates a thin boundary layer where the flow velocity is influenced by the surface through viscosity, and can transition from laminar to turbulent at sufficiently high speeds. Understanding and forecasting the wind dynamics under these conditions is one of the most challenging scientific problems in fluid dynamics. It is therefore of high interest to formulate models able to capture the nonlinear spatio-temporal velocity structure as well as produce forecasts in a computationally efficient manner. Traditional statistical approaches are limited in their ability to produce timely forecasts of complex, nonlinear spatio-temporal structures which are at the same time able to incorporate the underlying flow physics. In this work, we propose a model to accurately forecast boundary layer velocities with a deep double reservoir computing network which is capable of capturing the complex, nonlinear dynamics of the boundary layer while at the same time incorporating physical constraints via a penalty obtained by a Partial Differential Equation (PDE). Simulation studies on a one-dimensional viscous fluid demonstrate how the proposed model is able to produce accurate forecasts while simultaneously accounting for energy loss. The application focuses on boundary layer data on a wind tunnel with a PDE penalty derived from an appropriate simplification of the Navier-Stokes equations, showing forecasts more compliant with mass conservation

    Direct Learning-Based Deep Spiking Neural Networks: A Review

    Full text link
    The spiking neural network (SNN), as a promising brain-inspired computational model with binary spike information transmission mechanism, rich spatially-temporal dynamics, and event-driven characteristics, has received extensive attention. However, its intricately discontinuous spike mechanism brings difficulty to the optimization of the deep SNN. Since the surrogate gradient method can greatly mitigate the optimization difficulty and shows great potential in directly training deep SNNs, a variety of direct learning-based deep SNN works have been proposed and achieved satisfying progress in recent years. In this paper, we present a comprehensive survey of these direct learning-based deep SNN works, mainly categorized into accuracy improvement methods, efficiency improvement methods, and temporal dynamics utilization methods. In addition, we also divide these categorizations into finer granularities further to better organize and introduce them. Finally, the challenges and trends that may be faced in future research are prospected.Comment: Accepted by Frontiers in Neuroscienc

    Spiking PointNet: Spiking Neural Networks for Point Clouds

    Full text link
    Recently, Spiking Neural Networks (SNNs), enjoying extreme energy efficiency, have drawn much research attention on 2D visual recognition and shown gradually increasing application potential. However, it still remains underexplored whether SNNs can be generalized to 3D recognition. To this end, we present Spiking PointNet in the paper, the first spiking neural model for efficient deep learning on point clouds. We discover that the two huge obstacles limiting the application of SNNs in point clouds are: the intrinsic optimization obstacle of SNNs that impedes the training of a big spiking model with large time steps, and the expensive memory and computation cost of PointNet that makes training a big spiking point model unrealistic. To solve the problems simultaneously, we present a trained-less but learning-more paradigm for Spiking PointNet with theoretical justifications and in-depth experimental analysis. In specific, our Spiking PointNet is trained with only a single time step but can obtain better performance with multiple time steps inference, compared to the one trained directly with multiple time steps. We conduct various experiments on ModelNet10, ModelNet40 to demonstrate the effectiveness of Spiking PointNet. Notably, our Spiking PointNet even can outperform its ANN counterpart, which is rare in the SNN field thus providing a potential research direction for the following work. Moreover, Spiking PointNet shows impressive speedup and storage saving in the training phase.Comment: Accepted by NeurIP

    Understanding Quantum Technologies 2022

    Full text link
    Understanding Quantum Technologies 2022 is a creative-commons ebook that provides a unique 360 degrees overview of quantum technologies from science and technology to geopolitical and societal issues. It covers quantum physics history, quantum physics 101, gate-based quantum computing, quantum computing engineering (including quantum error corrections and quantum computing energetics), quantum computing hardware (all qubit types, including quantum annealing and quantum simulation paradigms, history, science, research, implementation and vendors), quantum enabling technologies (cryogenics, control electronics, photonics, components fabs, raw materials), quantum computing algorithms, software development tools and use cases, unconventional computing (potential alternatives to quantum and classical computing), quantum telecommunications and cryptography, quantum sensing, quantum technologies around the world, quantum technologies societal impact and even quantum fake sciences. The main audience are computer science engineers, developers and IT specialists as well as quantum scientists and students who want to acquire a global view of how quantum technologies work, and particularly quantum computing. This version is an extensive update to the 2021 edition published in October 2021.Comment: 1132 pages, 920 figures, Letter forma

    Apprentissage automatique pour le codage cognitif de la parole

    Get PDF
    Depuis les années 80, les codecs vocaux reposent sur des stratégies de codage à court terme qui fonctionnent au niveau de la sous-trame ou de la trame (généralement 5 à 20 ms). Les chercheurs ont essentiellement ajusté et combiné un nombre limité de technologies disponibles (transformation, prédiction linéaire, quantification) et de stratégies (suivi de forme d'onde, mise en forme du bruit) pour construire des architectures de codage de plus en plus complexes. Dans cette thèse, plutôt que de s'appuyer sur des stratégies de codage à court terme, nous développons un cadre alternatif pour la compression de la parole en codant les attributs de la parole qui sont des caractéristiques perceptuellement importantes des signaux vocaux. Afin d'atteindre cet objectif, nous résolvons trois problèmes de complexité croissante, à savoir la classification, la prédiction et l'apprentissage des représentations. La classification est un élément courant dans les conceptions de codecs modernes. Dans un premier temps, nous concevons un classifieur pour identifier les émotions, qui sont parmi les attributs à long terme les plus complexes de la parole. Dans une deuxième étape, nous concevons un prédicteur d'échantillon de parole, qui est un autre élément commun dans les conceptions de codecs modernes, pour mettre en évidence les avantages du traitement du signal de parole à long terme et non linéaire. Ensuite, nous explorons les variables latentes, un espace de représentations de la parole, pour coder les attributs de la parole à court et à long terme. Enfin, nous proposons un réseau décodeur pour synthétiser les signaux de parole à partir de ces représentations, ce qui constitue notre dernière étape vers la construction d'une méthode complète de compression de la parole basée sur l'apprentissage automatique de bout en bout. Bien que chaque étape de développement proposée dans cette thèse puisse faire partie d'un codec à elle seule, chaque étape fournit également des informations et une base pour la prochaine étape de développement jusqu'à ce qu'un codec entièrement basé sur l'apprentissage automatique soit atteint. Les deux premières étapes, la classification et la prédiction, fournissent de nouveaux outils qui pourraient remplacer et améliorer des éléments des codecs existants. Dans la première étape, nous utilisons une combinaison de modèle source-filtre et de machine à état liquide (LSM), pour démontrer que les caractéristiques liées aux émotions peuvent être facilement extraites et classées à l'aide d'un simple classificateur. Dans la deuxième étape, un seul réseau de bout en bout utilisant une longue mémoire à court terme (LSTM) est utilisé pour produire des trames vocales avec une qualité subjective élevée pour les applications de masquage de perte de paquets (PLC). Dans les dernières étapes, nous nous appuyons sur les résultats des étapes précédentes pour concevoir un codec entièrement basé sur l'apprentissage automatique. un réseau d'encodage, formulé à l'aide d'un réseau neuronal profond (DNN) et entraîné sur plusieurs bases de données publiques, extrait et encode les représentations de la parole en utilisant la prédiction dans un espace latent. Une approche d'apprentissage non supervisé basée sur plusieurs principes de cognition est proposée pour extraire des représentations à partir de trames de parole courtes et longues en utilisant l'information mutuelle et la perte contrastive. La capacité de ces représentations apprises à capturer divers attributs de la parole à court et à long terme est démontrée. Enfin, une structure de décodage est proposée pour synthétiser des signaux de parole à partir de ces représentations. L'entraînement contradictoire est utilisé comme une approximation des mesures subjectives de la qualité de la parole afin de synthétiser des échantillons de parole à consonance naturelle. La haute qualité perceptuelle de la parole synthétisée ainsi obtenue prouve que les représentations extraites sont efficaces pour préserver toutes sortes d'attributs de la parole et donc qu'une méthode de compression complète est démontrée avec l'approche proposée.Abstract: Since the 80s, speech codecs have relied on short-term coding strategies that operate at the subframe or frame level (typically 5 to 20ms). Researchers essentially adjusted and combined a limited number of available technologies (transform, linear prediction, quantization) and strategies (waveform matching, noise shaping) to build increasingly complex coding architectures. In this thesis, rather than relying on short-term coding strategies, we develop an alternative framework for speech compression by encoding speech attributes that are perceptually important characteristics of speech signals. In order to achieve this objective, we solve three problems of increasing complexity, namely classification, prediction and representation learning. Classification is a common element in modern codec designs. In a first step, we design a classifier to identify emotions, which are among the most complex long-term speech attributes. In a second step, we design a speech sample predictor, which is another common element in modern codec designs, to highlight the benefits of long-term and non-linear speech signal processing. Then, we explore latent variables, a space of speech representations, to encode both short-term and long-term speech attributes. Lastly, we propose a decoder network to synthesize speech signals from these representations, which constitutes our final step towards building a complete, end-to-end machine-learning based speech compression method. The first two steps, classification and prediction, provide new tools that could replace and improve elements of existing codecs. In the first step, we use a combination of source-filter model and liquid state machine (LSM), to demonstrate that features related to emotions can be easily extracted and classified using a simple classifier. In the second step, a single end-to-end network using long short-term memory (LSTM) is shown to produce speech frames with high subjective quality for packet loss concealment (PLC) applications. In the last steps, we build upon the results of previous steps to design a fully machine learning-based codec. An encoder network, formulated using a deep neural network (DNN) and trained on multiple public databases, extracts and encodes speech representations using prediction in a latent space. An unsupervised learning approach based on several principles of cognition is proposed to extract representations from both short and long frames of data using mutual information and contrastive loss. The ability of these learned representations to capture various short- and long-term speech attributes is demonstrated. Finally, a decoder structure is proposed to synthesize speech signals from these representations. Adversarial training is used as an approximation to subjective speech quality measures in order to synthesize natural-sounding speech samples. The high perceptual quality of synthesized speech thus achieved proves that the extracted representations are efficient at preserving all sorts of speech attributes and therefore that a complete compression method is demonstrated with the proposed approach

    Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks

    Get PDF
    Inspired by detailed modelling of biological neurons, spiking neural networks (SNNs) are investigated as biologically plausible and high-performance models of neural computation. The sparse and binary communication between spiking neurons potentially enables powerful and energy-efficient neural networks. The performance of SNNs, however, has remained lacking compared with artificial neural networks. Here we demonstrate how an activity-regularizing surrogate gradient combined with recurrent networks of tunable and adaptive spiking neurons yields the state of the art for SNNs on challenging benchmarks in the time domain, such as speech and gesture recognition. This also exceeds the performance of standard classical recurrent neural networks and approaches that of the best modern artificial neural networks. As these SNNs exhibit sparse spiking, we show that they are theoretically one to three orders of magnitude more computationally efficient compared to recurrent neural networks with similar performance. Together, this positions SNNs as an attractive solution for AI hardware implementations

    The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks

    Get PDF
    Brains process information in spiking neural networks. Their intricate connections shape the diverse functions these networks perform. In comparison, the functional capabilities of models of spiking networks are still rudimentary. This shortcoming is mainly due to the lack of insight and practical algorithms to construct the necessary connectivity. Any such algorithm typically attempts to build networks by iteratively reducing the error compared to a desired output. But assigning credit to hidden units in multi-layered spiking networks has remained challenging due to the non-differentiable nonlinearity of spikes. To avoid this issue, one can employ surrogate gradients to discover the required connectivity in spiking network models. However, the choice of a surrogate is not unique, raising the question of how its implementation influences the effectiveness of the method. Here, we use numerical simulations to systematically study how essential design parameters of surrogate gradients impact learning performance on a range of classification problems. We show that surrogate gradient learning is robust to different shapes of underlying surrogate derivatives, but the choice of the derivative’s scale can substantially affect learning performance. When we combine surrogate gradients with a suitable activity regularization technique, robust information processing can be achieved in spiking networks even at the sparse activity limit. Our study provides a systematic account of the remarkable robustness of surrogate gradient learning and serves as a practical guide to model functional spiking neural networks

    Phase imaging for reducing macrovascular signal contributions in high-resolution fMRI

    Get PDF
    High resolution functional MRI allows for the investigation of neural activity within the cortical sheet. One consideration in high resolution fMRI is the choice of which sequence to use during imaging, as all methods come with sensitivity and specificity tradeoffs. The most used fMRI sequence is gradient-echo echo planar imaging (GE-EPI) which has the highest sensitivity but is not specific to microvasculature. GE-EPI results in a signal with pial vessel bias which increases complexity of performing studies targeted at structures within the cortex. This work seeks to explore the use of MRI phase signal as a macrovascular filter to correct this bias. First, an in-house phase combination method was designed and tested on the 7T MRI system. This method, the fitted SVD method, uses a low-resolution singular value decomposition and fitting to a polynomial basis to provide computationally efficient, phase sensitive, coil combination that is insensitive to motion. Second, a direct comparison of GE-EPI, GE-EPI with phase regression (GE-EPI-PR), and spin echo EPI (SE-EPI) was performed in humans completing a visual task. The GE-EPI-PR activation showed higher spatial similarity with SE-EPI than GE-EPI across the cortical surface. GE-EPI-PR produced a similar laminar profile to SE-EPI while maintaining a higher contrast-to-noise ratio across layers, making it a useful method in low SNR studies such as high-resolution fMRI. The final study extended this work to a resting state macaque experiment. Macaques are a common model for laminar fMRI as they allow for simultaneous imaging and electrophysiology. We hypothesized that phase regression could improve spatial specificity of the resting state data. Further analysis showed the phase data contained both system and respiratory artifacts which prevented the technique performing as expected under two physiological cleaning strategies. Future work will have to examine on-scanner physiology correction to obtain a phase timeseries without artifacts to allow for the phase regression technique to be used in macaques. This work demonstrates that phase regression reduces signal contributions from pial vessels and will improve specificity in human layer fMRI studies. This method can be completed easily with complex fMRI data which can be created using our fitted SVD method
    • …
    corecore