CORTICAL DYNAMICS OF AUDITORY-VISUAL SPEECH: A FORWARD MODEL OF MULTISENSORY INTEGRATION.

In noisy settings, seeing the interlocutor's face helps to disambiguate what is being said. For this to happen, the brain must integrate auditory and visual information. Three major problems are (1) bringing together separate sensory streams of information, (2) extracting auditory and visual speech information, and (3) identifying this information as a unified auditory-visual percept. In this dissertation, a new representational framework for auditory visual (AV) speech integration is offered. The experimental work (psychophysics and electrophysiology (EEG)) suggests specific neural mechanisms for solving problems (1), (2), and (3) that are consistent with a (forward) 'analysis-by-synthesis' view of AV speech integration. In Chapter I, multisensory perception and integration are reviewed. A unified conceptual framework serves as background for the study of AV speech integration. In Chapter II, psychophysics testing the perception of desynchronized AV speech inputs show the existence of a ~250ms temporal window of integration in AV speech integration. In Chapter III, an EEG study shows that visual speech modulates early on the neural processing of auditory speech. Two functionally independent modulations are (i) a ~250ms amplitude reduction of auditory evoked potentials (AEPs) and (ii) a systematic temporal facilitation of the same AEPs as a function of the saliency of visual speech. In Chapter IV, an EEG study of desynchronized AV speech inputs shows that (i) fine-grained (gamma, ~25ms) and (ii) coarse-grained (theta, ~250ms) neural mechanisms simultaneously mediate the processing of AV speech. In Chapter V, a new illusory effect is proposed, where non-speech visual signals modify the perceptual quality of auditory objects. EEG results show very different patterns of activation as compared to those observed in AV speech integration. An MEG experiment is subsequently proposed to test hypotheses on the origins of these differences. In Chapter VI, the 'analysis-by-synthesis' model of AV speech integration is contrasted with major speech theories. From a Cognitive Neuroscience perspective, the 'analysis-by-synthesis' model is argued to offer the most sensible representational system for AV speech integration. This thesis shows that AV speech integration results from both the statistical nature of stimulation and the inherent predictive capabilities of the nervous system

Almost complete and equable heteroclinic networks

Heteroclinic connections are trajectories that link invariant sets for an autonomous dynamical flow: these connections can robustly form networks between equilibria, for systems with flow-invariant spaces. In this paper we examine the relation between the heteroclinic network as a flow-invariant set and directed graphs of possible connections between nodes. We consider realizations of a large class of transitive digraphs as robust heteroclinic networks and show that although robust realizations are typically not complete (i.e. not all unstable manifolds of nodes are part of the network), they can be almost complete (i.e. complete up to a set of zero measure within the unstable manifold) and equable (i.e. all sets of connections from a node have the same dimension). We show there are almost complete and equable realizations that can be closed by adding a number of extra nodes and connections. We discuss some examples and describe a sense in which an equable almost complete network embedding is an optimal description of stochastically perturbed motion on the network

Brain Rhythms in Object Recognition and Manipulation

Our manual interactions with objects represent the most fundamental activity in our everyday life. Whereas the grasp of an object is driven by the perceptual senses, using an object for its function relies on learnt experience to retrieve. Recent theories explain how the brain takes decisions based on perceptual information, yet the question of how does it retrieve object knowledge to use tools remains unanswered. Discovering the neuronal implementation of the retrieval of object knowledge would help understanding praxic impairments and provide appropriate neurorehabilitation. This thesis reports five investigations on the neuronal oscillatory activity involved in accessing object knowledge. Employing an original paradigm combining EEG recordings with tool use training in virtual reality, I demonstrated that beta oscillations are crucial to the retrieval of object knowledge during object recognition. Multiple evidence points toward an access to object knowledge during the 300 to 400 ms of visual processing. The different topographies of the beta oscillations suggest that tool knowledge is encoded in distinct brain areas but generally located within the left hemisphere. Importantly, learning action information about an object has consequences on its manipulations. Multiplying tool use knowledge about an object increases the beta desynchronization and slows down motor control. Furthermore, the present data report an influence of language on object manipulations and beta oscillations, in a way that learning the name of an object speeds up its use while impedes its grasp. This shred of evidence led to the formulation of three testable hypotheses extending contemporary theories of object manipulation and semantic memory. First, the preparation of object transportation or use could be distinguished by the synchronization/desynchronization patterns of mu and beta rhythms. Second, action competitions originate from both perceptuo-motor and memory systems. Third, accessing to semantic object knowledge during object processing could be indexed by the bursts of desynchronization of high-beta oscillations in the brain.MSCA-ETN SECURE [642667

Survey on Lightweight Primitives and Protocols for RFID in Wireless Sensor Networks

The use of radio frequency identification (RFID) technologies is becoming widespread in all kind of wireless network-based applications. As expected, applications based on sensor networks, ad-hoc or mobile ad hoc networks (MANETs) can be highly benefited from the adoption of RFID solutions. There is a strong need to employ lightweight cryptographic primitives for many security applications because of the tight cost and constrained resource requirement of sensor based networks. This paper mainly focuses on the security analysis of lightweight protocols and algorithms proposed for the security of RFID systems. A large number of research solutions have been proposed to implement lightweight cryptographic primitives and protocols in sensor and RFID integration based resource constraint networks. In this work, an overview of the currently discussed lightweight primitives and their attributes has been done. These primitives and protocols have been compared based on gate equivalents (GEs), power, technology, strengths, weaknesses and attacks. Further, an integration of primitives and protocols is compared with the possibilities of their applications in practical scenarios

Measuring High-Order Interactions in Rhythmic Processes through Multivariate Spectral Information Decomposition

Many complex systems in physics, biology and engineering are modeled as dynamical networks and described using multivariate time series analysis. Recent developments have shown that the emergent dynamics of a network system are significantly affected by interactions involving multiple network nodes which cannot be described using pairwise links. While these higher-order interactions can be probed using information-theoretic measures, a rigorous framework to describe them in the frequency domain is still lacking. This work presents an approach for the spectral decomposition of multivariate information measures, capable of identifying higher-order synergistic and redundant interactions between oscillatory processes. We show theoretically that synergy and redundancy can coexist at different frequencies among the output signals of a network system and can be detected only using the proposed spectral method. To demonstrate the broad applicability of the framework, we provide parametric and non-parametric data-efficient estimators for the spectral information measures, and employ them to describe multivariate interactions in three complex systems producing rich oscillatory dynamics, namely the human brain, a ring of electronic oscillators, and the global climate system. In these systems, we show that the use of our framework for the spectral decomposition of information measures reveals multivariate and higher-order interactions not detectable in the time domain. Our results are exemplary of how the frequency-specific analysis of multivariate dynamics can aid the implementation of assessment and control strategies in realworld network systems

Network-based brain computer interfaces: principles and applications

Brain-computer interfaces (BCIs) make possible to interact with the external environment by decoding the mental intention of individuals. BCIs can therefore be used to address basic neuroscience questions but also to unlock a variety of applications from exoskeleton control to neurofeedback (NFB) rehabilitation. In general, BCI usability critically depends on the ability to comprehensively characterize brain functioning and correctly identify the user s mental state. To this end, much of the efforts have focused on improving the classification algorithms taking into account localized brain activities as input features. Despite considerable improvement BCI performance is still unstable and, as a matter of fact, current features represent oversimplified descriptors of brain functioning. In the last decade, growing evidence has shown that the brain works as a networked system composed of multiple specialized and spatially distributed areas that dynamically integrate information. While more complex, looking at how remote brain regions functionally interact represents a grounded alternative to better describe brain functioning. Thanks to recent advances in network science, i.e. a modern field that draws on graph theory, statistical mechanics, data mining and inferential modelling, scientists have now powerful means to characterize complex brain networks derived from neuroimaging data. Notably, summary features can be extracted from these networks to quantitatively measure specific organizational properties across a variety of topological scales. In this topical review, we aim to provide the state-of-the-art supporting the development of a network theoretic approach as a promising tool for understanding BCIs and improve usability

On the functions, mechanisms, and malfunctions of intracortical contextual modulation

A broad neuron-centric conception of contextual modulation is reviewed and re-assessed in the light of recent neurobiological studies of amplification, suppression, and synchronization. Behavioural and computational studies of perceptual and higher cognitive functions that depend on these processes are outlined, and evidence that those functions and their neuronal mechanisms are impaired in schizophrenia is summarized. Finally, we compare and assess the long-term biological functions of contextual modulation at the level of computational theory as formalized by the theories of coherent infomax and free energy reduction. We conclude that those theories, together with the many empirical findings reviewed, show how contextual modulation at the neuronal level enables the cortex to flexibly adapt the use of its knowledge to current circumstances by amplifying and grouping relevant activities and by suppressing irrelevant activities

Synchrony and bifurcations in coupled dynamical systems and effects of time delay

Dynamik auf Netzwerken ist ein mathematisches Feld, das in den letzten Jahrzehnten schnell gewachsen ist und Anwendungen in zahlreichen Disziplinen wie z.B. Physik, Biologie und Soziologie findet. Die Funktion vieler Netzwerke hängt von der Fähigkeit ab, die Elemente des Netzwerkes zu synchronisieren. Mit anderen Worten, die Existenz und die transversale Stabilität der synchronen Mannigfaltigkeit sind zentrale Eigenschaften. Erst seit einigen Jahren wird versucht, den verwickelten Zusammenhang zwischen der Kopplungsstruktur und den Stabilitätseigenschaften synchroner Zustände zu verstehen. Genau das ist das zentrale Thema dieser Arbeit. Zunächst präsentiere ich erste Ergebnisse zur Klassifizierung der Kanten eines gerichteten Netzwerks bezüglich ihrer Bedeutung für die Stabilität des synchronen Zustands. Folgend untersuche ich ein komplexes Verzweigungsszenario in einem gerichteten Ring von Stuart-Landau Oszillatoren und zeige, dass das Szenario persistent ist, wenn dem Netzwerk eine schwach gewichtete Kante hinzugefügt wird. Daraufhin untersuche ich synchrone Zustände in Ringen von Phasenoszillatoren die mit Zeitverzögerung gekoppelt sind. Ich bespreche die Koexistenz synchroner Lösungen und analysiere deren Stabilität und Verzweigungen. Weiter zeige ich, dass eine Zeitverschiebung genutzt werden kann, um Muster im Ring zu speichern und wiederzuerkennen. Diese Zeitverschiebung untersuche ich daraufhin für beliebige Kopplungsstrukturen. Ich zeige, dass invariante Mannigfaltigkeiten des Flusses sowie ihre Stabilität unter der Zeitverschiebung erhalten bleiben. Darüber hinaus bestimme ich die minimale Anzahl von Zeitverzögerungen, die gebraucht werden, um das System äquivalent zu beschreiben. Schließlich untersuche ich das auffällige Phänomen eines nichtstetigen Übergangs zu Synchronizität in Klassen großer Zufallsnetzwerke indem ich einen kürzlich eingeführten Zugang zur Beschreibung großer Zufallsnetzwerke auf den Fall zeitverzögerter Kopplungen verallgemeinere.Since a couple of decades, dynamics on networks is a rapidly growing branch of mathematics with applications in various disciplines such as physics, biology or sociology. The functioning of many networks heavily relies on the ability to synchronize the network’s nodes. More precisely, the existence and the transverse stability of the synchronous manifold are essential properties. It was only in the last few years that people tried to understand the entangled relation between the coupling structure of a network, given by a (di-)graph, and the stability properties of synchronous states. This is the central theme of this dissertation. I first present results towards a classification of the links in a directed, diffusive network according to their impact on the stability of synchronization. Then I investigate a complex bifurcation scenario observed in a directed ring of Stuart-Landau oscillators. I show that under the addition of a single weak link, this scenario is persistent. Subsequently, I investigate synchronous patterns in a directed ring of phase oscillators coupled with time delay. I discuss the coexistence of multiple of synchronous solutions and investigate their stability and bifurcations. I apply these results by showing that a certain time-shift transformation can be used in order to employ the ring as a pattern recognition device. Next, I investigate the same time-shift transformation for arbitrary coupling structures in a very general setting. I show that invariant manifolds of the flow together with their stability properties are conserved under the time-shift transformation. Furthermore, I determine the minimal number of delays needed to equivalently describe the system’s dynamics. Finally, I investigate a peculiar phenomenon of non-continuous transition to synchrony observed in certain classes of large random networks, generalizing a recently introduced approach for the description of large random networks to the case of delayed couplings

Concepts, Frames and Cascades in Semantics, Cognition and Ontology

This open access book presents novel theoretical, empirical and experimental work exploring the nature of mental representations that support natural language production and understanding, and other manifestations of cognition. One fundamental question raised in the text is whether requisite knowledge structures can be adequately modeled by means of a uniform representational format, and if so, what exactly is its nature. Frames are a key topic covered which have had a strong impact on the exploration of knowledge representations in artificial intelligence, psychology and linguistics; cascades are a novel development in frame theory. Other key subject areas explored are: concepts and categorization, the experimental investigation of mental representation, as well as cognitive analysis in semantics. This book is of interest to students, researchers, and professionals working on cognition in the fields of linguistics, philosophy, and psychology