The Anatomy of an adaptive multimedia presentation system

The use of multimedia presentations within learning environments is described and guidelines for the design of good E-Learning systems are identified. It is argued that a linear sequential presentation of knowledge segments is effective, but that the user is provided with optional links to relevant segments during the presentation. The synchronisation of multiple media is considered and the design of a prototype E-Learning system is discussed. The segmentation of material is then discussed and how the information can be stored in a data repository consider with respect to the requirement of accessing linked segments. Finally, the nature of adaptivity is discussed leading to a discussion of the salient parts of an adaptive multimedia presentation system

Oscillatory architecture of memory circuits

The coordinated activity between remote brain regions underlies cognition and memory function. Although neuronal oscillations have been proposed as a mechanistic substrate for the coordination of information transfer and memory consolidation during sleep, little is known about the mechanisms that support the widespread synchronization of brain regions and the relationship of neuronal dynamics with other bodily rhythms, such as breathing. During exploratory behavior, the hippocampus and the prefrontal cortex are organized by theta oscillations, known to support memory encoding and retrieval, while during sleep the same structures are dominated by slow oscillations that are believed to underlie the consolidation of recent experiences. The expression of conditioned fear and extinction memories relies on the coordinated activity between the mPFC and the basolateral amygdala (BLA), a neuronal structure encoding associative fear memories. However, to date, the mechanisms allowing this long-range network synchronization of neuronal activity between the mPFC and BLA during fear behavior remain virtually unknown. Using a combination of extracellular recordings and open- and closed-loop optogenetic manipulations, we investigated the oscillatory and coding mechanisms mediating the organization and coupling of the limbic circuit in the awake and asleep brain, as well as during memory encoding and retrieval. We found that freezing, a behavioral expression of fear, is tightly associated with an internally generated brain state that manifests in sustained 4Hz oscillatory dynamics in prefrontal-amygdala circuits. 4Hz oscillations accurately predict the onset and termination of the freezing state. These oscillations synchronize prefrontal-amygdala circuits and entrain neuronal activity to dynamically regulate the development of neuronal ensembles. This enables the precise timing of information transfer between the two structures and the expression of fear responses. Optogenetic induction of prefrontal 4Hz oscillations promotes freezing behavior and the formation of long-lasting fear memory, while closed-loop phase specific manipulations bidirectionally modulate fear expression. Our results unravel a physiological signature of fear memory and identify a novel internally generated brain state, characterized by 4Hz oscillations. This oscillation enables the temporal coordination and information transfer in the prefrontal-amygdala circuit via a phase-specific coding mechanism, facilitating the encoding and expression of fear memory. In the search for the origin of this oscillation, we focused our attention on breathing, the most fundamental and ubiquitous rhythmic activity in life. Using large-scale extracellular recordings from a number of structures, including the medial prefrontal cortex, hippocampus, thalamus, amygdala and nucleus accumbens in mice we identified and characterized the entrainment by breathing of a host of network dynamics across the limbic circuit. We established that fear-related 4Hz oscillations are a state-specific manifestation of this cortical entrainment by the respiratory rhythm. We characterized the translaminar and transregional profile of this entrainment and demonstrated a causal role of breathing in synchronizing neuronal activity and network dynamics between these structures in a variety of behavioral scenarios in the awake and sleep state. We further revealed a dual mechanism of respiratory entrainment, in the form of an intracerebral corollary discharge that acts jointly with an olfactory reafference to coordinate limbic network dynamics, such as hippocampal ripples and cortical UP and DOWN states, involved in memory consolidation. Respiration provides a perennial stream of rhythmic input to the brain. In addition to its role as the condicio sine qua non for life, here we provide evidence that breathing rhythm acts as a global pacemaker for the brain, providing a reference signal that enables the integration of exteroceptive and interoceptive inputs with the internally generated dynamics of the hippocampus and the neocortex. Our results highlight breathing, a perennial rhythmic input to the brain, as an oscillatory scaffold for the functional coordination of the limbic circuit, enabling the segregation and integration of information flow across neuronal networks

Oscillation Phase Locking and Late ERP Components of Intracranial Hippocampal Recordings Correlate to Patient Performance in a Working Memory Task

In working memory tasks, stimulus presentation induces a resetting of intracranial temporal lobe oscillations in multiple frequency bands. To further understand the functional relevance of this phenomenon, we investigated whether working memory performance depends on the phase precision of ongoing oscillations in the hippocampus. We recorded intra-hippocampal local field potentials in individuals performing a working memory task. Two types of trials were administered. For high memory trials presentation of a list of four letters ( List ) was followed by a single letter memory probe ( Test ). Low memory load trials, consisting of four identical letters (AAAA) followed by a probe with the same letter (A), were interspersed. Significant phase locking of ongoing oscillations across trials, estimated by the Pairwise Phase Consistency Index (PPCI) was observed in delta (0.5-4 Hz), theta (5-7 Hz), and alpha (8-12 Hz) bands during stimulus presentation and recall but was increased in low memory load trials. Across patients however, higher delta PPCIs during recall in the left hippocampus were associated with faster reaction times. Because phase locking could also be interpreted as a consequence of a stimulus evoked potential, we performed event related potential analysis (ERP) and examined the relationship of ERP components with performance. We found that both amplitude and latency of late ERP components correlated with both reaction time and accuracy. We propose that, in the Sternberg task, phase locking of oscillations, or alternatively its ERP correlate, synchronizes networks within the hippocampus and connected structures that are involved in working memory

Oscillatory brain responses to own names uttered by unfamiliar and familiar voices

AbstractAmong auditory stimuli, the own name is one of the most powerful and it is able to automatically capture attention and elicit a robust electrophysiological response. The subject’s own name (SON) is preferentially processed in the right hemisphere, mainly because of its self-relevance and emotional content, together with other personally relevant information such as the voice of a familiar person. Whether emotional and self-relevant information are able to attract attention and can be, in future, introduced in clinical studies remains unclear. In the present study we used EEG and asked participants to count a target name (active condition) or to just listen to the SON or other unfamiliar names uttered by a familiar or unfamiliar voice (passive condition). Data reveals that the target name elicits a strong alpha event related desynchronization with respect to non-target names and triggers in addition a left lateralized theta synchronization as well as delta synchronization.In the passive condition alpha desynchronization was observed for familiar voice and SON stimuli in the right hemisphere.Altogether we speculate that participants engage additional attentional resources when counting a target name or when listening to personally relevant stimuli which is indexed by alpha desynchronization whereas left lateralized theta synchronization may be related to verbal working memory load. After validating the present protocol in healthy volunteers it is suggested to move one step further and apply the protocol to patients with disorders of consciousness in which the degree of residual cognitive processing and self-awareness is still insufficiently understood

Multiple Media Correlation: Theory and Applications

This thesis introduces multiple media correlation, a new technology for the automatic alignment of multiple media objects such as text, audio, and video. This research began with the question: what can be learned when multiple multimedia components are analyzed simultaneously? Most ongoing research in computational multimedia has focused on queries, indexing, and retrieval within a single media type. Video is compressed and searched independently of audio, text is indexed without regard to temporal relationships it may have to other media data. Multiple media correlation provides a framework for locating and exploiting correlations between multiple, potentially heterogeneous, media streams. The goal is computed synchronization, the determination of temporal and spatial alignments that optimize a correlation function and indicate commonality and synchronization between media objects. The model also provides a basis for comparison of media in unrelated domains. There are many real-world applications for this technology, including speaker localization, musical score alignment, and degraded media realignment. Two applications, text-to-speech alignment and parallel text alignment, are described in detail with experimental validation. Text-to-speech alignment computes the alignment between a textual transcript and speech-based audio. The presented solutions are effective for a wide variety of content and are useful not only for retrieval of content, but in support of automatic captioning of movies and video. Parallel text alignment provides a tool for the comparison of alternative translations of the same document that is particularly useful to the classics scholar interested in comparing translation techniques or styles. The results presented in this thesis include (a) new media models more useful in analysis applications, (b) a theoretical model for multiple media correlation, (c) two practical application solutions that have wide-spread applicability, and (d) Xtrieve, a multimedia database retrieval system that demonstrates this new technology and demonstrates application of multiple media correlation to information retrieval. This thesis demonstrates that computed alignment of media objects is practical and can provide immediate solutions to many information retrieval and content presentation problems. It also introduces a new area for research in media data analysis

Grip Force Reveals the Context Sensitivity of Language-Induced Motor Activity during “Action Words

Studies demonstrating the involvement of motor brain structures in language processing typically focus on \ud time windows beyond the latencies of lexical-semantic access. Consequently, such studies remain inconclusive regarding whether motor brain structures are recruited directly in language processing or through post-linguistic conceptual imagery. In the present study, we introduce a grip-force sensor that allows online measurements of language-induced motor activity during sentence listening. We use this tool to investigate whether language-induced motor activity remains constant or is modulated in negative, as opposed to affirmative, linguistic contexts. Our findings demonstrate that this simple experimental paradigm can be used to study the online crosstalk between language and the motor systems in an ecological and economical manner. Our data further confirm that the motor brain structures that can be called upon during action word processing are not mandatorily involved; the crosstalk is asymmetrically\ud governed by the linguistic context and not vice versa

DocMIR: An automatic document-based indexing system for meeting retrieval

This paper describes the DocMIR system which captures, analyzes and indexes automatically meetings, conferences, lectures, etc. by taking advantage of the documents projected (e.g. slideshows, budget tables, figures, etc.) during the events. For instance, the system can automatically apply the above-mentioned procedures to a lecture and automatically index the event according to the presented slides and their contents. For indexing, the system requires neither specific software installed on the presenter's computer nor any conscious intervention of the speaker throughout the presentation. The only material required by the system is the electronic presentation file of the speaker. Even if not provided, the system would temporally segment the presentation and offer a simple storyboard-like browsing interface. The system runs on several capture boxes connected to cameras and microphones that records events, synchronously. Once the recording is over, indexing is automatically performed by analyzing the content of the captured video containing projected documents and detects the scene changes, identifies the documents, computes their duration and extracts their textual content. Each of the captured images is identified from a repository containing all original electronic documents, captured audio-visual data and metadata created during post-production. The identification is based on documents' signatures, which hierarchically structure features from both layout structure and color distributions of the document images. Video segments are finally enriched with textual content of the identified original documents, which further facilitate the query and retrieval without using OCR. The signature-based indexing method proposed in this article is robust and works with low-resolution images and can be applied to several other applications including real-time document recognition, multimedia IR and augmented reality system

Inhibition in the dynamics of selective attention: an integrative model for negative priming

We introduce a computational model of the negative priming (NP) effect that includes perception, memory, attention, decision making, and action. The model is designed to provide a coherent picture across competing theories of NP. The model is formulated in terms of abstract dynamics for the activations of features, their binding into object entities, their semantic categorization as well as related memories and appropriate reactions. The dynamic variables interact in a connectionist network which is shown to be adaptable to a variety of experimental paradigms. We find that selective attention can be modeled by means of inhibitory processes and by a threshold dynamics. From the necessity of quantifying the experimental paradigms, we conclude that the specificity of the experimental paradigm must be taken into account when predicting the nature of the NP effect

The oscillatory fingerprints of self-prioritization : Novel markers in spectral EEG for self-relevant processing

Funding Information: The research reported in this article was supported by a Grant from the Deutsche Forschungsgemeinschaft (SCHA 2253/1–1). Open Access funding enabled and organized by Projekt DEAL.Peer reviewedPublisher PD