Top-Down Control of Lateral Interactions in Visual Cortex

V1 neurons are capable of integrating information over a large area of visual field. Their responses to local features are dependent on the global characteristics of contours and surfaces that extend well beyond their receptive fields. These contextual influences in V1 are subject to cognitive influences of attention, perceptual task and expectation. Previously it’s been shown that the response properties of V1 neurons change to carry more information about behaviorally relevant stimulus features (Li et al. 2004). We hypothesized that top-down modulation of effective connectivity within V1 underlies the behaviorally dependent modulations of contextual interactions in V1. To test this idea, we used a chronically implanted multi-electrode array in awake primates and studied the mechanisms of top-down control of contextual interactions in V1. We used a behavioral paradigm in which the animals performed two different perceptual tasks on the same stimulus and studied task-dependent changes in connectivity between V1 sites that encode the stimulus. We found that V1 interactions-both spiking and LFP interactions-showed significant task-dependent changes. The direction of the task-dependent changes observed in LFP interactions, measured by coherence between LFP signals, was dependent on the perceptual strategy used by the animal. Bisection task involving perceptual grouping of parallel lines increased LFP coherence while vernier task involving segregation of collinear line decrease LFP coherence. Also, grouping of collinear lines to detect a contour resulted in increased LFP interactions. Since noise correlations can affect the coding accuracy of a cortical network, we investigated how top-down processes of attention and perceptual task affect V1 noise correlations. We were able to study the noise correlation dynamics that were due to attentional shift separately from the changes due to the perceptual task being performed at the attended location. Top-down influences reduced V1 noise-correlations to a greater extent when the animal performed a discrimination task at the recorded locations compared to when the animal shifted its attention to the location. The reduction in noise correlation during the perceptual task was accompanied by a significant increase in the information carried about the stimulus (calculated as Fisher information). Our analysis was also able to determine the degree to which the task dependent change in information was due to the alteration in neuronal tuning compared to changes in correlated activity. Interestingly, the largest effects on information were seen between stimuli that had the greatest difficulty of discrimination

Video browsing interfaces and applications: a review

We present a comprehensive review of the state of the art in video browsing and retrieval systems, with special emphasis on interfaces and applications. There has been a significant increase in activity (e.g., storage, retrieval, and sharing) employing video data in the past decade, both for personal and professional use. The ever-growing amount of video content available for human consumption and the inherent characteristics of video data—which, if presented in its raw format, is rather unwieldy and costly—have become driving forces for the development of more effective solutions to present video contents and allow rich user interaction. As a result, there are many contemporary research efforts toward developing better video browsing solutions, which we summarize. We review more than 40 different video browsing and retrieval interfaces and classify them into three groups: applications that use video-player-like interaction, video retrieval applications, and browsing solutions based on video surrogates. For each category, we present a summary of existing work, highlight the technical aspects of each solution, and compare them against each other

Enhancing person annotation for personal photo management using content and context based technologies

Rapid technological growth and the decreasing cost of photo capture means that we are all taking more digital photographs than ever before. However, lack of technology for automatically organising personal photo archives has resulted in many users left with poorly annotated photos, causing them great frustration when such photo collections are to be browsed or searched at a later time. As a result, there has recently been significant research interest in technologies for supporting effective annotation. This thesis addresses an important sub-problem of the broad annotation problem, namely "person annotation" associated with personal digital photo management. Solutions to this problem are provided using content analysis tools in combination with context data within the experimental photo management framework, called “MediAssist”. Readily available image metadata, such as location and date/time, are captured from digital cameras with in-built GPS functionality, and thus provide knowledge about when and where the photos were taken. Such information is then used to identify the "real-world" events corresponding to certain activities in the photo capture process. The problem of enabling effective person annotation is formulated in such a way that both "within-event" and "cross-event" relationships of persons' appearances are captured. The research reported in the thesis is built upon a firm foundation of content-based analysis technologies, namely face detection, face recognition, and body-patch matching together with data fusion. Two annotation models are investigated in this thesis, namely progressive and non-progressive. The effectiveness of each model is evaluated against varying proportions of initial annotation, and the type of initial annotation based on individual and combined face, body-patch and person-context information sources. The results reported in the thesis strongly validate the use of multiple information sources for person annotation whilst emphasising the advantage of event-based photo analysis in real-life photo management systems

CA1-projecting subiculum neurons facilitate object-place learning.

Recent anatomical evidence suggests a functionally significant back-projection pathway from the subiculum to the CA1. Here we show that the afferent circuitry of CA1-projecting subicular neurons is biased by inputs from CA1 inhibitory neurons and the visual cortex, but lacks input from the entorhinal cortex. Efferents of the CA1-projecting subiculum neurons also target the perirhinal cortex, an area strongly implicated in object-place learning. We identify a critical role for CA1-projecting subicular neurons in object-location learning and memory, and show that this projection modulates place-specific activity of CA1 neurons and their responses to displaced objects. Together, these experiments reveal a novel pathway by which cortical inputs, particularly those from the visual cortex, reach the hippocampal output region CA1. Our findings also implicate this circuitry in the formation of complex spatial representations and learning of object-place associations

Action Recognition in Videos: from Motion Capture Labs to the Web

This paper presents a survey of human action recognition approaches based on visual data recorded from a single video camera. We propose an organizing framework which puts in evidence the evolution of the area, with techniques moving from heavily constrained motion capture scenarios towards more challenging, realistic, "in the wild" videos. The proposed organization is based on the representation used as input for the recognition task, emphasizing the hypothesis assumed and thus, the constraints imposed on the type of video that each technique is able to address. Expliciting the hypothesis and constraints makes the framework particularly useful to select a method, given an application. Another advantage of the proposed organization is that it allows categorizing newest approaches seamlessly with traditional ones, while providing an insightful perspective of the evolution of the action recognition task up to now. That perspective is the basis for the discussion in the end of the paper, where we also present the main open issues in the area.Comment: Preprint submitted to CVIU, survey paper, 46 pages, 2 figures, 4 table

Computational Mechanisms Underlying Perception Of Visual Motion

Motion is a fundamental property estimated by human sensory-perception. When visual shapes and patterns change their positions over time, we perceive motion. Relating properties of perceived motion—speed and direction—to properties of visual stimuli is an important endeavor in vision science. Understanding this relationship requires an understanding of the computations performed by the visual system to extract motion information from visual stimuli. The present research sheds light on the nature of these computations. In the first study, human performance in a speed discrimination task with naturalistic stimuli is compared to performance of an ideal observer model. The ideal observer model utilizes computations that have been optimized for discriminating speed among a large training set of naturalistic stimuli. Although human performance falls short of ideal observer performance because of the presence of internal noise, the remarkable finding is that the computations performed minimize, to the maximum possible extent, the performance limits imposed by external stimulus variability. In other words, humans perform computations that are optimal. The second study focuses on how spatial frequency, a basic characteristic of visual patterns, impacts the process by which the visual system integrates motion across time (temporal integration). A continuous target-tracking task demonstrates that longer temporal integration periods are associated with higher spatial frequencies. This predicts a visual depth illusion when the left and right eyes are simultaneously presented stimuli having different spatial frequencies. A second experiment using traditional forced-choice psychophysics confirms this prediction. The third study explores how color impacts estimates of spatial position during motion. We parameterize color in terms of L-cone and S-cone activity modulations in the eye. Using the same continuous target-tracking paradigm from Chapter 2, we demonstrate that position estimates for stimuli comprised of pure S-cone modulations lag behind position estimates for stimuli comprised of pure L-cone modulations. A key finding is that when L-cone and S-cone modulations are combined, processing lag is almost exclusively determined by L-cone modulations

Mechanisms of memory consolidation : Analyzing the coordinated activity of concept neurons in the human medial temporal lobe during waking and sleep

The aim of this thesis is to investigate the role of human concept neurons in memory consolidation during sleep. Memory consolidation is a process by which memories initially dependent on the hippocampus are transferred to cortical areas, thereby gradually becoming independent of the hippocampus. Theories of memory consolidation posit that memory traces encoding autobiographic episodes are rapidly formed in the hippocampus during waking, and reactivated during subsequent slow-wave sleep to be transformed into a long-lasting form. Concept neurons in the human medial temporal lobe are neurons tuned to semantic concepts in a selective, sparse, and invariant manner. These neurons respond to pictures or written and spoken words representing their preferred concept (for example, a person, an animal, an object), regardless of physical stimulus properties. Concept neurons have been speculated to be building blocks for episodic memory. We used whole-night recordings from concept neurons in the medial temporal lobe of epilepsy patients implanted with depth electrodes for presurgical monitoring to test the hypothesis that the coordinated activity of concept neurons during sleep is a neurophysiological correlate of memory consolidation in humans. To conduct this study, we developed software methods for artifact removal and spike sorting of long-term recordings from single neurons. In an evaluation on both simulated model data and visual stimulus presentation experiments, our software outperformed previous methods. Starting from the conceptual analogy between rodent place cells and human concept neurons, we developed an episodic memory task in which participants learned a story eliciting sequential activity in concept neurons. We found that concept neurons preserved their semantic tuning across whole-night recordings. Hippocampal concept neurons had, on average, lower firing rates during rapid-eye-movement (REM) sleep than during waking. During slow-wave sleep, firing rates did not significantly differ from waking. The activity of concept neurons increased during ripples in the local field potential. Furthermore, concept neurons whose preferred stimuli participated in the memorized story were conjointly reactivated after learning, most pronouncedly during slow-wave sleep. Cross-correlations of concept neurons were most asymmetric during slow-wave sleep. Cross-correlation peak times were often in the range believed to be relevant for spike-timing-dependent plasticity. However, time lags of peak cross-correlations did not correlate with the positional order of stimuli in the memorized story. Our findings support the hypothesis that concept neurons rapidly encode a memory trace during learning, and that the reactivation of the same neurons during subsequent slow-wave sleep and ripples contributes to the consolidation of the memory episode. However, the consolidation of the temporal order of events in humans appears to differ from what rodent research suggests.Mechanismen der Gedächtniskonsolidierung : Analyse der Aktivität von Konzeptzellen im menschlichen Schläfenlappen während Wachheit und Schlaf In dieser Arbeit wird die Rolle von Konzeptzellen ("concept neurons") im Gehirn des Menschen bei der Gedächtniskonsolidierung im Schlaf untersucht. Gedächtniskonsolidierung ist ein Prozess, durch den Gedächtnisinhalte, die zunächst vom Hippokampus abhängen, in die Großhirnrinde übertragen werden. Dadurch reduziert sich im Laufe der Zeit die Abhängigkeit der Gedächtnisinhalte vom Hippokampus. In der Theorie der Gedächtniskonsolidierung wird angenommen, dass während wachem Erleben sehr schnell Gedächtnisspuren im Hippokampus entstehen, welche im darauffolgenden Tiefschlaf reaktiviert werden, um so eine langfristig stabile Gedächtnisspur zu erzeugen. Konzeptzellen im Schläfenlappen des Menschen sind Nervenzellen, die auf den semantischen Inhalt eines Stimulus selektiv und semantisch invariant reagieren. Konzeptzellen antworten auf Abbildungen ihres präferierten Konzepts (zum Beispiel einer Person, eines Tieres oder eines Objekts) oder auf geschriebene und gesprochene Wörter, die das gleiche Konzept darstellen, unabhängig von den speziellen Eigenschaften des Stimulus, wie zum Beispiel Bildgröße oder -farbe. Auf jedes Konzept reagiert dabei nur ein sehr kleiner Teil dieser Zellen. Man vermutet, dass Konzeptzellen Bausteine des episodischen Gedächtnisses sind. Die vorliegende Studie nutzt Aufzeichnungen der Aktivität einzelner Konzeptzellen während ganzer Nächte, um zu untersuchen, inwiefern die koordinierte Aktivität von Konzeptzellen im Schlaf ein neurophysiologisches Korrelat der Gedächtniskonsolidierung darstellt. Die Teilnehmer der Studie waren Epilepsiepatienten, in deren mediale Schläfenlappen aus klinischen Gründen Tiefenelektroden zur Anfallsaufzeichnung implantiert worden waren. Zur Analyse der Daten wurde zunächst eine Software entwickelt, die eine Artefaktbereinigung und das Spike-Sorting von neuronalen Langzeitaufzeichnungen leistet. Diese Software zeigte deutliche Vorteile gegenüber vorhandenen Methoden, und zwar sowohl in Tests mit simulierten Modelldatensätzen als auch im Falle tatsächlicher Aufzeichnungen (hier Experimente, in denen visuelle Stimuli auf einem Laptop dargestellt wurden). Ausgehend von einer Analogie zwischen Ortszellen ("place cells") bei Nagetieren und Konzeptzellen bei Menschen wurde ein Experiment entwickelt, das episodisches Gedächtnis operationalisierte. Darin lernten die Teilnehmer eine kurze Geschichte auswendig, was sequentielle Aktivität von Konzeptzellen auslöste. Konzeptzellen zeigten ein stabiles Antwortverhalten: am Abend und nächsten Morgen antworteten sie auf die gleichen Stimuli. Konzeptzellen im Hippokampus hatten im Mittel im Rapid-Eye-Movement-Schlaf (REM-Schlaf) niedrigere Feuerraten als während Wachheit. Im Tiefschlaf unterschieden sich die Feuerraten nicht signifikant von Wachheit. Die Aktivität der Konzeptzellen war während "ripples" im lokalen Feldpotential erhöht, und Konzeptzellen, deren präferierte Stimuli in der erinnerten Geschichte auftauchten, feuerten im darauffolgenden Schlaf gemeinsam, ein Effekt, der im Tiefschlaf besonders ausgeprägt war. Die Kreuzkorrelationen von Konzeptzellen waren im Tiefschlaf asymmetrischer als während Wachheit und REM-Schlaf, und die typischen Zeitabstände des Feuerns von Konzeptzellen lagen in einem Bereich, der als relevant für "spike-timing-dependent plasticity" gilt. Die Zeitabstände waren jedoch nicht mit dem Abstand der präferierten Stimuli in der erinnerten Geschichte korreliert. Diese Befunde stützen die Theorie, dass die Aktivität von Konzeptzellen während des Lernens instantan eine Gedächtnisspur erzeugt, und dass die Reaktivierung der gleichen Nervenzellen im Tiefschlaf nach dem Lernen zur Konsolidierung der Gedächtnisinhalte beiträgt. Die zeitliche Reihenfolge von Ereignissen wird offenbar im menschlichen Gehirn nicht auf die Weise konsolidiert, die sich aus der Forschung an Nagetieren nahelegte

Identification, indexing, and retrieval of cardio-pulmonary resuscitation (CPR) video scenes of simulated medical crisis.

Medical simulations, where uncommon clinical situations can be replicated, have proved to provide a more comprehensive training. Simulations involve the use of patient simulators, which are lifelike mannequins. After each session, the physician must manually review and annotate the recordings and then debrief the trainees. This process can be tedious and retrieval of specific video segments should be automated. In this dissertation, we propose a machine learning based approach to detect and classify scenes that involve rhythmic activities such as Cardio-Pulmonary Resuscitation (CPR) from training video sessions simulating medical crises. This applications requires different preprocessing techniques from other video applications. In particular, most processing steps require the integration of multiple features such as motion, color and spatial and temporal constrains. The first step of our approach consists of segmenting the video into shots. This is achieved by extracting color and motion information from each frame and identifying locations where consecutive frames have different features. We propose two different methods to identify shot boundaries. The first one is based on simple thresholding while the second one uses unsupervised learning techniques. The second step of our approach consists of selecting one key frame from each shot and segmenting it into homogeneous regions. Then few regions of interest are identified for further processing. These regions are selected based on the type of motion of their pixels and their likelihood to be skin-like regions. The regions of interest are tracked and a sequence of observations that encode their motion throughout the shot is extracted. The next step of our approach uses an HMM classiffier to discriminate between regions that involve CPR actions and other regions. We experiment with both continuous and discrete HMM. Finally, to improve the accuracy of our system, we also detect faces in each key frame, track them throughout the shot, and fuse their HMM confidence with the region\u27s confidence. To allow the user to view and analyze the video training session much more efficiently, we have also developed a graphical user interface (GUI) for CPR video scene retrieval and analysis with several desirable features. To validate our proposed approach to detect CPR scenes, we use one video simulation session recorded by the SPARC group to train the HMM classifiers and learn the system\u27s parameters. Then, we analyze the proposed system on other video recordings. We show that our approach can identify most CPR scenes with few false alarms

Distinct ensembles in the noradrenergic locus coeruleus are associated with diverse cortical states

The noradrenergic locus coeruleus (LC) is a controller of brain and behavioral states. Activating LC neurons en masse by electrical or optogenetic stimulation promotes a stereotypical "activated" cortical state of high-frequency oscillations. However, it has been recently reported that spontaneous activity of LC cell pairs has sparse yet structured time-averaged cross-correlations, which is unlike the highly synchronous neuronal activity evoked by stimulation. Therefore, LC population activity could consist of distinct multicell ensembles each with unique temporal evolution of activity. We used nonnegative matrix factorization (NMF) to analyze large populations of simultaneously recorded LC single units in the rat LC. NMF identified ensembles of spontaneously coactive LC neurons and their activation time courses. Since LC neurons selectively project to specific forebrain regions, we hypothesized that distinct ensembles activate during different cortical states. To test this hypothesis, we calculated band-limited power and spectrograms of local field potentials in cortical area 24a aligned to spontaneous activations of distinct LC ensembles. A diversity of state modulations occurred around activation of different LC ensembles, including a typical activated state with increased highfrequency power as well as other states including decreased high-frequency power. Thus-in contrast to the stereotypical activated brain state evoked by en masse LC stimulation-spontaneous activation of distinct LC ensembles is associated with a multitude of cortical states.Peer reviewe

The Timescales of Transformation Across Brain Structures in the Thalamocortical Circuit

Sensory processing requires reliable transmission of sensory information across multiple brain regions, from peripheral sensors, through sub-cortical structures, to sensory cortex, ultimately producing the sensory representations that drive perception and behavior. Despite decades of research, we do not yet have a mechanistic understanding of how neural representations are transformed across these critical brain structures. This is primarily due to the fact that what we know at the circuit level has been mainly derived from electrophysiological recordings targeted at single regions and upon gross anatomical connection patterns across brain regions without specific, precise knowledge of synaptic connectivity. To fill this gap in knowledge and to uncover how signaling changes across brain regions in response to changes in the sensory environment, this thesis work has two primary contributions. First, we developed a work-flow of topographic mapping and histological validation for extracellular multi-electrode recordings of neurons in the thalamocortical circuit in rodents, followed by a novel statistical approach for inferring synaptic connectivity across the brain regions. Specifically, we developed a signal-detection based classification of synaptic connectivity in the thalamus and S1 cortex, with an assessment of classification confidence that is scalable to the large-scale recording approaches that are emerging in the field. Utilizing this experimental and computational framework, we next investigated the neural mechanisms that underlie an important sensory phenomenon that emerges in this early sensory circuit: rapid sensory adaptation. While this phenomenon has been well-studied over very rapid timescales of hundreds of milliseconds, other studies suggest that longer time scales of 10’s of seconds may also be relevant. Here, we demonstrated that the thalamus and the thalamorecipient layer 4 excitatory and inhibitory neurons in S1 exhibit differential adaptation dynamics, and that the neuronal dynamics across these different regions and cell types show common signatures of multiple timescales in response to sensory adaptation. We characterized the adaptation profiles at the TC junction and further identified several mechanisms that potentially underlie the adaptation effects on the circuit dynamics, including synaptic depression of the TC synapse in identified monosynaptically connected thalamic and cortical neurons, and changes in spike timing and synchronization in the thalamic population. These mechanisms together mediate a dynamic trade-off in the theoretical detectability and discriminability of stimulus inputs. These results suggest that adaptation of the thalamocortical circuit across timescales results from a complex interaction between distinct mechanisms, and notably the engagement of different mechanisms can shift depending on the timescale of environmental changes.Ph.D