Prä- und postnatale Entwicklung topographischer Transformationen im Gehirn

This dissertation connects two independent fields of theoretical neuroscience: on the one hand, the self-organization of topographic connectivity patterns, and on the other hand, invariant object recognition, that is the recognition of objects independently of their various possible retinal representations (for example due to translations or scalings). The topographic representation is used in the presented approach, as a coordinate system, which then allows for the implementation of invariance transformations. Hence this study shows, that it is possible that the brain self-organizes before birth, so that it is able to invariantly recognize objects immediately after birth. Besides the core hypothesis that links prenatal work with object recognition, advancements in both fields themselves are also presented. In the beginning of the thesis, a novel analytically solvable probabilistic generative model for topographic maps is introduced. And at the end of the thesis, a model that integrates classical feature-based ideas with the normalization-based approach is presented. This bilinear model makes use of sparseness as well as slowness to implement "optimal" topographic representations. It is therefore a good candidate for hierarchical processing in the brain and for future research.Die vorliegende Arbeit verbindet zwei bisher unabhängig untersuchte Gebiete der theoretischen Neurowissenschaften: zum Einen die vorgeburtliche Selbstorganisation topographischer Verbindungsstrukturen und zum Anderen die invariante Objekterkennung, das heisst, die Erkennung von Objekten trotz ihrer mannigfaltigen retinalen Darstellungen (zum Beispiel durch Verschiebungen oder Skalierungen). Die topographische Repräsentierung wird hierbei während der Selbstorganisation als Koordinatensystem genutzt, um Invarianztransformationen zu implementieren. Dies zeigt die Möglichkeit auf, dass sich das Gehirn bereits vorgeburtlich detailliert selbstorganisieren kann, um nachgeburtlich sofort invariant Erkennen zu können. Im Detail führt Kapitel 2 in ein neues, probabilistisch generatives und analytisch lösbares Modell zur Ontogenese topographischer Transformationen ein. Dem Modell liegt die Annahme zugrunde, dass Ausgabezellen des Systems nicht völlig unkorreliert sind, sondern eine a priori gegebene Korrelation erreichen wollen. Da die Eingabezellen nachbarschaftskorreliert sind, hervorgerufen durch retinale Wellen, ergibt sich mit der Annahme rein erregender Verbindungen eine eindeutige topographische synaptische Verbindungsstruktur. Diese entspricht der bei vielen Spezies gefundenen topographischen Karten, z.B. der Retinotopie zwischen der Retina und dem LGN, oder zwischen dem LGN und dem Neokortex. Kapitel 3 nutzt eine abstraktere Formulierung des Retinotopiemechanismus, welche durch adiabitische Elimination der Aktivitätsvariablen erreicht wird, um den Effekt retinaler Wellen auf ein Modell höherer kortikaler Informationsverarbeitung zu untersuchen. Zu diesem Zweck wird der Kortex vereinfacht als bilineares Modell betrachtet, um einfache modulatorische Nichtlinearitäten mit in Betracht ziehen zu können. Zusätzlich zu den Ein- und Ausgabezellen kommen in diesem Modell Kontrolleinheiten zum Einsatz, welche den Informationsfluss aktiv steuern können und sich durch Wettbewerb und pränatalem Lernen auf verschiedene Muster retinaler Wellen spezialisieren. Die Ergebnisse zeigen, dass die entstehenden Verbindungsstrukturen affinen topographischen Abbildungen (insbesondere Translation, Skalierung und Orientierung) entsprechen, die nach Augenöffnen invariante Erkennung ermöglichen, da sie Objekte in der Eingabe in eine normalisierte Repräsentierung transformieren können. Das Modell wird für den eindimensionalen Fall ausführlich analysiert und die Funktionalität für den biologisch relevanteren zweidimensionalen Fall aufgezeigt. Kapitel 4 verallgemeinert das bilineare Modell des dritten Kapitels zu einem mehrschichtigen Modell, die shifter curcuits''. Diese ermöglichen eine logarithmisch in der Anzahl der Eingabezellen wachsende Anzahl an Synapsen, statt einer prohibitiv quadratischen Anzahl. Ausgenutzt wird die Orthogonalität von Translationen im Raum der Verbindungsstrukturen um diese durch harten Wettbewerb an einzelnen Synapsen zu organisieren. Neurobiologisch ist dieser Mechanismus durch Wettbewerb um einen wachstumsregulierenden Transmitter realisierbar. Kapitel 5 nutzt Methoden des probabilistischen Lernens, um das bilineare Modell auf das Lernen von optimalen Repräsentation der Eingabestatistiken zu optimieren. Da statistischen Methoden zweiter Ordnung, wie zum Beispiel das generative Modell aus Kapitel 2, keine lokalisierten rezeptiven Felder ermöglichen und somit keine (örtliche) Topographie möglich ist, wird sparseness'' verwendet um statistischen Abhängigkeiten höherer Ordnung zu lernen und gleichzeitig Topographie zu implementieren. Anwendungen des so formulierten Modells auf natürliche Bilder zeigen, dass lokalisierte, bandpass filternde rezeptive Felder entstehen, die primären kortikalen rezeptiven Feldern stark ähneln. Desweiteren entstehen durch die erzwungene Topographie Orientierungs- und Frequenzkarten, die ebenfalls kortikalen Karten ähneln. Eine Untersuchung des Modells mit zusätzlicher slowness'' der Ausgabezellen und in zeitlicher Nähe gezeigten transformierten natürlichen Eingabemustern zeigt, dass verschiedene Kontrolleinheiten konsistente und den Eingabetransformationen entsprechende rezeptive Felder entwickeln und somit invariante Darstellungen bezüglich der gezeigten Eingaben entwickeln

Independent EEG Sources Are Dipolar

Independent component analysis (ICA) and blind source separation (BSS) methods are increasingly used to separate individual brain and non-brain source signals mixed by volume conduction in electroencephalographic (EEG) and other electrophysiological recordings. We compared results of decomposing thirteen 71-channel human scalp EEG datasets by 22 ICA and BSS algorithms, assessing the pairwise mutual information (PMI) in scalp channel pairs, the remaining PMI in component pairs, the overall mutual information reduction (MIR) effected by each decomposition, and decomposition ‘dipolarity’ defined as the number of component scalp maps matching the projection of a single equivalent dipole with less than a given residual variance. The least well-performing algorithm was principal component analysis (PCA); best performing were AMICA and other likelihood/mutual information based ICA methods. Though these and other commonly-used decomposition methods returned many similar components, across 18 ICA/BSS algorithms mean dipolarity varied linearly with both MIR and with PMI remaining between the resulting component time courses, a result compatible with an interpretation of many maximally independent EEG components as being volume-conducted projections of partially-synchronous local cortical field activity within single compact cortical domains. To encourage further method comparisons, the data and software used to prepare the results have been made available (http://sccn.ucsd.edu/wiki/BSSComparison)

The Embedding Capacity of Information Flows Under Renewal Traffic

Given two independent point processes and a certain rule for matching points between them, what is the fraction of matched points over infinitely long streams? In many application contexts, e.g., secure networking, a meaningful matching rule is that of a maximum causal delay, and the problem is related to embedding a flow of packets in cover traffic such that no traffic analysis can detect it. We study the best undetectable embedding policy and the corresponding maximum flow rate ---that we call the embedding capacity--- under the assumption that the cover traffic can be modeled as arbitrary renewal processes. We find that computing the embedding capacity requires the inversion of very structured linear systems that, for a broad range of renewal models encountered in practice, admits a fully analytical expression in terms of the renewal function of the processes. Our main theoretical contribution is a simple closed form of such relationship. This result enables us to explore properties of the embedding capacity, obtaining closed-form solutions for selected distribution families and a suite of sufficient conditions on the capacity ordering. We evaluate our solution on real network traces, which shows a noticeable match for tight delay constraints. A gap between the predicted and the actual embedding capacities appears for looser constraints, and further investigation reveals that it is caused by inaccuracy of the renewal traffic model rather than of the solution itself.Comment: Sumbitted to IEEE Trans. on Information Theory on March 10, 201

An Adaptive Strategy for Sensory Processing

Recognizing objects and detecting associations among them is essential for the survival of organisms. The ability to perform these tasks is derived from the representations of objects obtained through processing information along sensory pathways. Our current understanding of sensory processing is based on two sets of foundational theories – The Efficient Coding Hypothesis and hierarchical assembly of object representations. These theories suggest that sensory processing aims to identify independent features of the environment and progressively represent objects in terms of comprehensive combinations of these features. Separately, the two sets of theories have successfully explained the detection of associations and perceptual invariance, respectively; however, reconciling them together in one unified theory has remained challenging. Independent features are deemed essential for detecting association by the Efficient coding hypothesis, but to achieve consistency in representations, multiple comprehensive structures corresponding to the same object must be hierarchically assembled, ignoring independence among such structures. Here we propose an alternative framework for sensory processing in which the system, instead of finding the truly independent components of the environment, aims to represent objects based on their most informative structures. Using theoretical arguments, we show that following such a strategy allows the system to efficiently represent sensory cues without necessarily acquiring knowledge about statistical properties of all possible inputs. Through mathematical simulations, we find that the framework can describe the known characteristics of early sensory processing stages and permits consistent input representations observed at later stages of processing. We also demonstrate that the framework can be implemented in a biologically plausible neuronal circuit and explain aspects of experience and learning from corrupted inputs. Thus, this framework provides a novel perspective and a unified description of sensory processing in its entirety

Epälineaarisen visuaalisen prosessoinnin oppiminen luonnollisista kuvista

The paradigm of computational vision hypothesizes that any visual function -- such as the recognition of your grandparent -- can be replicated by computational processing of the visual input. What are these computations that the brain performs? What should or could they be? Working on the latter question, this dissertation takes the statistical approach, where the suitable computations are attempted to be learned from the natural visual data itself. In particular, we empirically study the computational processing that emerges from the statistical properties of the visual world and the constraints and objectives specified for the learning process. This thesis consists of an introduction and 7 peer-reviewed publications, where the purpose of the introduction is to illustrate the area of study to a reader who is not familiar with computational vision research. In the scope of the introduction, we will briefly overview the primary challenges to visual processing, as well as recall some of the current opinions on visual processing in the early visual systems of animals. Next, we describe the methodology we have used in our research, and discuss the presented results. We have included some additional remarks, speculations and conclusions to this discussion that were not featured in the original publications. We present the following results in the publications of this thesis. First, we empirically demonstrate that luminance and contrast are strongly dependent in natural images, contradicting previous theories suggesting that luminance and contrast were processed separately in natural systems due to their independence in the visual data. Second, we show that simple cell -like receptive fields of the primary visual cortex can be learned in the nonlinear contrast domain by maximization of independence. Further, we provide first-time reports of the emergence of conjunctive (corner-detecting) and subtractive (opponent orientation) processing due to nonlinear projection pursuit with simple objective functions related to sparseness and response energy optimization. Then, we show that attempting to extract independent components of nonlinear histogram statistics of a biologically plausible representation leads to projection directions that appear to differentiate between visual contexts. Such processing might be applicable for priming, \ie the selection and tuning of later visual processing. We continue by showing that a different kind of thresholded low-frequency priming can be learned and used to make object detection faster with little loss in accuracy. Finally, we show that in a computational object detection setting, nonlinearly gain-controlled visual features of medium complexity can be acquired sequentially as images are encountered and discarded. We present two online algorithms to perform this feature selection, and propose the idea that for artificial systems, some processing mechanisms could be selectable from the environment without optimizing the mechanisms themselves. In summary, this thesis explores learning visual processing on several levels. The learning can be understood as interplay of input data, model structures, learning objectives, and estimation algorithms. The presented work adds to the growing body of evidence showing that statistical methods can be used to acquire intuitively meaningful visual processing mechanisms. The work also presents some predictions and ideas regarding biological visual processing.Laskennallisen näön paradigma esittää, että mikä tahansa näkötoiminto - esimerkiksi jonkun esineen tunnistaminen - voidaan toistaa keinotekoisesti käyttäen laskennallisia menetelmiä. Minkälaisia nämä laskennalliset menetelmät voisivat olla, tai minkälaisia niiden tulisi olla? Tässä väitöskirjassa tutkitaan tilastollista lähestymistapaa näkemisen mekanismien muodostamiseen. Sovelletussa lähestymistavassa laskennallista käsittelyä yritetään muodostaa optimoimalla (tai 'oppimalla') siten, että toivotulle käsittelylle asetetaan erilaisia tavoitteita jonkin annetun luonnollisten kuvien joukon suhteen. Väitöskirja koostuu johdannosta ja seitsemästä kansainvälisillä foorumeilla julkaistusta tutkimusartikkelista. Johdanto esittelee väitöskirjan poikkitieteellistä tutkimusaluetta niille, jotka eivät entuudestaan tunne laskennallista näkötutkimusta. Johdannossa käydään läpi visuaalisen prosessoinnin haasteita sekä valotetaan hieman tämänhetkisiä mielipiteitä biologisista näkömekanismeista. Seuraavaksi lukija tutustutetaan työssä käytettyyn tutkimusmetodologiaan, jonka voi pitkälti nähdä koneoppimisen (tilastotieteen) soveltamisena. Johdannon lopuksi käydään läpi työn tutkimusartikkelit. Tämä katsaus on varustettu sellaisilla lisäkommenteilla, havainnoilla ja kritiikeillä, jotka eivät sisältyneet alkuperäisiin artikkeleihin. Varsinaiset tulokset väitöskirjassa liittyvät siihen, minkälaisia yksinkertaisia prosessointimekanismeja muodostuu yhdistelemällä erilaisia oppimistavoitteita, funktioluokkia, epälineaarisuuksia ja luonnollista kuvadataa. Työssä tarkastellaan erityisesti representaatioiden riippumattomuuteen ja harvuuteen tähtääviä oppimistavoitteita, mutta myös sellaisia, jotka pyrkivät edesauttamaan objektintunnistuksessa. Esitämme näiden aiheiden tiimoilta uusia löydöksiä, jotka listataan tarkemmin sekä englanninkielisessä tiivistelmässä että väitöskirjan alkusivuilla. Esitetty väitöskirjatyö tarjoaa lisänäyttöä siitä, että intuitiivisesti mielekkäitä visuaalisia prosessointimekanismeja voidaan muodostaa tilastollisin keinoin. Työ tarjoaa myös joitakin ennusteita ja ideoita liittyen biologisiin näkömekanismeihin

A Probabilistic Approach to the Primary Visual Cortex

What can the statistical structure of natural images teach us about the human brain? Even though the visual cortex is one of the most studied parts of the brain, surprisingly little is known about how exactly images are processed to leave us with a coherent percept of the world around us, so we can recognize a friend or drive on a crowded street without any effort. By constructing probabilistic models of natural images, the goal of this thesis is to understand the structure of the stimulus that is the raison d etre for the visual system. Following the hypothesis that the optimal processing has to be matched to the structure of that stimulus, we attempt to derive computational principles, features that the visual system should compute, and properties that cells in the visual system should have. Starting from machine learning techniques such as principal component analysis and independent component analysis we construct a variety of sta- tistical models to discover structure in natural images that can be linked to receptive field properties of neurons in primary visual cortex such as simple and complex cells. We show that by representing images with phase invariant, complex cell-like units, a better statistical description of the vi- sual environment is obtained than with linear simple cell units, and that complex cell pooling can be learned by estimating both layers of a two-layer model of natural images. We investigate how a simplified model of the processing in the retina, where adaptation and contrast normalization take place, is connected to the nat- ural stimulus statistics. Analyzing the effect that retinal gain control has on later cortical processing, we propose a novel method to perform gain control in a data-driven way. Finally we show how models like those pre- sented here can be extended to capture whole visual scenes rather than just small image patches. By using a Markov random field approach we can model images of arbitrary size, while still being able to estimate the model parameters from the data