GANs schön kompliziert: Applications of Generative Adversarial Networks

Scientific research progresses via model-building. Researchers attempt to build realistic models of real-world phenomena, ranging from bacterial growth to galactic motion, and study these models as a means of understanding these phenomena. However, making these models as realistic as possible often involves fitting them to experimentally measured data. Recent advances in experimental methods have allowed for the collection of large-scale datasets. Simultaneously, advancements in computational capacity have allowed for more complex model-building. The confluence of these two factors accounts for the rise of machine learning methods as powerful tools, both for building models and fitting these models to large scale datasets. In this thesis, we use a particular machine learning technique: generative adversarial networks (GANs). GANs are a flexible and powerful tool, capable of fitting a wide variety of models. We explore the properties of GANs that underpin this flexibility, and show how we can capitalize on them in different scientific applications, beyond the image- and text-generating applications they are well-known for. Here we present three different applications of GANs. First, we show how GANs can be used as generative models of neural spike trains, and how they are capable of capturing more features of these spike trains compared to other approaches. We also show how this could enable insight into how information about stimuli are encoded in the spike trains. Second, we demonstrate how GANs can be used as density estimators for extending simulation-based Bayesian inference to high-dimensional parameter spaces. In this form, we also show how GANs bridge Bayesian inference methods and variational inference with autoencoders and use them to fit complex climate models to data. Finally, we use GANs to infer synaptic plasticity rules for biological rate networks directly from data. We then show how GANs be used to test the robustness of the inferred rules to differences in data and network initialisation. Overall, we repurpose GANs in new ways for a variety of scientific domains, and show that they confer specific advantages over the state-of-the-art methods in each of these domains

Biologically inspired feature extraction for rotation and scale tolerant pattern analysis

Biologically motivated information processing has been an important area of scientific research for decades. The central topic addressed in this dissertation is utilization of lateral inhibition and more generally, linear networks with recurrent connectivity along with complex-log conformal mapping in machine based implementations of information encoding, feature extraction and pattern recognition. The reasoning behind and method for spatially uniform implementation of inhibitory/excitatory network model in the framework of non-uniform log-polar transform is presented. For the space invariant connectivity model characterized by Topelitz-Block-Toeplitz matrix, the overall network response is obtained without matrix inverse operations providing the connection matrix generating function is bound by unity. It was shown that for the network with the inter-neuron connection function expandable in a Fourier series in polar angle, the overall network response is steerable. The decorrelating/whitening characteristics of networks with lateral inhibition are used in order to develop space invariant pre-whitening kernels specialized for specific category of input signals. These filters have extremely small memory footprint and are successfully utilized in order to improve performance of adaptive neural whitening algorithms. Finally, the method for feature extraction based on localized Independent Component Analysis (ICA) transform in log-polar domain and aided by previously developed pre-whitening filters is implemented. Since output codes produced by ICA are very sparse, a small number of non-zero coefficients was sufficient to encode input data and obtain reliable pattern recognition performance

Toward a further understanding of object feature binding: a cognitive neuroscience perspective.

The aim of this thesis is to lead to a further understanding of the neural mechanisms underlying object feature binding in the human brain. The focus is on information processing and integration in the visual system and visual shortterm memory. From a review of the literature it is clear that there are three major competing binding theories, however, none of these individually solves the binding problem satisfactorily. Thus the aim of this research is to conduct behavioural experimentation into object feature binding, paying particular attention to visual short-term memory. The behavioural experiment was designed and conducted using a within-subjects delayed responset ask comprising a battery of sixty-four composite objects each with three features and four dimensions in each of three conditions (spatial, temporal and spatio-temporal).Findings from the experiment,which focus on spatial and temporal aspects of object feature binding and feature proximity on binding errors, support the spatial theories on object feature binding, in addition we propose that temporal theories and convergence, through hierarchical feature analysis, are also involved. Because spatial properties have a dedicated processing neural stream, and temporal properties rely on limited capacity memory systems, memories for sequential information would likely be more difficult to accuratelyr ecall. Our study supports other studies which suggest that both spatial and temporal coherence to differing degrees,may be involved in object feature binding. Traditionally, these theories have purported to provide individual solutions, but this thesis proposes a novel unified theory of object feature binding in which hierarchical feature analysis, spatial attention and temporal synchrony each plays a role. It is further proposed that binding takes place in visual short-term memory through concerted and integrated information processing in distributed cortical areas. A cognitive model detailing this integrated proposal is given. Next, the cognitive model is used to inform the design and suggested implementation of a computational model which would be able to test the theory put forward in this thesis. In order to verify the model, future work is needed to implement the computational model.Thus it is argued that this doctoral thesis provides valuable experimental evidence concerning spatio-temporal aspects of the binding problem and as such is an additional building block in the quest for a solution to the object feature binding problem

Toward a further understanding of object feature binding : a cognitive neuroscience perspective

The aim of this thesis is to lead to a further understanding of the neural mechanisms underlying object feature binding in the human brain. The focus is on information processing and integration in the visual system and visual shortterm memory. From a review of the literature it is clear that there are three major competing binding theories, however, none of these individually solves the binding problem satisfactorily. Thus the aim of this research is to conduct behavioural experimentation into object feature binding, paying particular attention to visual short-term memory. The behavioural experiment was designed and conducted using a within-subjects delayed responset ask comprising a battery of sixty-four composite objects each with three features and four dimensions in each of three conditions (spatial, temporal and spatio-temporal).Findings from the experiment,which focus on spatial and temporal aspects of object feature binding and feature proximity on binding errors, support the spatial theories on object feature binding, in addition we propose that temporal theories and convergence, through hierarchical feature analysis, are also involved. Because spatial properties have a dedicated processing neural stream, and temporal properties rely on limited capacity memory systems, memories for sequential information would likely be more difficult to accuratelyr ecall. Our study supports other studies which suggest that both spatial and temporal coherence to differing degrees,may be involved in object feature binding. Traditionally, these theories have purported to provide individual solutions, but this thesis proposes a novel unified theory of object feature binding in which hierarchical feature analysis, spatial attention and temporal synchrony each plays a role. It is further proposed that binding takes place in visual short-term memory through concerted and integrated information processing in distributed cortical areas. A cognitive model detailing this integrated proposal is given. Next, the cognitive model is used to inform the design and suggested implementation of a computational model which would be able to test the theory put forward in this thesis. In order to verify the model, future work is needed to implement the computational model.Thus it is argued that this doctoral thesis provides valuable experimental evidence concerning spatio-temporal aspects of the binding problem and as such is an additional building block in the quest for a solution to the object feature binding problem.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Modelling the anaestheto-dynamic phase transition of the cerebral cortex

This thesis examines a stochastic model for the electrical behaviour of the cerebral cortex under the influence of a general anaesthetic agent. The modelling element is the macrocolumn, an organized assembly of ∼10⁵ cooperating neurons (85% excitatory, 15% inhibitory) within a small cylindrical volume (∼1 mm³) of the cortex. The state variables are hₑ and hᵢ, the mean-field average soma voltages for the populations of excitatory (e) and inhibitory (i) neurons comprising the macrocolumn. The random fluctuations of hₑ about its steady-state value are taken as the source of the scalp-measured EEG signal. The randomness enters by way of four independent white-noise inputs representing fluctuations in the four types (e-e, i-e, e-i, i-i) of subcortic alactivity. Our model is a spatial and temporal simplification of the original set of eight coupled partial differential equations (PDEs) due to Liley et al. [Neurocomputing 26-27, 795 (1999)] describing the electrical rhythms of the cortex. We assume (i) spatial homogeneity (i.e., the entire cortex can be represented by a single macrocolumn), and (ii) a separation of temporal scales in which all inputs to the soma “capacitor” are treated as fast variables that settle to steady state very much more rapidly than do the soma voltages themselves: this is the “adiabatic approximation.” These simplifications permit the eight-equation Liley set to be collapsed to a single pair of first-order PDEs in hₑ and hᵢ. We incorporate the effect of general anaesthetic as a lengthening of the duration of the inhibitory post-synaptic potential (PSP) (i.e., we are modelling the GABAergic class of anaesthetics), thus the effectiveness of the inhibitory firings increases monotonically with anaesthetic concentration. These simplified equations of motion for hₑ,ᵢ are transformed into Langevin (stochastic) equations by adding small white-noise fluctuations to each of the four subcortical spike-rate averages. In order to anchor the analysis, I first identify the t → ∞ steady-state values for the soma voltages. This is done by turning off all noise sources and setting the dhₑ/dt and dhᵢ/dt time derivatives to zero, then numerically locating the steady-state coordinates as a function of anaesthetic effect λ, the scale-factor for the lengthening of the inhibitory PSP. We find that, when plotted as a function of λ, the steady-state soma voltages map out a reverse-S trajectory consisting of a pair of stable branches—the upper (active, high-firing) branch, and the lower (quiescent, low-firing) branch—joined by an unstable mid-branch. Because the two stable phases are not contiguous, the model predicts that a transit from one phase to the other must be first-order discontinuous in soma voltage, and that the downward (induction) jump from active-awareness to unconscious-quiescence will be hysteretically separated from (i.e., will occur at a larger concentration of anaesthetic than) the upward (emergence) jump for the return of consciousness. By reenabling the noise terms, then linearizing the Langevin equations about one of the stable steady states, we obtain a two-dimensional Ornstein-Uhlenbeck (Brownian motion) system which can be analyzed using standard results from stochastic calculus. Accordingly, we calculate the covariance, time-correlation, and spectral matrices, and find the interesting predictions of vastly increased EEG fluctuation power, attended by simultaneous redistribution of spectral energy towards low frequencies with divergent increases in fluctuation correlation times (i.e., critical slowing down), as the macrocolumn transition points are approached. These predictions are qualitatively confirmed by clinical measurements reported by Kuizenga et al. [British Journal of Anaesthesia 80, 725 (1998)] of the so-called EEG biphasic effect. He used a slew-rate technique known as aperiodic analysis, and I demonstrate that this is approximately equivalent to a frequency-scaling of the power spectral density. Changes in the frequency distribution of spectral energy can be quantified using the notion of spectral entropy, a modern measure of spectral “whiteness.” We compare the spectral entropy predicted by the model against the clinical values reported recently by Viertiӧ-Oja et al. [Journal of Clinical Monitoring 16, 60 (2000)], and find excellent qualitative agreement for the induction of anaesthesia. To the best of my knowledge, the link between spectral entropy and correlation time has not previously been reported. For the special case of Lorentzian spectrum (arising from a 1-D OU process), I prove that spectral entropy is proportional to the negative logarithm of the correlation time, and uncover the formula which relates the discrete H₁ Shannon information to the continuous H₂ “histogram entropy,” giving an unbiased estimate of the underlying continuous spectral entropy Hω. The inverse entropy-correlation relationship suggests that, to the extent that anaesthetic induction can be modelled as a 1-D OU process, cortical state can be assessed either in the time domain via correlation time or, equivalently, in the frequency domain via spectral entropy. In order to investigate a thermodynamic analogy for the anaesthetic-driven (“anaestheto-dynamic”) phase transition of the cortex, we use the steady-state trajectories as an effective equation of state to uncouple the macrocolumn into a pair of (apparently) independent “pseudocolumns.” The stable steady states may now be pictured as local minima in a landscape of potential hills and valleys. After identifying a plausible temperature analogy, we compute the analogous entropy and predict discontinous entropy change—with attendant “heat capacity” anomalies—at transition. The Stullken dog experiments [Stullken et al., Anesthesiology 46, 28(1977)], measuring cerebral metabolic rate changes, seem to confirm these model predictions. The penultimate chapter examines the impact of incorporating NMDA, an important excitatory neurotransmitter, in the adiabatic model. This work predicts the existence of a new stable state for the cortex, midway between normal activity and quiescence. An induction attempt using a pure anti-NMDA anaesthetic agent (e.g., xenon or nitrous oxide) will take the patient to this mid-state, but no further. I find that for an NMDA-enabled macrocolumn, a GABA induction can produce a second biphasic power event, depending on the brain state at commencement. The latest clinical report from Kuizenga et al. [British Journal of Anaesthesia 86, 354 (2001)] provides apparent confirmation