Implementation of gaussian process models for non-linear system identification

This thesis is concerned with investigating the use of Gaussian Process (GP) models for the identification of nonlinear dynamic systems. The Gaussian Process model is a non-parametric approach to system identification where the model of the underlying system is to be identified through the application of Bayesian analysis to empirical data. The GP modelling approach has been proposed as an alternative to more conventional methods of system identification due to a number of attractive features. In particular, the Bayesian probabilistic framework employed by the GP model has been shown to have potential in tackling the problems found in the optimisation of complex nonlinear models such as those based on multiple model or neural network structures. Furthermore, due to this probabilistic framework, the predictions made by the GP model are probability distributions composed of mean and variance components. This is in contrast to more conventional methods where a predictive point estimate is typically the output of the model. This additional variance component of the model output has been shown to be of potential use in model-predictive or adaptive control implementations. A further property that is of potential interest to those working on system identification problems is that the GP model has been shown to be particularly effective in identifying models from sparse datasets. Therefore, the GP model has been proposed for the identification of models in off-equilibrium regions of operating space, where more established methods might struggle due to a lack of data. The majority of the existing research into modelling with GPs has concentrated on detailing the mathematical methodology and theoretical possibilities of the approach. Furthermore, much of this research has focused on the application of the method toward statistics and machine learning problems. This thesis investigates the use of the GP model for identifying nonlinear dynamic systems from an engineering perspective. In particular, it is the implementation aspects of the GP model that are the main focus of this work. Due to its non-parametric nature, the GP model may also be considered a ‘black-box’ method as the identification process relies almost exclusively on empirical data, and not on prior knowledge of the system. As a result, the methods used to collect and process this data are of great importance, and the experimental design and data pre-processing aspects of the system identification procedure are investigated in detail. Therefore, in the research presented here the inclusion of prior system knowledge into the overall modelling procedure is shown to be an invaluable asset in improving the overall performance of the GP model. In previous research, the computational implementation of the GP modelling approach has been shown to become problematic for applications where the size of training dataset is large (i.e. one thousand or more points). This is due to the requirement in the GP modelling approach for repeated inversion of a covariance matrix whose size is dictated by the number of points included in the training dataset. Therefore, in order to maintain the computational viability of the approach, a number of different strategies have been proposed to lessen the computational burden. Many of these methods seek to make the covariance matrix sparse through the selection of a subset of existing training data. However, instead of operating on an existing training dataset, in this thesis an alternative approach is proposed where the training dataset is specifically designed to be as small as possible whilst still containing as much information. In order to achieve this goal of improving the ‘efficiency’ of the training dataset, the basis of the experimental design involves adopting a more deterministic approach to exciting the system, rather than the more common random excitation approach used for the identification of black-box models. This strategy is made possible through the active use of prior knowledge of the system. The implementation of the GP modelling approach has been demonstrated on a range of simulated and real-world examples. The simulated examples investigated include both static and dynamic systems. The GP model is then applied to two laboratory-scale nonlinear systems: a Coupled Tanks system where the volume of liquid in the second tank must be predicted, and a Heat Transfer system where the temperature of the airflow along a tube must be predicted. Further extensions to the GP model are also investigated including the propagation of uncertainty from one prediction to the next, the application of sparse matrix methods, and also the use of derivative observations. A feature of the application of GP modelling approach to nonlinear system identification problems is the reliance on the squared exponential covariance function. In this thesis the benefits and limitations of this particular covariance function are made clear, and the use of alternative covariance functions and ‘mixed-model’ implementations is also discussed

Downscaling ocean conditions with application to the Gulf of Maine, Scotian Shelf and adjacent deep ocean

The overall goal is to downscale ocean conditions predicted by an existing global prediction system and evaluate the results using observations from the Gulf of Maine, Scotian Shelf and adjacent deep ocean. The first step is to develop a one-way nested regional model and evaluate its predictions using observations from multiple sources including satellite-borne sensors of surface temperature and sea level, CTDs, Argo floats and moored current meters. It is shown that the regional model predicts more realistic fields than the global system on the shelf because it has higher resolution and includes tides that are absent from the global system. However, in deep water the regional model misplaces deep ocean eddies and meanders associated with the Gulf Stream. This is not because the regional model’s dynamics are flawed but rather is the result of internally generated variability in deep water that leads to decoupling of the regional model from the global system. To overcome this problem, the next step is to spectrally nudge the regional model to the large scales (length scales > 90 km) of the global system. It is shown this leads to more realistic predictions off the shelf. Wavenumber spectra show that even though spectral nudging constrains the large scales, it does not suppress the variability on small scales; on the contrary, it favours the formation of eddies with length scales below the cutoff wavelength of the spectral nudging

Context-Dependent Memory under Stressful Conditions: The Case of Skydiving

Two experiments examined the effect of differing levels of emotional arousal on learning and memory for words in matching and mismatching contexts. In Experiment 1, experienced skydivers learned words either in the air or on the ground and recalled them in the same context or in the other context. Experiment 2 replicated the stimuli and design of the first experiment except that participants were shown a skydiving video in lieu of skydiving. Recall was poor in air-learning conditions with actual skydiving, but when lists were learned on land, recall was higher in the matching context than in the mismatching context. In the skydiving video experiment, recall was higher in matching learn-recall contexts regardless of the situation in which learning occurred. We propose that under extremely emotionally arousing circumstances, environmental and/or mood cues are unlikely to become encoded or linked to newly acquired information and thus cannot serve as cues to retrieval. Results can be applied to understanding variations in context-dependent memory in occupations (e.g., police, military special operations, and Special Weapons and Tactics teams) in which the worker experiences considerable emotional stress while learning or recalling new information

Probing of RNA structures in a positive sense RNA virus reveals selection pressures for structural elements.

In single stranded (+)-sense RNA viruses, RNA structural elements (SEs) play essential roles in the infection process from replication to encapsidation. Using selective 2'-hydroxyl acylation analyzed by primer extension sequencing (SHAPE-Seq) and covariation analysis, we explore the structural features of the third genome segment of cucumber mosaic virus (CMV), RNA3 (2216 nt), both in vitro and in plant cell lysates. Comparing SHAPE-Seq and covariation analysis results revealed multiple SEs in the coat protein open reading frame and 3' untranslated region. Four of these SEs were mutated and serially passaged in Nicotiana tabacum plants to identify biologically selected changes to the original mutated sequences. After passaging, loop mutants showed partial reversion to their wild-type sequence and SEs that were structurally disrupted by mutations were restored to wild-type-like structures via synonymous mutations in planta. These results support the existence and selection of virus open reading frame SEs in the host organism and provide a framework for further studies on the role of RNA structure in viral infection. Additionally, this work demonstrates the applicability of high-throughput chemical probing in plant cell lysates and presents a new method for calculating SHAPE reactivities from overlapping reverse transcriptase priming sites

A novel approach to bivariate meta-analysis of binary outcomes and its application in the context of surrogate endpoints

Bivariate meta-analysis provides a useful framework for combining information across related studies and has been widely utilised to combine evidence from clinical studies in order to evaluate treatment efficacy. Bivariate meta-analysis has also been used to investigate surrogacy patterns between treatment effects on the surrogate and the final outcome. Surrogate endpoints play an important role in drug development when they can be used to measure treatment effect early compared to the final clinical outcome and to predict clinical benefit or harm. The standard bivariate meta-analytic approach models the observed treatment effects on the surrogate and final outcomes jointly, at both the within-study and between-studies levels, using a bivariate normal distribution. For binomial data a normal approximation can be used on log odds ratio scale, however, this method may lead to biased results when the proportions of events are close to one or zero, affecting the validation of surrogate endpoints. In this paper, two Bayesian meta-analytic approaches are introduced which allow for modelling the within-study variability using binomial data directly. The first uses independent binomial likelihoods to model the within-study variability avoiding to approximate the observed treatment effects, however, ignores the within-study association. The second, models the summarised events in each arm jointly using a bivariate copula with binomial marginals. This allows the model to take into account the within-study association through the copula dependence parameter. We applied the methods to an illustrative example in chronic myeloid leukemia to investigate the surrogate relationship between complete cytogenetic response (CCyR) and event-free-survival (EFS).Comment: 20 pages, 6 figure

Seasonal Variation of the North Atlantic Current

The seasonal circulation of the upper 1000 m of the North Atlantic between 40 degrees -55 degreesN and 20 degrees -40 degreesW is calculated using the traditional dynamic method and a circulation model with a density field that evolves with the flow. The model is of finite difference form and is based on dynamics that describe the nonlinear evolution of the ocean at low Rossby number. The model is controlled by initial and boundary conditions that include air-sea buoyancy and momentum fluxes. The model is run in two ways: with controls specified directly from observations and with controls inferred by the assimilation of all available data. These data include surface drifter trajectories, sea levels from the TOPEX/Poseidon altimeter, Bunker air-sea fluxes, and the Levitus climatological monthly means of temperature and salinity. We conclude that the North Atlantic Current transport is 40 +/- 18 Sv with seasonal variations of the order of 2 Sv. The mean vertical transport out of the region is 2 +/- 9 Sv and is subject to seasonal variations of 2 Sv. Overall, these estimates are in good agreement with integral North Atlantic Current features derived from independent long-term measurements made in the region over the past decade. The optimal ocean state has a volume transport across the western boundary of 51 +/- 3 Sv with a maximum transport of 61 +/- 5 Sv in April-May and a minimum of 42 +/- 3 Sv in October-November, This western inflow is compensated by mean outflows of 28 +/- 2 (east), 16 +/- 2 (north), 5 +/- 2 (south), and 1.8 +/- 0.4 Sv out of the domain at 1000 m. Sensitivity studies show that nonlinear mixing and seasonality are important in determining the overall circulation. Specifically, steady boundary forcing leads to annual mean transports that are 15-25% smaller than transports obtained with seasonal forcing. Winter convection is also shown to play a significant role in determining the overall circulation pattern

Prefrontal Cortical Response to Conflict during Semantic and Phonological Tasks

Debates about the function of the prefrontal cortex are as old as the field of neuropsychology—often dated to Paul Broca’s seminal work. Theories of the functional organization of the prefrontal cortex can be roughly divided into those that describe organization by process and those that describe organization by material. Recent studies of the function of the posterior, left inferior frontal gyrus (pLIFG) have yielded two quite different interpretations: One hypothesis holds that the pLIFG plays a domain-specific role in phonological processing, whereas another hypothesis describes a more general function of the pLIFG in cognitive control. In the current study, we distinguish effects of increasing cognitive control demands from effects of phonological processing. The results support the hypothesized role for the pLIFG in cognitive control, and more task-specific roles for posterior areas in phonology and semantics. Thus, these results suggest an alternative explanation of previously reported phonology-specific effects in the pLIFG

Interaction between the tidal and seasonal variability of the Gulf of Maine and Scotian Shelf Region

As part of a broader study of ocean downscaling, the seasonal and tidal variability of the Gulf of Maine and Scotian shelf, and their dynamical interaction, are investigated using a high-resolution (1/36°) circulation model. The model’s seasonal hydrography and circulation, and its tidal elevations and currents, are compared with an observed seasonal climatology, local observations, and results from previous studies. Numerical experiments with and without density stratification demonstrate the influence of stratification on the tides. The model is then used to interpret the physical mechanisms responsible for the largest seasonal variations in the M2 surface current that occur over, and to the north of, Georges Bank. The model generates a striation pattern of alternating highs and lows, aligned with Georges Bank, in the M2 surface summer maximum speed in the Gulf of Maine. The striations are consistent with observations by a high-frequency coastal radar system and can be explained in terms of a linear superposition of the barotropic tide and the first-mode baroclinic tide, generated on the north side of Georges Bank, as it propagates into the Gulf of Maine. The seasonal changes in tidal currents in the well-mixed area on Georges Bank are due to a combination of increased sea level gradients, and lower vertical viscosity, in summer

Mapping Hierarchical Structures with Synchrony for Binding: Preliminary Investigations

Synchrony of firing has recently become a popular technique for dynamic binding in neural networks, and has been applied to numerous problem domains. However, hierarchical structures are difficult to represent using synchrony for binding. This paper presents our progress toward a framework for representing hierarchies in a neural network using synchrony for dynamic binding. We illustrate the approach with a model of analogical mapping. The model (IMM2) uses synchrony to bind case roles to objects within propositions. Hierarchies are established by allowing units representing propositions to play a dual role, acting both as the argument of one proposition and as a pointer to another