Structures for Sophisticated Behaviour: Feudal Hierarchies and World Models

This thesis explores structured, reward-based behaviour in artificial agents and in animals. In Part I we investigate how reinforcement learning agents can learn to cooperate. Drawing inspiration from the hierarchical organisation of human societies, we propose the framework of Feudal Multi-agent Hierarchies (FMH), in which coordination of many agents is facilitated by a manager agent. We outline the structure of FMH and demonstrate its potential for decentralised learning and control. We show that, given an adequate set of subgoals from which to choose, FMH performs, and particularly scales, substantially better than cooperative approaches that use shared rewards. We next investigate training FMH in simulation to solve a complex information gathering task. Our approach introduces a ‘Centralised Policy Actor-Critic’ (CPAC) and an alteration to the conventional multi-agent policy gradient, which allows one multi-agent system to advise the training of another. We further exploit this idea for communicating agents with shared rewards and demonstrate its efficacy. In Part II we examine how animals discover and exploit underlying statistical structure in their environments, even when such structure is difficult to learn and use. By analysing behavioural data from an extended experiment with rats, we show that such hidden structure can indeed be learned, but also that subjects suffer from imperfections in their ability to infer their current state. We account for their behaviour using a Hidden Markov Model, in which recent observations are integrated imperfectly with evidence from the past. We find that over the course of training, subjects learn to track their progress through the task more accurately, a change that our model largely attributes to the more reliable integration of past evidenc

Individual topological analysis of synchronization-based brain connectivity

Functional connectivity analysis aims at assessing the strength of functional coupling between the signal responses in distinct brain areas. Usually, functional magnetic resonance imaging (fMRI) time series connections are estimated through zero-lag correlation metrics that quantify the statistical similarity between pairs of regions or spectral measures that assess synchronization at a frequency band of interest. Here, we explored the application of a new metric to assess the functional synchronization in phase space between fMRI time series in a resting state. We applied a complete topological analysis to the resulting connectivity matrix to uncover both the macro-scale organization of the brain and detect the most important nodes. The synchronization metric is also compared with Pearson's correlation coefficient and spectral coherence to highlight similarities and differences between the topologies of the three functional networks. We found that the individual topological organization of the resulting synchronization-based connectivity networks shows a finer modular organization than that identified with the other two metrics and a low overlap with the modular partitions of the other two networks suggesting that the derived topological information is not redundant and could be potentially integrated to provide a multi-scale description of functional connectivity

Characteristics of tropical cyclones in the North Atlantic and East Pacific.

In this dissertation, I present a series of investigations to expand our understanding of TCs in the East Pacific and North Atlantic basins. First, I developed and applied a climatological tool that quickly and succinctly displays the spread of historical TC tracks for any point in the North Atlantic basin. This tool is useful in all parts of a basin because it is derived from prior storm motion trajectories and summarily captures the historical synoptic and mesoscale steering patterns. It displays the strength of the climatological signal and allow for rapid qualitative comparison between historical TC tracks and NWP models. Second, I have used a robust statistical technique to quantify the relationships between fifteen different metrics of TC activity in nine ocean basins and twelve climate indices of the leading modes of atmospheric and oceanic variability. In a thorough, encyclopedic manner, over 12,000 Spearman rank correlation coefficients were calculated and examined to identify relationships between TCs and their environment. This investigation was not limited to the East Pacific or North Atlantic, and new climatic associations were found between seasonal levels of TC activity and the major climate indices across the nine basins. This information is critical to forecasters, economists, actuaries, energy traders, and societal planners who apply knowledge of levels of TC activity on intraseasonal to interdecadal timescales. The statistics are also valuable to climatologists seeking to understand how regional TC frequency will change as the global climate warms. Third, I have examined the leading intraseasonal mode of atmospheric and oceanic variability, the Madden-Julian Oscillation (MJO), and discovered statistically significant relationships with the frequency of TC genesis, intensification, and landfall over the nine basins. Like the significance of the longer-period oscillations to the frequency of TC activity on intraseasonal and longer timescales, these results are highly relevant to the problem of short-term (one- to two-week) predictability of TC activity. These three investigations demonstrate the utility of historical datasets across a wide range of applications, from short-term forecasting to climate studies. In this way, the results highlighted in this dissertation represent a significant and positive contribution to meteorology. Collectively, they reveal multiple characteristics of TCs in the East Pacific and North Atlantic and provide greater understanding of the complex interactions between TCs and their surrounding larger-scale environment

Sport action zones: empowering local communities? The regional manifestation of a national initiative

In response to the Social Exclusion Unit's Policy Action Team 10 (1999), Sport Action Zones (SAZs) were launched in 2000 as Sport England's attempt to create an effective and sustainable sporting infrastructure in areas of high economic and social deprivation, ensuring more equitable participation in sport (Sport England, 2003a). Such an initiative was supported by the firm belief that although sport and physical activity could not in themselves prevent or eradicate social exclusion (Pitter and Andrews, 1997), they were one potential means of positively affecting health, education and crime (PAT 10, 1999; Farrell and Shields, 2002; Sandford et al., 2006). It is essential to adopt a holistic approach that concentrates on those problems experienced by individuals engaged in a physical activity intervention (the context), rather than the intervention itself. This is a consequence of the fact that any quest to abolish dimensions of social exclusion will involve a plethora of complex associations and factors which create a myriad of outcomes (Coalter et al., 2000). Using hybrid grounded theory as the theoretical underpinning, this study investigated over three years (2002-2005) the implementation of four projects within the Wear Valley SAZ in the north-east of England: Walking the Way to Health; Community Physical Activity Coordinators; Positive Futures; Modem Apprentices. Adopting an internal relative ontology and a subjective epistemology, case study methodology was employed in conjunction with participant observation, questionnaires, focus groups and interviews. Key themes emerged from the qualitative analysis of data, facilitating a deeper understanding of the strengths and weaknesses of the Sport Action Zone, and of the individuals who engaged with it. The Wear Valley SAZ was seen to positively impact upon health, education, and crime. Nevertheless, the extent of this contribution to social inclusion was limited as it failed to adequately engage with those at greatest risk of social exclusion. Sport Action Zones have the potential to contribute significantly to social inclusion in areas of high socioeconomic deprivation, but this impact will be determined by the extent of consideration given to both the context within which the intervention is to exist and the potential mechanisms responsible for change. Moreover, to be successful, any such intervention must (at the design stage) be cognisant of the interrelationships between: physical activity determinants, notions of empowerment, attitude towards physical activity, and sustainability of participation. These are fundamental intervention precursors required to maximise those positive impacts upon health, education, and crime. Furthermore, as the complexity of the multiple risk factors associated with social exclusion inevitably lead to delays in affecting significant change, allowing time to address individual risk factors will increase the probability of creating a more effective and sustainable sporting infrastructure, particularly in areas of high socioeconomic deprivation

Modelling Bacterial Growth for Applying Photodynamic Therapy with Indocyanine Green

Antimicrobial approaches using photodynamic therapy (PDT) have become popular in medication. However, to the best of our knowledge, no model has been developed for estimating the dose of light and photosensitiser with respect to bacterial inhibition. This thesis aims to model the growth of Escherichia coli which can be utilised in developing the aforementioned model when using PDT with near-infrared (NIR) light and indocyanine green (ICG). The project applied a spectroscopic method to measure the spectra of bacteria and chemometric methods to analyse the spectral data. The project consists of two main phases. The first phase conducted two phantoms to develop a measurement system and identify the possibility of utilising the system in controlled conditions. The first phantom analysed the spectra of LED lights. The second phantom determined the concentrations of different colour liquids. In the second phase, bacterial suspension was subjected to spectral analysis using the developed system. As a result of the whole project, a model has been established in order to monitor and estimate the concentrations of bacteria in liquid samples

Campus Planners\u27 Preferences For Residence Hall Project Delivery Methods

This research determined the preferred project delivery method utilized by campus planners when building student housing on a university campus. Four key performance indicators were also evaluated to determine if they influenced the campus planner to select a particular project delivery method. The four key performance indicators were: owner input, cost, safety, and on-time completion. Campus planners identified though the Society for Campus and University Planners (SCUP), were invited to participate in this survey research method. Five research questions were explored in this study. They were: 1. What is the preferred project delivery method (PDM) when building a residence hall on a college campus? 2. Do key performance indicators influence a campus planner’s choice of preferred project delivery method? 3. Does the preferred project delivery method differ by the number of beds? 4. Does the preferred project delivery method differ geographically? 5. Does the preferred project delivery method differ between public and private institutions? Based on the survey findings, analyzed using SPSS, sufficient information was garnered from the data to allow responses to the five research questions. In sum, there was a significant preference on the part of campus planners for the construction management at risk project delivery method when building student housing. This finding was consistent across regions and between public and private institutions. The design-bid-build project delivery method was the preferred approach when building student housing of less than 200 beds. Key performance indicators were shown to have little influence on the decision regarding which project delivery method to choose

The importance of institutional differences among countries in SDGs achievement: A cross‐country empirical study

Based upon the “quintuple helix” model (Carayannis et al., 2012; Carayannis & Campbell, 2010), this research analyzes whether the differences in the level of achievement of the Sustainable Development Goals (SDGs) among 64 countries worldwide can be explained by a series of 19 institutional variables related to five dimensions such as cultural orientation, economic development, the education-labor system, the political-legal system and innovation. Our results highlight the crucial role of public policies in attaining SDGs through the improvement of institutional quality, governance systems and economic freedom as well as with the promotion of education and the innovation ecosystem. In this regard, the adoption of the “quintuple helix” model through the interrelation of the government, Universities and the private sector is deemed relevant in order to face the social and environmental challenges posed by the SDGs.We thank to the Ministry of Science and Innovation, project GELESMAT (ref. PID2021-122419OB-I00) for financial support