12,769 research outputs found

    Policy Space Diversity for Non-Transitive Games

    Full text link
    Policy-Space Response Oracles (PSRO) is an influential algorithm framework for approximating a Nash Equilibrium (NE) in multi-agent non-transitive games. Many previous studies have been trying to promote policy diversity in PSRO. A major weakness in existing diversity metrics is that a more diverse (according to their diversity metrics) population does not necessarily mean (as we proved in the paper) a better approximation to a NE. To alleviate this problem, we propose a new diversity metric, the improvement of which guarantees a better approximation to a NE. Meanwhile, we develop a practical and well-justified method to optimize our diversity metric using only state-action samples. By incorporating our diversity regularization into the best response solving in PSRO, we obtain a new PSRO variant, Policy Space Diversity PSRO (PSD-PSRO). We present the convergence property of PSD-PSRO. Empirically, extensive experiments on various games demonstrate that PSD-PSRO is more effective in producing significantly less exploitable policies than state-of-the-art PSRO variants

    An Investigation into the Transition Experiences of A-Level Students and Staff in a Further Education College

    Get PDF
    Post-16 transition is an inevitability in the life of every student. Yet post-16 educational transition in England has to date been under-researched. The age of participation in education in the United Kingdom was raised to 18 years in 2015, which along with concurrent reductions in funding have had ramifications for Post-16 transition provision and experiences. Transition experience impacts on students’ future engagement with education and long-term economic wellbeing. The present research investigated the transition experience of four first year A-Level students in a Further Education college, and five members of staff. Data were collected from all participants, using Interpretative Phenomenological Analysis (IPA), via semistructured interviews in the first term. Data were further collected from the student participants in the second term, using semi-structured interviews and a form of photo elicitation. From the analysis, three super-ordinate themes were highlighted from the student participants: Relationships and Interactions, Influences which Impact on Transition and Transition in Learning. The three super-ordinate themes from the staff participants were: Teacher practices, Management and Transition Provision. The implications from the findings indicate that having a longer-term and well considered plan towards Higher Education and career pathways can enhance the transition experience of all students, particularly those who find the process confusing or difficult. The overarching conclusion is that transition can be a challenging process which requires greater attention from strategic leadership and the inclusion of both staff and students in the decision-making process

    Reinforcement learning in large state action spaces

    Get PDF
    Reinforcement learning (RL) is a promising framework for training intelligent agents which learn to optimize long term utility by directly interacting with the environment. Creating RL methods which scale to large state-action spaces is a critical problem towards ensuring real world deployment of RL systems. However, several challenges limit the applicability of RL to large scale settings. These include difficulties with exploration, low sample efficiency, computational intractability, task constraints like decentralization and lack of guarantees about important properties like performance, generalization and robustness in potentially unseen scenarios. This thesis is motivated towards bridging the aforementioned gap. We propose several principled algorithms and frameworks for studying and addressing the above challenges RL. The proposed methods cover a wide range of RL settings (single and multi-agent systems (MAS) with all the variations in the latter, prediction and control, model-based and model-free methods, value-based and policy-based methods). In this work we propose the first results on several different problems: e.g. tensorization of the Bellman equation which allows exponential sample efficiency gains (Chapter 4), provable suboptimality arising from structural constraints in MAS(Chapter 3), combinatorial generalization results in cooperative MAS(Chapter 5), generalization results on observation shifts(Chapter 7), learning deterministic policies in a probabilistic RL framework(Chapter 6). Our algorithms exhibit provably enhanced performance and sample efficiency along with better scalability. Additionally, we also shed light on generalization aspects of the agents under different frameworks. These properties have been been driven by the use of several advanced tools (e.g. statistical machine learning, state abstraction, variational inference, tensor theory). In summary, the contributions in this thesis significantly advance progress towards making RL agents ready for large scale, real world applications

    Unstable Periodic Orbits: a language to interpret the complexity of chaotic systems

    Get PDF
    Unstable periodic orbits (UPOs), exact periodic solutions of the evolution equation, offer a very powerful framework for studying chaotic dynamical systems, as they allow one to dissect their dynamical structure. UPOs can be considered the skeleton of chaotic dynamics, its essential building blocks. In fact, it is possible to prove that in a chaotic system, UPOs are dense in the attractor, meaning that it is always possible to find a UPO arbitrarily near any chaotic trajectory. We can thus think of the chaotic trajectory as being approximated by different UPOs as it evolves in time, jumping from one UPO to another as a result of their instability. In this thesis we provide a contribution towards the use of UPOs as a tool to understand and distill the dynamical structure of chaotic dynamical systems. We will focus on two models, characterised by different properties, the Lorenz-63 and Lorenz-96 model. The process of approximation of a chaotic trajectory in terms of UPOs will play a central role in our investigation. In fact, we will use this tool to explore the properties of the attractor of the system under the lens of its UPOs. In the first part of the thesis we consider the Lorenz-63 model with the classic parameters’ value. We investigate how a chaotic trajectory can be approximated using a complete set of UPOs up to symbolic dynamics’ period 14. At each instant in time, we rank the UPOs according to their proximity to the position of the orbit in the phase space. We study this process from two different perspectives. First, we find that longer period UPOs overwhelmingly provide the best local approximation to the trajectory. Second, we construct a finite-state Markov chain by studying the scattering of the trajectory between the neighbourhood of the various UPOs. Each UPO and its neighbourhood are taken as a possible state of the system. Through the analysis of the subdominant eigenvectors of the corresponding stochastic matrix we provide a different interpretation of the mixing processes occurring in the system by taking advantage of the concept of quasi-invariant sets. In the second part of the thesis we provide an extensive numerical investigation of the variability of the dynamical properties across the attractor of the much studied Lorenz ’96 dynamical system. By combining the Lyapunov analysis of the tangent space with the study of the shadowing of the chaotic trajectory performed by a very large set of unstable periodic orbits, we show that the observed variability in the number of unstable dimensions, which shows a serious breakdown of hyperbolicity, is associated with the presence of a substantial number of finite-time Lyapunov exponents that fluctuate about zero also when very long averaging times are considered

    Aspect-Driven Structuring of Historical Dutch Newspaper Archives

    Full text link
    Digital libraries oftentimes provide access to historical newspaper archives via keyword-based search. Historical figures and their roles are particularly interesting cognitive access points in historical research. Structuring and clustering news articles would allow more sophisticated access for users to explore such information. However, real-world limitations such as the lack of training data, licensing restrictions and non-English text with OCR errors make the composition of such a system difficult and cost-intensive in practice. In this work we tackle these issues with the showcase of the National Library of the Netherlands by introducing a role-based interface that structures news articles on historical persons. In-depth, component-wise evaluations and interviews with domain experts highlighted our prototype's effectiveness and appropriateness for a real-world digital library collection.Comment: TPDL2023, Full Paper, 16 page

    Chaos persists in large-scale multi-agent learning despite adaptive learning rates

    Full text link
    Multi-agent learning is intrinsically harder, more unstable and unpredictable than single agent optimization. For this reason, numerous specialized heuristics and techniques have been designed towards the goal of achieving convergence to equilibria in self-play. One such celebrated approach is the use of dynamically adaptive learning rates. Although such techniques are known to allow for improved convergence guarantees in small games, it has been much harder to analyze them in more relevant settings with large populations of agents. These settings are particularly hard as recent work has established that learning with fixed rates will become chaotic given large enough populations.In this work, we show that chaos persists in large population congestion games despite using adaptive learning rates even for the ubiquitous Multiplicative Weight Updates algorithm, even in the presence of only two strategies. At a technical level, due to the non-autonomous nature of the system, our approach goes beyond conventional period-three techniques Li-Yorke by studying fundamental properties of the dynamics including invariant sets, volume expansion and turbulent sets. We complement our theoretical insights with experiments showcasing that slight variations to system parameters lead to a wide variety of unpredictable behaviors.Comment: 30 pages, 6 figure

    The home and mental health: an exploration of perceptions, connections and attachments to social housing for residents and housing professionals in Manchester, United Kingdom

    Get PDF
    This doctoral study is an exploration of the home environment, mental health, and the role of housing associations in supporting resident welfare. The home is significant, having the ability to provide protection and privacy, a secure base to develop relationships, build identity and feel safe from the outside world. However, many individuals are unable to experience these protective factors within their home setting for multiple reasons. This thesis is framed within the context of the Devolution Agenda in Manchester, where there has been an emphasis on the integration of health and social care services (such as social housing providers) in efforts to tackle mental ill health and persistent health inequalities. This thesis undertook a multi-method approach that consists of a qualitative analysis of two focus groups with a housing provider and seven semi-structured interviews with residents within Gorton, an area located North of Manchester, UK. The research was conducted in two phases to capture the perceptions and experiences of socially housed residents, alongside investigating the understandings and professional realities of housing providers. My research is informed by a critical realist approach, where I apply concepts such as ontological security, place, social capital, and stigma, to make sense of interactions between the home and mental health. The findings in this thesis highlight how the home environment can interact with mental health outcomes through various psychological, social, and environmental dimensions. Being able to establish and maintain a sense of home was important for residents, and this provided a source of security, self-esteem, and autonomy. Residents demonstrated complex articulations of place, where the importance of social networks, local value systems and a sense of belonging all reflected the deep connections and meanings attached to the home. Physical deterioration, the closure of local amenities and fragmented relationships with the service provider undermined the mental wellbeing of residents. The findings from this thesis identified a disconnect between how residents experienced their home and community and how the housing provider interprets residents support needs and the delegation of resources. In this thesis I highlight how integrated health approaches require further consideration to ensure services are needs-sensitive and reflective of residents’ priorities. Through bringing together the perspectives of residents and housing professionals, this research makes a unique contribution to the integration of health approaches, the less tangible aspects of the home setting, and mental wellbeing

    Knowledge Graph Building Blocks: An easy-to-use Framework for developing FAIREr Knowledge Graphs

    Full text link
    Knowledge graphs and ontologies provide promising technical solutions for implementing the FAIR Principles for Findable, Accessible, Interoperable, and Reusable data and metadata. However, they also come with their own challenges. Nine such challenges are discussed and associated with the criterion of cognitive interoperability and specific FAIREr principles (FAIR + Explorability raised) that they fail to meet. We introduce an easy-to-use, open source knowledge graph framework that is based on knowledge graph building blocks (KGBBs). KGBBs are small information modules for knowledge-processing, each based on a specific type of semantic unit. By interrelating several KGBBs, one can specify a KGBB-driven FAIREr knowledge graph. Besides implementing semantic units, the KGBB Framework clearly distinguishes and decouples an internal in-memory data model from data storage, data display, and data access/export models. We argue that this decoupling is essential for solving many problems of knowledge management systems. We discuss the architecture of the KGBB Framework as we envision it, comprising (i) an openly accessible KGBB-Repository for different types of KGBBs, (ii) a KGBB-Engine for managing and operating FAIREr knowledge graphs (including automatic provenance tracking, editing changelog, and versioning of semantic units); (iii) a repository for KGBB-Functions; (iv) a low-code KGBB-Editor with which domain experts can create new KGBBs and specify their own FAIREr knowledge graph without having to think about semantic modelling. We conclude with discussing the nine challenges and how the KGBB Framework provides solutions for the issues they raise. While most of what we discuss here is entirely conceptual, we can point to two prototypes that demonstrate the principle feasibility of using semantic units and KGBBs to manage and structure knowledge graphs

    Characterising Modal Formulas with Examples

    Full text link
    We study the existence of finite characterisations for modal formulas. A finite characterisation of a modal formula φ\varphi is a finite collection of positive and negative examples that distinguishes φ\varphi from every other, non-equivalent modal formula, where an example is a finite pointed Kripke structure. This definition can be restricted to specific frame classes and to fragments of the modal language: a modal fragment LL admits finite characterisations with respect to a frame class FF if every formula φ∈L\varphi\in L has a finite characterisation with respect to LL consting of examples that are based on frames in FF. Finite characterisations are useful for illustration, interactive specification, and debugging of formal specifications, and their existence is a precondition for exact learnability with membership queries. We show that the full modal language admits finite characterisations with respect to a frame class FF only when the modal logic of FF is locally tabular. We then study which modal fragments, freely generated by some set of connectives, admit finite characterisations. Our main result is that the positive modal language without the truth-constants ⊤\top and ⊥\bot admits finite characterisations w.r.t. the class of all frames. This result is essentially optimal: finite characterizability fails when the language is extended with the truth constant ⊥\bot or with all but very limited forms of negation.Comment: Expanded version of material from Raoul Koudijs's MSc thesis (2022
    • …
    corecore