9,822 research outputs found

    Reinforcement learning in large state action spaces

    Get PDF
    Reinforcement learning (RL) is a promising framework for training intelligent agents which learn to optimize long term utility by directly interacting with the environment. Creating RL methods which scale to large state-action spaces is a critical problem towards ensuring real world deployment of RL systems. However, several challenges limit the applicability of RL to large scale settings. These include difficulties with exploration, low sample efficiency, computational intractability, task constraints like decentralization and lack of guarantees about important properties like performance, generalization and robustness in potentially unseen scenarios. This thesis is motivated towards bridging the aforementioned gap. We propose several principled algorithms and frameworks for studying and addressing the above challenges RL. The proposed methods cover a wide range of RL settings (single and multi-agent systems (MAS) with all the variations in the latter, prediction and control, model-based and model-free methods, value-based and policy-based methods). In this work we propose the first results on several different problems: e.g. tensorization of the Bellman equation which allows exponential sample efficiency gains (Chapter 4), provable suboptimality arising from structural constraints in MAS(Chapter 3), combinatorial generalization results in cooperative MAS(Chapter 5), generalization results on observation shifts(Chapter 7), learning deterministic policies in a probabilistic RL framework(Chapter 6). Our algorithms exhibit provably enhanced performance and sample efficiency along with better scalability. Additionally, we also shed light on generalization aspects of the agents under different frameworks. These properties have been been driven by the use of several advanced tools (e.g. statistical machine learning, state abstraction, variational inference, tensor theory). In summary, the contributions in this thesis significantly advance progress towards making RL agents ready for large scale, real world applications

    ON EXPRESSIVENESS, INFERENCE, AND PARAMETER ESTIMATION OF DISCRETE SEQUENCE MODELS

    Get PDF
    Huge neural autoregressive sequence models have achieved impressive performance across different applications, such as NLP, reinforcement learning, and bioinformatics. However, some lingering problems (e.g., consistency and coherency of generated texts) continue to exist, regardless of the parameter count. In the first part of this thesis, we chart a taxonomy of the expressiveness of various sequence model families (Ch 3). In particular, we put forth complexity-theoretic proofs that string latent-variable sequence models are strictly more expressive than energy-based sequence models, which in turn are more expressive than autoregressive sequence models. Based on these findings, we introduce residual energy-based sequence models, a family of energy-based sequence models (Ch 4) whose sequence weights can be evaluated efficiently, and also perform competitively against autoregressive models. However, we show how unrestricted energy-based sequence models can suffer from uncomputability; and how such a problem is generally unfixable without knowledge of the true sequence distribution (Ch 5). In the second part of the thesis, we study practical sequence model families and algorithms based on theoretical findings in the first part of the thesis. We introduce neural particle smoothing (Ch 6), a family of approximate sampling methods that work with conditional latent variable models. We also introduce neural finite-state transducers (Ch 7), which extend weighted finite state transducers with the introduction of mark strings, allowing scoring transduction paths in a finite state transducer with a neural network. Finally, we propose neural regular expressions (Ch 8), a family of neural sequence models that are easy to engineer, allowing a user to design flexible weighted relations using Marked FSTs, and combine these weighted relations together with various operations

    Patients with Alzheimer's disease dementia show partially preserved parietal 'hubs' modeled from resting-state alpha electroencephalographic rhythms

    Get PDF
    IntroductionGraph theory models a network by its nodes (the fundamental unit by which graphs are formed) and connections. 'Degree' hubs reflect node centrality (the connection rate), while 'connector' hubs are those linked to several clusters of nodes (mainly long-range connections). MethodsHere, we compared hubs modeled from measures of interdependencies of between-electrode resting-state eyes-closed electroencephalography (rsEEG) rhythms in normal elderly (Nold) and Alzheimer's disease dementia (ADD) participants. At least 5 min of rsEEG was recorded and analyzed. As ADD is considered a 'network disease' and is typically associated with abnormal rsEEG delta (<4 Hz) and alpha rhythms (8-12 Hz) over associative posterior areas, we tested the hypothesis of abnormal posterior hubs from measures of interdependencies of rsEEG rhythms from delta to gamma bands (2-40 Hz) using eLORETA bivariate and multivariate-directional techniques in ADD participants versus Nold participants. Three different definitions of 'connector' hub were used. ResultsConvergent results showed that in both the Nold and ADD groups there were significant parietal 'degree' and 'connector' hubs derived from alpha rhythms. These hubs had a prominent outward 'directionality' in the two groups, but that 'directionality' was lower in ADD participants than in Nold participants. DiscussionIn conclusion, independent methodologies and hub definitions suggest that ADD patients may be characterized by low outward 'directionality' of partially preserved parietal 'degree' and 'connector' hubs derived from rsEEG alpha rhythms

    CITIES: Energetic Efficiency, Sustainability; Infrastructures, Energy and the Environment; Mobility and IoT; Governance and Citizenship

    Get PDF
    This book collects important contributions on smart cities. This book was created in collaboration with the ICSC-CITIES2020, held in San José (Costa Rica) in 2020. This book collects articles on: energetic efficiency and sustainability; infrastructures, energy and the environment; mobility and IoT; governance and citizenship

    Bibliographic Control in the Digital Ecosystem

    Get PDF
    With the contributions of international experts, the book aims to explore the new boundaries of universal bibliographic control. Bibliographic control is radically changing because the bibliographic universe is radically changing: resources, agents, technologies, standards and practices. Among the main topics addressed: library cooperation networks; legal deposit; national bibliographies; new tools and standards (IFLA LRM, RDA, BIBFRAME); authority control and new alliances (Wikidata, Wikibase, Identifiers); new ways of indexing resources (artificial intelligence); institutional repositories; new book supply chain; “discoverability” in the IIIF digital ecosystem; role of thesauri and ontologies in the digital ecosystem; bibliographic control and search engines

    Individual differences in early instructed language learning

    Get PDF
    Variability in predispositions for language learning has attracted scholarly curiosity for over 100 years. Despite major changes in theoretical explanations and foreign/second language teaching paradigms, some patterns of associations between predispositions and learning outcomes seem timelessly robust. This book discusses evidence from a research project investigating individual differences in a wide variety of domains, ranging from language aptitude over general cognitive abilities to motivational and other affective and social constructs. The focus lies on young learners aged 10 to 12, a less frequently investigated age in aptitude research. The data stem from two samples of multilingual learners in German-speaking Switzerland. The target languages are French and English. The chapters of the book offer two complementary perspectives on the topic: On the one hand, cross-sectional investigations of the underlying structure of these individual differences and their association with the target languages are discussed. Drawing on factor analytical and multivariable analyses, the different components are scrutinized with respect to their mutual dependence and their relative impact on target language skills. The analyses also take into account contextual factors such as the learners’ family background and differences across the two contexts investigated. On the other hand, the potential to predict learner’s skills in the target language over time based on the many different indicators is investigated using machine learning algorithms. The results provide new insights into the stability of the individual dispositions, on the impact of contextual variables, and on empirically robust dimensions within the array of variables tested

    Charms and Charming

    Get PDF
    In the book are presented studies of 18 renowned researchers focussing on the verbal aspects of everyday magic, placing in the centre the richest and most poetic manifestation of verbal magic – the charm or incantatio. Incantations are in Europe well spread folklore genre, which contain very old magical elemrnts. The book covers wide spectrum of regions, from United Kingdom to Russia and Iran, and includes also Slovenia. The researchers have devoted their attention to phenomenological and theoretical studies of incantatio, and have discussed various topics, from the origin of charms and ancient European magical practices, to the receptions and diffusions of different types of charms

    Early Childhood Science Education: Research Trends in Learning and Teaching

    Get PDF
    This volume consists of a collection of articles that touch on very different research aspects within a broad scientific field known in recent years as Early Childhood Science Education. The field has gradually emerged from the interaction between three distinct scientific areas of theory and research: Early Childhood Education, Psychology, which is oriented towards the study of learning, and Science Education. At the center of the progress in this field are efforts to initiate children aged 4-8 years in the Physical and Biological Sciences. A wide range of research themes have developed around this main axis: children's mental representations of phenomena of the natural world and scientific concepts, the study of the implementation and effectiveness of specific teaching activities related to curricula or activities focusing on the specific characteristics of teaching processes such as reasoning, explanation, communication, interaction or argumentation, the issue of teachers' relevance to the teaching of science, the use of pecialized teaching materials, the emergence of the issue of scientific skills, the highly contemporary issue of the differentiation and inclusion of children in the world of science, important socio-scientific issues, the role of family-related factors etc. Within this context, this collective book aims to reflect contemporary research trends in the field of Early Childhood Science Education

    New Research and Trends in Higher Education

    Get PDF
    This book aims to discuss new research and trends on all dimensions of Higher Education, as there is a growing interest in the field of Higher Education, regarding new methodologies, contexts, and technologies. It includes investigations of diverse issues that affect the learning processes in Higher Education: innovations in learning, new pedagogical methods, and new learning contexts.In this sense, original research contributions of research papers, case studies and demonstrations that present original scientific results, methodological aspects, concepts and educational technologies, on the following topics:a) Technological Developments in Higher Education: mobile technology, virtual environments, augmented reality, automation and robotics, and other tools for universal learning, focusing on issues that are not addressed by existing research;b) Digital Higher Education: mobile learning, eLearning, Game-based Learning, social media in education, new learning models and technologies and wearable technologies for education;c) Case Studies in Higher Education: empirical studies in higher education regarding digital technologies, new methodologies, new evaluation techniques and tools, perceptions of learning processes efficiency and digital learning best practice
    • …
    corecore