3,993 research outputs found

    Reinforcement learning in large state action spaces

    Get PDF
    Reinforcement learning (RL) is a promising framework for training intelligent agents which learn to optimize long term utility by directly interacting with the environment. Creating RL methods which scale to large state-action spaces is a critical problem towards ensuring real world deployment of RL systems. However, several challenges limit the applicability of RL to large scale settings. These include difficulties with exploration, low sample efficiency, computational intractability, task constraints like decentralization and lack of guarantees about important properties like performance, generalization and robustness in potentially unseen scenarios. This thesis is motivated towards bridging the aforementioned gap. We propose several principled algorithms and frameworks for studying and addressing the above challenges RL. The proposed methods cover a wide range of RL settings (single and multi-agent systems (MAS) with all the variations in the latter, prediction and control, model-based and model-free methods, value-based and policy-based methods). In this work we propose the first results on several different problems: e.g. tensorization of the Bellman equation which allows exponential sample efficiency gains (Chapter 4), provable suboptimality arising from structural constraints in MAS(Chapter 3), combinatorial generalization results in cooperative MAS(Chapter 5), generalization results on observation shifts(Chapter 7), learning deterministic policies in a probabilistic RL framework(Chapter 6). Our algorithms exhibit provably enhanced performance and sample efficiency along with better scalability. Additionally, we also shed light on generalization aspects of the agents under different frameworks. These properties have been been driven by the use of several advanced tools (e.g. statistical machine learning, state abstraction, variational inference, tensor theory). In summary, the contributions in this thesis significantly advance progress towards making RL agents ready for large scale, real world applications

    Mean field game of mutual holding with defaultable agents, and systemic risk

    Full text link
    We introduce the possibility of default in the mean field game of mutual holding of Djete and Touzi [11]. This is modeled by introducing absorption at the origin of the equity process. We provide an explicit solution of this mean field game. Moreover, we provide a particle system approximation, and we derive an autonomous equation for the time evolution of the default probability, or equivalently the law of the hitting time of the origin by the equity process. The systemic risk is thus described by the evolution of the default probability

    Tradition and Innovation in Construction Project Management

    Get PDF
    This book is a reprint of the Special Issue 'Tradition and Innovation in Construction Project Management' that was published in the journal Buildings

    Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentation

    Full text link
    Vessel segmentation in medical images is one of the important tasks in the diagnosis of vascular diseases and therapy planning. Although learning-based segmentation approaches have been extensively studied, a large amount of ground-truth labels are required in supervised methods and confusing background structures make neural networks hard to segment vessels in an unsupervised manner. To address this, here we introduce a novel diffusion adversarial representation learning (DARL) model that leverages a denoising diffusion probabilistic model with adversarial learning, and apply it to vessel segmentation. In particular, for self-supervised vessel segmentation, DARL learns the background signal using a diffusion module, which lets a generation module effectively provide vessel representations. Also, by adversarial learning based on the proposed switchable spatially-adaptive denormalization, our model estimates synthetic fake vessel images as well as vessel segmentation masks, which further makes the model capture vessel-relevant semantic information. Once the proposed model is trained, the model generates segmentation masks in a single step and can be applied to general vascular structure segmentation of coronary angiography and retinal images. Experimental results on various datasets show that our method significantly outperforms existing unsupervised and self-supervised vessel segmentation methods.Comment: Accepted at ICLR 202

    Scalable Primal-Dual Actor-Critic Method for Safe Multi-Agent RL with General Utilities

    Full text link
    We investigate safe multi-agent reinforcement learning, where agents seek to collectively maximize an aggregate sum of local objectives while satisfying their own safety constraints. The objective and constraints are described by {\it general utilities}, i.e., nonlinear functions of the long-term state-action occupancy measure, which encompass broader decision-making goals such as risk, exploration, or imitations. The exponential growth of the state-action space size with the number of agents presents challenges for global observability, further exacerbated by the global coupling arising from agents' safety constraints. To tackle this issue, we propose a primal-dual method utilizing shadow reward and κ\kappa-hop neighbor truncation under a form of correlation decay property, where κ\kappa is the communication radius. In the exact setting, our algorithm converges to a first-order stationary point (FOSP) at the rate of O(T2/3)\mathcal{O}\left(T^{-2/3}\right). In the sample-based setting, we demonstrate that, with high probability, our algorithm requires O~(ϵ3.5)\widetilde{\mathcal{O}}\left(\epsilon^{-3.5}\right) samples to achieve an ϵ\epsilon-FOSP with an approximation error of O(ϕ02κ)\mathcal{O}(\phi_0^{2\kappa}), where ϕ0(0,1)\phi_0\in (0,1). Finally, we demonstrate the effectiveness of our model through extensive numerical experiments.Comment: 50 page

    Current issues of the management of socio-economic systems in terms of globalization challenges

    Get PDF
    The authors of the scientific monograph have come to the conclusion that the management of socio-economic systems in the terms of global challenges requires the use of mechanisms to ensure security, optimise the use of resource potential, increase competitiveness, and provide state support to economic entities. Basic research focuses on assessment of economic entities in the terms of global challenges, analysis of the financial system, migration flows, logistics and product exports, territorial development. The research results have been implemented in the different decision-making models in the context of global challenges, strategic planning, financial and food security, education management, information technology and innovation. The results of the study can be used in the developing of directions, programmes and strategies for sustainable development of economic entities and regions, increasing the competitiveness of products and services, decision-making at the level of ministries and agencies that regulate the processes of managing socio-economic systems. The results can also be used by students and young scientists in the educational process and conducting scientific research on the management of socio-economic systems in the terms of global challenges

    Designing core-selecting payment rules: a computational search approach

    Get PDF
    CMMI-1761163 - National Science Foundationhttps://doi.org/10.1287/isre.2022.1108Published versio

    An In Silico Imaging Framework for Microstructure-Sensitive Myocardial Diffusion-Weighted MRI

    Get PDF
    Cardiovascular diseases (CVDs) are a major global health concern, re- responsible for more than a quarter of annual deaths (around 17.5 million). Non-invasive dMRI models, like apparent diffusion coefficient and diffusion tensor imaging, can assess changes in the myocardium due to CVDs. How- ever, these models lack biophysical interpretations preferred by clinicians. To address this, biophysical models are being developed to better under- stand the myocardium’s microstructure. A virtual imaging framework is essential to validate these models and analyze dMRI sensitivity to micro-structural changes. In this thesis, we present a virtual imaging framework to simulate dMRI signals in cardiac microstructure and microvasculature. The framework in- includes a numerical phantom mimicking cardiac microstructure and a solver for the generalized Bloch-Torrey equation, termed SpinDoctor-IVIM. With the first objective, the morphometric study found no significant difference (p > 0.01) between the volume, length, and primary and secondary axes of the simulated and real cardiomyocyte data from the literature. Structural correlation analysis confirmed that the in-silico tissue shows a similar disorderliness as the real tissue. The absolute angle differences between the simulated helical angles (HA) and the input HA of the cardiomyocytes (4.3◦ ± 3.1◦) closely match the angle differences reported in previous studies using experimental cardiac diffusion tensor imaging (cDTI) and histology (3.7◦ ± 6.4◦) and (4.9◦ ± 14.6◦). With the second objective, the SpinDoctor-IVIM stands out for accounting for volumetric microvasculature during blood flow simulations, incorporating diffusion phenomena in the intravascular space, and considering permeability between the intravascular and extravascular spaces, providing more accurate and comprehensive results. Overall, this thesis contributes valuable insights into the microstructure and microvasculature of the myocardium, offering promising advancements in studying CVD using dMRI. The developed virtual imaging framework is a crucial step towards improving cardiac research based on dMRI

    Modern meat: the next generation of meat from cells

    Get PDF
    Modern Meat is the first textbook on cultivated meat, with contributions from over 100 experts within the cultivated meat community. The Sections of Modern Meat comprise 5 broad categories of cultivated meat: Context, Impact, Science, Society, and World. The 19 chapters of Modern Meat, spread across these 5 sections, provide detailed entries on cultivated meat. They extensively tour a range of topics including the impact of cultivated meat on humans and animals, the bioprocess of cultivated meat production, how cultivated meat may become a food option in Space and on Mars, and how cultivated meat may impact the economy, culture, and tradition of Asia

    On the Impossibility of Surviving (Iterated) Deletion of Weakly Dominated Strategies in Rational MPC

    Get PDF
    Rational multiparty computation (rational MPC) provides a framework for analyzing MPC protocols through the lens of game theory. One way to judge whether an MPC protocol is rational is through weak domination: Rational players would not adhere to an MPC protocol if deviating never decreases their utility, but sometimes increases it. Secret reconstruction protocols are of particular importance in this setting because they represent the last phase of most (rational) MPC protocols. We show that most secret reconstruction protocols from the literature are not, in fact, stable with respect to weak domination. Furthermore, we formally prove that (under certain assumptions) it is impossible to design a secret reconstruction protocol which is a Nash equlibrium but not weakly dominated if (1) shares are authenticated or (2) half of all players may form a coalition
    corecore