3,993 research outputs found
Reinforcement learning in large state action spaces
Reinforcement learning (RL) is a promising framework for training intelligent agents which learn to optimize long term utility by directly interacting with the environment. Creating RL methods which scale to large state-action spaces is a critical problem towards ensuring real world deployment of RL systems. However, several challenges limit the applicability of RL to large scale settings. These include difficulties with exploration, low sample efficiency, computational intractability, task constraints like decentralization and lack of guarantees about important properties like performance, generalization and robustness in potentially unseen scenarios.
This thesis is motivated towards bridging the aforementioned gap. We propose several principled algorithms and frameworks for studying and addressing the above challenges RL. The proposed methods cover a wide range of RL settings (single and multi-agent systems (MAS) with all the variations in the latter, prediction and control, model-based and model-free methods, value-based and policy-based methods). In this work we propose the first results on several different problems: e.g. tensorization of the Bellman equation which allows exponential sample efficiency gains (Chapter 4), provable suboptimality arising from structural constraints in MAS(Chapter 3), combinatorial generalization results in cooperative MAS(Chapter 5), generalization results on observation shifts(Chapter 7), learning deterministic policies in a probabilistic RL framework(Chapter 6). Our algorithms exhibit provably enhanced performance and sample efficiency along with better scalability. Additionally, we also shed light on generalization aspects of the agents under different frameworks. These properties have been been driven by the use of several advanced tools (e.g. statistical machine learning, state abstraction, variational inference, tensor theory).
In summary, the contributions in this thesis significantly advance progress towards making RL agents ready for large scale, real world applications
Mean field game of mutual holding with defaultable agents, and systemic risk
We introduce the possibility of default in the mean field game of mutual
holding of Djete and Touzi [11]. This is modeled by introducing absorption at
the origin of the equity process. We provide an explicit solution of this mean
field game. Moreover, we provide a particle system approximation, and we derive
an autonomous equation for the time evolution of the default probability, or
equivalently the law of the hitting time of the origin by the equity process.
The systemic risk is thus described by the evolution of the default
probability
Tradition and Innovation in Construction Project Management
This book is a reprint of the Special Issue 'Tradition and Innovation in Construction Project Management' that was published in the journal Buildings
Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentation
Vessel segmentation in medical images is one of the important tasks in the
diagnosis of vascular diseases and therapy planning. Although learning-based
segmentation approaches have been extensively studied, a large amount of
ground-truth labels are required in supervised methods and confusing background
structures make neural networks hard to segment vessels in an unsupervised
manner. To address this, here we introduce a novel diffusion adversarial
representation learning (DARL) model that leverages a denoising diffusion
probabilistic model with adversarial learning, and apply it to vessel
segmentation. In particular, for self-supervised vessel segmentation, DARL
learns the background signal using a diffusion module, which lets a generation
module effectively provide vessel representations. Also, by adversarial
learning based on the proposed switchable spatially-adaptive denormalization,
our model estimates synthetic fake vessel images as well as vessel segmentation
masks, which further makes the model capture vessel-relevant semantic
information. Once the proposed model is trained, the model generates
segmentation masks in a single step and can be applied to general vascular
structure segmentation of coronary angiography and retinal images. Experimental
results on various datasets show that our method significantly outperforms
existing unsupervised and self-supervised vessel segmentation methods.Comment: Accepted at ICLR 202
Scalable Primal-Dual Actor-Critic Method for Safe Multi-Agent RL with General Utilities
We investigate safe multi-agent reinforcement learning, where agents seek to
collectively maximize an aggregate sum of local objectives while satisfying
their own safety constraints. The objective and constraints are described by
{\it general utilities}, i.e., nonlinear functions of the long-term
state-action occupancy measure, which encompass broader decision-making goals
such as risk, exploration, or imitations. The exponential growth of the
state-action space size with the number of agents presents challenges for
global observability, further exacerbated by the global coupling arising from
agents' safety constraints. To tackle this issue, we propose a primal-dual
method utilizing shadow reward and -hop neighbor truncation under a
form of correlation decay property, where is the communication radius.
In the exact setting, our algorithm converges to a first-order stationary point
(FOSP) at the rate of . In the sample-based
setting, we demonstrate that, with high probability, our algorithm requires
samples to achieve an
-FOSP with an approximation error of ,
where . Finally, we demonstrate the effectiveness of our model
through extensive numerical experiments.Comment: 50 page
Current issues of the management of socio-economic systems in terms of globalization challenges
The authors of the scientific monograph have come to the conclusion that the management of socio-economic systems in the terms of global challenges requires the use of mechanisms to ensure security, optimise the use of resource potential, increase competitiveness, and provide state support to economic entities. Basic research focuses on assessment of economic entities in the terms of global challenges, analysis of the financial system, migration flows, logistics and product exports, territorial development. The research results have been implemented in the different decision-making models in the context of global challenges, strategic planning, financial and food security, education management, information technology and innovation. The results of the study can be used in the developing of directions, programmes and strategies for sustainable development of economic entities and regions, increasing the competitiveness of products and services, decision-making at the level of ministries and agencies that regulate the processes of managing socio-economic systems. The results can also be used by students and young scientists in the educational process and conducting scientific research on the management of socio-economic systems in the terms of global challenges
Designing core-selecting payment rules: a computational search approach
CMMI-1761163 - National Science Foundationhttps://doi.org/10.1287/isre.2022.1108Published versio
An In Silico Imaging Framework for Microstructure-Sensitive Myocardial Diffusion-Weighted MRI
Cardiovascular diseases (CVDs) are a major global health concern, re- responsible for more than a quarter of annual deaths (around 17.5 million). Non-invasive dMRI models, like apparent diffusion coefficient and diffusion tensor imaging, can assess changes in the myocardium due to CVDs. How- ever, these models lack biophysical interpretations preferred by clinicians. To address this, biophysical models are being developed to better under- stand the myocardium’s microstructure. A virtual imaging framework is essential to validate these models and analyze dMRI sensitivity to micro-structural changes.
In this thesis, we present a virtual imaging framework to simulate dMRI signals in cardiac microstructure and microvasculature. The framework in- includes a numerical phantom mimicking cardiac microstructure and a solver for the generalized Bloch-Torrey equation, termed SpinDoctor-IVIM.
With the first objective, the morphometric study found no significant difference (p > 0.01) between the volume, length, and primary and secondary axes of the simulated and real cardiomyocyte data from the literature. Structural correlation analysis confirmed that the in-silico tissue shows a similar disorderliness as the real tissue. The absolute angle differences between the simulated helical angles (HA) and the input HA of the cardiomyocytes (4.3◦ ± 3.1◦) closely match the angle differences reported in previous studies using experimental cardiac diffusion tensor imaging (cDTI) and histology (3.7◦ ± 6.4◦) and (4.9◦ ± 14.6◦).
With the second objective, the SpinDoctor-IVIM stands out for accounting for volumetric microvasculature during blood flow simulations, incorporating diffusion phenomena in the intravascular space, and considering permeability between the intravascular and extravascular spaces, providing more accurate and comprehensive results.
Overall, this thesis contributes valuable insights into the microstructure and microvasculature of the myocardium, offering promising advancements in studying CVD using dMRI. The developed virtual imaging framework is a crucial step towards improving cardiac research based on dMRI
Modern meat: the next generation of meat from cells
Modern Meat is the first textbook on cultivated meat, with contributions from over 100 experts within the cultivated meat community.
The Sections of Modern Meat comprise 5 broad categories of cultivated meat: Context, Impact, Science, Society, and World.
The 19 chapters of Modern Meat, spread across these 5 sections, provide detailed entries on cultivated meat. They extensively tour a range of topics including the impact of cultivated meat on humans and animals, the bioprocess of cultivated meat production, how cultivated meat may become a food option in Space and on Mars, and how cultivated meat may impact the economy, culture, and tradition of Asia
On the Impossibility of Surviving (Iterated) Deletion of Weakly Dominated Strategies in Rational MPC
Rational multiparty computation (rational MPC) provides a framework for analyzing MPC protocols through the lens of game theory. One way to judge whether an MPC protocol is rational is through weak domination: Rational players would not adhere to an MPC protocol if deviating never decreases their utility, but sometimes increases it.
Secret reconstruction protocols are of particular importance in this setting because they represent the last phase of most (rational) MPC protocols. We show that most secret reconstruction protocols from the literature are not, in fact, stable with respect to weak domination. Furthermore, we formally prove that (under certain assumptions) it is impossible to design a secret reconstruction protocol which is a Nash equlibrium but not weakly dominated if (1) shares are authenticated or (2) half of all players may form a coalition
- …