788 research outputs found

    LIPIcs, Volume 251, ITCS 2023, Complete Volume

    Get PDF
    LIPIcs, Volume 251, ITCS 2023, Complete Volum

    Anchor Space Optimal Transport: Accelerating Batch Processing of Multiple OT Problems

    Full text link
    The optimal transport (OT) theory provides an effective way to compare probability distributions on a defined metric space, but it suffers from cubic computational complexity. Although the Sinkhorn's algorithm greatly reduces the computational complexity of OT solutions, the solutions of multiple OT problems are still time-consuming and memory-comsuming in practice. However, many works on the computational acceleration of OT are usually based on the premise of a single OT problem, ignoring the potential common characteristics of the distributions in a mini-batch. Therefore, we propose a translated OT problem designated as the anchor space optimal transport (ASOT) problem, which is specially designed for batch processing of multiple OT problem solutions. For the proposed ASOT problem, the distributions will be mapped into a shared anchor point space, which learns the potential common characteristics and thus help accelerate OT batch processing. Based on the proposed ASOT, the Wasserstein distance error to the original OT problem is proven to be bounded by ground cost errors. Building upon this, we propose three methods to learn an anchor space minimizing the distance error, each of which has its application background. Numerical experiments on real-world datasets show that our proposed methods can greatly reduce computational time while maintaining reasonable approximation performance.Comment: 26 pages, 4 figures, 6 table

    Reinforcement learning in large state action spaces

    Get PDF
    Reinforcement learning (RL) is a promising framework for training intelligent agents which learn to optimize long term utility by directly interacting with the environment. Creating RL methods which scale to large state-action spaces is a critical problem towards ensuring real world deployment of RL systems. However, several challenges limit the applicability of RL to large scale settings. These include difficulties with exploration, low sample efficiency, computational intractability, task constraints like decentralization and lack of guarantees about important properties like performance, generalization and robustness in potentially unseen scenarios. This thesis is motivated towards bridging the aforementioned gap. We propose several principled algorithms and frameworks for studying and addressing the above challenges RL. The proposed methods cover a wide range of RL settings (single and multi-agent systems (MAS) with all the variations in the latter, prediction and control, model-based and model-free methods, value-based and policy-based methods). In this work we propose the first results on several different problems: e.g. tensorization of the Bellman equation which allows exponential sample efficiency gains (Chapter 4), provable suboptimality arising from structural constraints in MAS(Chapter 3), combinatorial generalization results in cooperative MAS(Chapter 5), generalization results on observation shifts(Chapter 7), learning deterministic policies in a probabilistic RL framework(Chapter 6). Our algorithms exhibit provably enhanced performance and sample efficiency along with better scalability. Additionally, we also shed light on generalization aspects of the agents under different frameworks. These properties have been been driven by the use of several advanced tools (e.g. statistical machine learning, state abstraction, variational inference, tensor theory). In summary, the contributions in this thesis significantly advance progress towards making RL agents ready for large scale, real world applications

    Coordinate-Descent Augmented Lagrangian Methods for Interpretative and Adaptive Model Predictive Control

    Get PDF
    Model predictive control (MPC) of nonlinear systems suffers a trade-off between model accuracy and real-time compu- tational burden. This thesis presents an interpretative and adaptive MPC (IA-MPC) framework for nonlinear systems, which is related to the widely used approximation method based on successive linearization MPC and Extended Kalman Filtering (SL-MPC-EKF). First, we introduce a solution algo- rithm for linear MPC that is based on the combination of Co- ordinate Descent and Augmented Lagrangian (CDAL) ideas. The CDAL algorithm enjoys three features: (i) it is construction-free, in that it avoids explicitly constructing the quadratic pro-gramming (QP) problem associated with MPC; (ii) is matrix-free, as it avoids multiplications and factorizations of matri-ces; and (iii) is library-free, as it can be simply coded without any library dependency, 90-lines of C-code in our implemen-tation. We specialize the algorithm for both state-space for-mulations of MPC and formulations based on AutoRegres-sive with eXogenous terms models (CDAL-ARX). The thesis also presents a rapid-prototype MPC tool based on the gPROMS platform, in which the qpOASES and CDAL algorithm was integrated. In addition, based on an equivalence between SS-based and ARX-based MPC problems we show,we investigate the relation between the proposed IA-MPC and the classical SL-MPC-EKF method. Finally, we test and show the effectiveness of the proposed IA-MPC frameworkon four typical nonlinear MPC benchmark examples

    Behavior quantification as the missing link between fields: Tools for digital psychiatry and their role in the future of neurobiology

    Full text link
    The great behavioral heterogeneity observed between individuals with the same psychiatric disorder and even within one individual over time complicates both clinical practice and biomedical research. However, modern technologies are an exciting opportunity to improve behavioral characterization. Existing psychiatry methods that are qualitative or unscalable, such as patient surveys or clinical interviews, can now be collected at a greater capacity and analyzed to produce new quantitative measures. Furthermore, recent capabilities for continuous collection of passive sensor streams, such as phone GPS or smartwatch accelerometer, open avenues of novel questioning that were previously entirely unrealistic. Their temporally dense nature enables a cohesive study of real-time neural and behavioral signals. To develop comprehensive neurobiological models of psychiatric disease, it will be critical to first develop strong methods for behavioral quantification. There is huge potential in what can theoretically be captured by current technologies, but this in itself presents a large computational challenge -- one that will necessitate new data processing tools, new machine learning techniques, and ultimately a shift in how interdisciplinary work is conducted. In my thesis, I detail research projects that take different perspectives on digital psychiatry, subsequently tying ideas together with a concluding discussion on the future of the field. I also provide software infrastructure where relevant, with extensive documentation. Major contributions include scientific arguments and proof of concept results for daily free-form audio journals as an underappreciated psychiatry research datatype, as well as novel stability theorems and pilot empirical success for a proposed multi-area recurrent neural network architecture.Comment: PhD thesis cop

    The Self The Soul and The World: Affect Reason and Complexity

    Get PDF
    This book looks at the affective-cognitive roots of how the human mind inquires into the workings of nature and, more generally, how the mind confronts reality. Reality is an infinitely complex system, in virtue of which the mind can comprehend it only in bits and pieces, by making up interpretations of the myriads of signals received from the world by way of integrating those with information stored from the past. This constitutes a piecemeal interpretation by which we assemble our phenomenal reality. In perceiving the complex world and responding to it, the mind invokes the logic of affect and the logic of reason, the former mostly innate and implicit, and the latter generated consciously in explicit terms with reference to mind-independent relations between entities in nature. It is a strange combination of affect and reason that enables us to make decisions and inferences, --- the latter mostly of the inductive type --- thereby making possible the development of theories. Theories are our tool-kits for explaining and predicting phenomena, guiding us along in our journey in life. Theories, however, are defeasible, and need to be constantly updated, at times even radically. In this, the self and the soul are of enormous relevance. The former is the affect-based psychological engine driving all our mental processes, while the latter is the capacity of the conscious mind to examine and reconstruct the self by modulating repressed conflicts. If the soul remains inoperative, all our theories become misdirected and a rot spreads inexorably all around us

    LIPIcs, Volume 261, ICALP 2023, Complete Volume

    Get PDF
    LIPIcs, Volume 261, ICALP 2023, Complete Volum

    Solving Satisfiability Modulo Counting for Symbolic and Statistical AI Integration With Provable Guarantees

    Full text link
    Satisfiability Modulo Counting (SMC) encompasses problems that require both symbolic decision-making and statistical reasoning. Its general formulation captures many real-world problems at the intersection of symbolic and statistical Artificial Intelligence. SMC searches for policy interventions to control probabilistic outcomes. Solving SMC is challenging because of its highly intractable nature(NPPP\text{NP}^{\text{PP}}-complete), incorporating statistical inference and symbolic reasoning. Previous research on SMC solving lacks provable guarantees and/or suffers from sub-optimal empirical performance, especially when combinatorial constraints are present. We propose XOR-SMC, a polynomial algorithm with access to NP-oracles, to solve highly intractable SMC problems with constant approximation guarantees. XOR-SMC transforms the highly intractable SMC into satisfiability problems, by replacing the model counting in SMC with SAT formulae subject to randomized XOR constraints. Experiments on solving important SMC problems in AI for social good demonstrate that XOR-SMC finds solutions close to the true optimum, outperforming several baselines which struggle to find good approximations for the intractable model counting in SMC
    • …
    corecore