6,825 research outputs found
Optimality and Complexity in Measured Quantum-State Stochastic Processes
If an experimentalist observes a sequence of emitted quantum states via
either projective or positive-operator-valued measurements, the outcomes form a
time series. Individual time series are realizations of a stochastic process
over the measurements' classical outcomes. We recently showed that, in general,
the resulting stochastic process is highly complex in two specific senses: (i)
it is inherently unpredictable to varying degrees that depend on measurement
choice and (ii) optimal prediction requires using an infinite number of
temporal features. Here, we identify the mechanism underlying this
complicatedness as generator nonunifilarity -- the degeneracy between sequences
of generator states and sequences of measurement outcomes. This makes it
possible to quantitatively explore the influence that measurement choice has on
a quantum process' degrees of randomness and structural complexity using
recently introduced methods from ergodic theory. Progress in this, though,
requires quantitative measures of structure and memory in observed time series.
And, success requires accurate and efficient estimation algorithms that
overcome the requirement to explicitly represent an infinite set of predictive
features. We provide these metrics and associated algorithms, using them to
design informationally-optimal measurements of open quantum dynamical systems.Comment: 31 pages, 6 appendices, 22 figures;
http://csc.ucdavis.edu/~cmg/compmech/pubs/qdic.ht
Reinforcement learning in large state action spaces
Reinforcement learning (RL) is a promising framework for training intelligent agents which learn to optimize long term utility by directly interacting with the environment. Creating RL methods which scale to large state-action spaces is a critical problem towards ensuring real world deployment of RL systems. However, several challenges limit the applicability of RL to large scale settings. These include difficulties with exploration, low sample efficiency, computational intractability, task constraints like decentralization and lack of guarantees about important properties like performance, generalization and robustness in potentially unseen scenarios.
This thesis is motivated towards bridging the aforementioned gap. We propose several principled algorithms and frameworks for studying and addressing the above challenges RL. The proposed methods cover a wide range of RL settings (single and multi-agent systems (MAS) with all the variations in the latter, prediction and control, model-based and model-free methods, value-based and policy-based methods). In this work we propose the first results on several different problems: e.g. tensorization of the Bellman equation which allows exponential sample efficiency gains (Chapter 4), provable suboptimality arising from structural constraints in MAS(Chapter 3), combinatorial generalization results in cooperative MAS(Chapter 5), generalization results on observation shifts(Chapter 7), learning deterministic policies in a probabilistic RL framework(Chapter 6). Our algorithms exhibit provably enhanced performance and sample efficiency along with better scalability. Additionally, we also shed light on generalization aspects of the agents under different frameworks. These properties have been been driven by the use of several advanced tools (e.g. statistical machine learning, state abstraction, variational inference, tensor theory).
In summary, the contributions in this thesis significantly advance progress towards making RL agents ready for large scale, real world applications
Recommended from our members
A Survey of Quantum-Cognitively Inspired Sentiment Analysis Models
Quantum theory, originally proposed as a physical theory to describe the motions of microscopic particles, has been applied to various non-physics domains involving human cognition and decision-making that are inherently uncertain and exhibit certain non-classical, quantum-like characteristics. Sentiment analysis is a typical example of such domains. In the last few years, by leveraging the modeling power of quantum probability (a non-classical probability stemming from quantum mechanics methodology) and deep neural networks, a range of novel quantum-cognitively inspired models for sentiment analysis have emerged and performed well. This survey presents a timely overview of the latest developments in this fascinating cross-disciplinary area. We first provide a background of quantum probability and quantum cognition at a theoretical level, analyzing their advantages over classical theories in modeling the cognitive aspects of sentiment analysis. Then, recent quantum-cognitively inspired models are introduced and discussed in detail, focusing on how they approach the key challenges of the sentiment analysis task. Finally, we discuss the limitations of the current research and highlight future research directions
Convex Optimization for Machine Learning
This book covers an introduction to convex optimization, one of the powerful and tractable optimization problems that can be efficiently solved on a computer. The goal of the book is to
help develop a sense of what convex optimization is, and how it can be used in a widening array of practical contexts with a particular emphasis on machine learning.
The first part of the book covers core concepts of convex sets, convex functions, and related basic definitions that serve understanding convex optimization and its corresponding models. The second part deals with one very useful theory, called duality, which enables us to: (1) gain algorithmic insights; and (2) obtain an approximate solution to non-convex optimization problems which are often difficult to solve. The last part focuses on modern applications in machine learning and deep learning.
A defining feature of this book is that it succinctly relates the “story” of how convex optimization plays a role, via historical examples and trending machine learning applications. Another key feature is that it includes programming implementation of a variety of machine learning algorithms inspired by optimization fundamentals, together with a brief tutorial of the used programming tools. The implementation is based on Python, CVXPY, and TensorFlow.
This book does not follow a traditional textbook-style organization, but is streamlined via a series of lecture notes that are intimately related, centered around coherent themes and concepts. It serves as a textbook mainly for a senior-level undergraduate course, yet is also suitable for a first-year graduate course. Readers benefit from having a good background in linear algebra, some exposure to probability, and basic familiarity with Python
Waiting Nets: State Classes and Taxonomy
In time Petri nets (TPNs), time and control are tightly connected: time
measurement for a transition starts only when all resources needed to fire it
are available. Further, upper bounds on duration of enabledness can force
transitions to fire (this is called urgency). For many systems, one wants to
decouple control and time, i.e. start measuring time as soon as a part of the
preset of a transition is filled, and fire it after some delay \underline{and}
when all needed resources are available. This paper considers an extension of
TPN called waiting nets that dissociates time measurement and control. Their
semantics allows time measurement to start with incomplete presets, and can
ignore urgency when upper bounds of intervals are reached but all resources
needed to fire are not yet available. Firing of a transition is then allowed as
soon as missing resources are available. It is known that extending bounded
TPNs with stopwatches leads to undecidability. Our extension is weaker, and we
show how to compute a finite state class graph for bounded waiting nets,
yielding decidability of reachability and coverability. We then compare
expressiveness of waiting nets with that of other models w.r.t. timed language
equivalence, and show that they are strictly more expressive than TPNs
Diddy: a Python toolbox for infinite discrete dynamical systems
We introduce Diddy, a collection of Python scripts for analyzing infinite
discrete dynamical systems. The main focus is on generalized multidimensional
shifts of finite type (SFTs). We show how Diddy can be used to easily define
SFTs and cellular automata, and analyze their basic properties. We also
showcase how to verify or rediscover some results from coding theory and
cellular automata theory.Comment: 12 page
ON EXPRESSIVENESS, INFERENCE, AND PARAMETER ESTIMATION OF DISCRETE SEQUENCE MODELS
Huge neural autoregressive sequence models have achieved impressive performance across different applications, such as NLP, reinforcement learning, and bioinformatics. However, some lingering problems (e.g., consistency and coherency of generated texts) continue to exist, regardless of the parameter count. In the first part of this thesis, we chart a taxonomy of the expressiveness of various sequence model families (Ch 3). In particular, we put forth complexity-theoretic proofs that string latent-variable sequence models are strictly more expressive than energy-based sequence models, which in turn are more expressive than autoregressive sequence models. Based on these findings, we introduce residual energy-based sequence models, a family of energy-based sequence models (Ch 4) whose sequence weights can be evaluated efficiently, and also perform competitively against autoregressive models. However, we show how unrestricted energy-based sequence models can suffer from uncomputability; and how such a problem is generally unfixable without knowledge of the true sequence distribution (Ch 5).
In the second part of the thesis, we study practical sequence model families and algorithms based on theoretical findings in the first part of the thesis. We introduce neural particle smoothing (Ch 6), a family of approximate sampling methods that work with conditional latent variable models. We also introduce neural finite-state transducers (Ch 7), which extend weighted finite state transducers with the introduction of mark strings, allowing scoring transduction paths in a finite state transducer with a neural network. Finally, we propose neural regular expressions (Ch 8), a family of neural sequence models that are easy to engineer, allowing a user to design flexible weighted relations using Marked FSTs, and combine these weighted relations together with various operations
A Quantum-Classical Model of Brain Dynamics
The study of the human psyche has elucidated a bipartite structure of
cognition reflecting the quantum-classical nature of any process that generates
knowledge and learning governed by brain activity. Acknowledging the importance
of such a finding for modelization, we posit an approach to study brain by
means of the quantum-classical dynamics of a Mixed Weyl symbol. The Mixed Weyl
symbol is used to describe brain processes at the microscopic level and
provides a link to the results of measurements made at the mesoscopic scale.
Within this approach, quantum variables (such as,for example, nuclear and
electron spins, dipole momenta of particles or molecules, tunneling degrees of
freedom, etc may be represented by spinors while the electromagnetic fields and
phonon modes involved in the processes are treated either classically or
semi-classically, by also considering quantum zero-point fluctuations.
Zero-point quantum effects can be incorporated into numerical simulations by
controlling the temperature of each field mode via coupling to a dedicated
Nos\`e-Hoover chain thermostat. The temperature of each thermostat is chosen in
order to reproduce quantum statistics in the canonical ensemble. In this first
paper, we introduce a quantum-classical model of brain dynamics, clarifying its
mathematical strucure and focusing the discussion on its predictive value.
Analytical consequences of the model are not reported in this paper, since they
are left for future work. Our treatment incorporates compatible features of
three well-known quantum approaches to brain dynamics - namely the
electromagnetic field theory approach, the orchestrated objective reduction
theory, and the dissipative quantum model of the brain - and hints at
convincing arguments that sustain the existence of quantum-classical processes
in the brain activity. All three models are reviewed.Comment: Submitted to Entropy [MDPI], Special Issue "Quantum Processes in
Living Systems
Multi-Scale Simulation of Complex Systems: A Perspective of Integrating Knowledge and Data
Complex system simulation has been playing an irreplaceable role in
understanding, predicting, and controlling diverse complex systems. In the past
few decades, the multi-scale simulation technique has drawn increasing
attention for its remarkable ability to overcome the challenges of complex
system simulation with unknown mechanisms and expensive computational costs. In
this survey, we will systematically review the literature on multi-scale
simulation of complex systems from the perspective of knowledge and data.
Firstly, we will present background knowledge about simulating complex system
simulation and the scales in complex systems. Then, we divide the main
objectives of multi-scale modeling and simulation into five categories by
considering scenarios with clear scale and scenarios with unclear scale,
respectively. After summarizing the general methods for multi-scale simulation
based on the clues of knowledge and data, we introduce the adopted methods to
achieve different objectives. Finally, we introduce the applications of
multi-scale simulation in typical matter systems and social systems
- …