13,827 research outputs found
Analytical results for the multi-objective design of model-predictive control
In model-predictive control (MPC), achieving the best closed-loop performance
under a given computational resource is the underlying design consideration.
This paper analyzes the MPC design problem with control performance and
required computational resource as competing design objectives. The proposed
multi-objective design of MPC (MOD-MPC) approach extends current methods that
treat control performance and the computational resource separately -- often
with the latter as a fixed constraint -- which requires the implementation
hardware to be known a priori. The proposed approach focuses on the tuning of
structural MPC parameters, namely sampling time and prediction horizon length,
to produce a set of optimal choices available to the practitioner. The posed
design problem is then analyzed to reveal key properties, including smoothness
of the design objectives and parameter bounds, and establish certain validated
guarantees. Founded on these properties, necessary and sufficient conditions
for an effective and efficient solver are presented, leading to a specialized
multi-objective optimizer for the MOD-MPC being proposed. Finally, two
real-world control problems are used to illustrate the results of the design
approach and importance of the developed conditions for an effective solver of
the MOD-MPC problem
Conclave: secure multi-party computation on big data (extended TR)
Secure Multi-Party Computation (MPC) allows mutually distrusting parties to
run joint computations without revealing private data. Current MPC algorithms
scale poorly with data size, which makes MPC on "big data" prohibitively slow
and inhibits its practical use.
Many relational analytics queries can maintain MPC's end-to-end security
guarantee without using cryptographic MPC techniques for all operations.
Conclave is a query compiler that accelerates such queries by transforming them
into a combination of data-parallel, local cleartext processing and small MPC
steps. When parties trust others with specific subsets of the data, Conclave
applies new hybrid MPC-cleartext protocols to run additional steps outside of
MPC and improve scalability further.
Our Conclave prototype generates code for cleartext processing in Python and
Spark, and for secure MPC using the Sharemind and Obliv-C frameworks. Conclave
scales to data sets between three and six orders of magnitude larger than
state-of-the-art MPC frameworks support on their own. Thanks to its hybrid
protocols, Conclave also substantially outperforms SMCQL, the most similar
existing system.Comment: Extended technical report for EuroSys 2019 pape
Meta Reinforcement Learning with Latent Variable Gaussian Processes
Learning from small data sets is critical in many practical applications
where data collection is time consuming or expensive, e.g., robotics, animal
experiments or drug design. Meta learning is one way to increase the data
efficiency of learning algorithms by generalizing learned concepts from a set
of training tasks to unseen, but related, tasks. Often, this relationship
between tasks is hard coded or relies in some other way on human expertise. In
this paper, we frame meta learning as a hierarchical latent variable model and
infer the relationship between tasks automatically from data. We apply our
framework in a model-based reinforcement learning setting and show that our
meta-learning model effectively generalizes to novel tasks by identifying how
new tasks relate to prior ones from minimal data. This results in up to a 60%
reduction in the average interaction time needed to solve tasks compared to
strong baselines.Comment: 11 pages, 7 figure
Controlling the level of sparsity in MPC
In optimization routines used for on-line Model Predictive Control (MPC),
linear systems of equations are usually solved in each iteration. This is true
both for Active Set (AS) methods as well as for Interior Point (IP) methods,
and for linear MPC as well as for nonlinear MPC and hybrid MPC. The main
computational effort is spent while solving these linear systems of equations,
and hence, it is of greatest interest to solve them efficiently. Classically,
the optimization problem has been formulated in either of two different ways.
One of them leading to a sparse linear system of equations involving relatively
many variables to solve in each iteration and the other one leading to a dense
linear system of equations involving relatively few variables. In this work, it
is shown that it is possible not only to consider these two distinct choices of
formulations. Instead it is shown that it is possible to create an entire
family of formulations with different levels of sparsity and number of
variables, and that this extra degree of freedom can be exploited to get even
better performance with the software and hardware at hand. This result also
provides a better answer to an often discussed question in MPC; should the
sparse or dense formulation be used. In this work, it is shown that the answer
to this question is that often none of these classical choices is the best
choice, and that a better choice with a different level of sparsity actually
can be found
Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning
Model-free deep reinforcement learning algorithms have been shown to be
capable of learning a wide range of robotic skills, but typically require a
very large number of samples to achieve good performance. Model-based
algorithms, in principle, can provide for much more efficient learning, but
have proven difficult to extend to expressive, high-capacity models such as
deep neural networks. In this work, we demonstrate that medium-sized neural
network models can in fact be combined with model predictive control (MPC) to
achieve excellent sample complexity in a model-based reinforcement learning
algorithm, producing stable and plausible gaits to accomplish various complex
locomotion tasks. We also propose using deep neural network dynamics models to
initialize a model-free learner, in order to combine the sample efficiency of
model-based approaches with the high task-specific performance of model-free
methods. We empirically demonstrate on MuJoCo locomotion tasks that our pure
model-based approach trained on just random action data can follow arbitrary
trajectories with excellent sample efficiency, and that our hybrid algorithm
can accelerate model-free learning on high-speed benchmark tasks, achieving
sample efficiency gains of 3-5x on swimmer, cheetah, hopper, and ant agents.
Videos can be found at https://sites.google.com/view/mbm
- …