2,392 research outputs found
Synthesis of Minimal Error Control Software
Software implementations of controllers for physical systems are at the core
of many embedded systems. The design of controllers uses the theory of
dynamical systems to construct a mathematical control law that ensures that the
controlled system has certain properties, such as asymptotic convergence to an
equilibrium point, while optimizing some performance criteria. However, owing
to quantization errors arising from the use of fixed-point arithmetic, the
implementation of this control law can only guarantee practical stability:
under the actions of the implementation, the trajectories of the controlled
system converge to a bounded set around the equilibrium point, and the size of
the bounded set is proportional to the error in the implementation. The problem
of verifying whether a controller implementation achieves practical stability
for a given bounded set has been studied before. In this paper, we change the
emphasis from verification to automatic synthesis. Using synthesis, the need
for formal verification can be considerably reduced thereby reducing the design
time as well as design cost of embedded control software.
We give a methodology and a tool to synthesize embedded control software that
is Pareto optimal w.r.t. both performance criteria and practical stability
regions. Our technique is a combination of static analysis to estimate
quantization errors for specific controller implementations and stochastic
local search over the space of possible controllers using particle swarm
optimization. The effectiveness of our technique is illustrated using examples
of various standard control systems: in most examples, we achieve controllers
with close LQR-LQG performance but with implementation errors, hence regions of
practical stability, several times as small.Comment: 18 pages, 2 figure
An Information-Theoretic Analysis of Thompson Sampling
We provide an information-theoretic analysis of Thompson sampling that
applies across a broad range of online optimization problems in which a
decision-maker must learn from partial feedback. This analysis inherits the
simplicity and elegance of information theory and leads to regret bounds that
scale with the entropy of the optimal-action distribution. This strengthens
preexisting results and yields new insight into how information improves
performance
Algorithms for Optimal Control with Fixed-Rate Feedback
We consider a discrete-time linear quadratic Gaussian networked control setting where the (full information) observer and controller are separated by a fixed-rate noiseless channel. The minimal rate required to stabilize such a system has been well studied. However, for a given fixed rate, how to quantize the states so as to optimize performance is an open question of great theoretical and practical significance. We concentrate on minimizing the control cost for first-order scalar systems. To that end, we use the Lloyd-Max algorithm and leverage properties of logarithmically-concave functions and sequential Bayesian filtering to construct the optimal quantizer that greedily minimizes the cost at every time instant. By connecting the globally optimal scheme to the problem of scalar successive refinement, we argue that its gain over the proposed greedy algorithm is negligible. This is significant since the globally optimal scheme is often computationally intractable. All the results are proven for the more general case of disturbances with logarithmically-concave distributions and rate-limited time-varying noiseless channels. We further extend the framework to event-triggered control by allowing to convey information via an additional "silent symbol", i.e., by avoiding transmitting bits; by constraining the minimal probability of silence we attain a tradeoff between the transmission rate and the control cost for rates below one bit per sample
Quantization for decentralized learning under subspace constraints
In this paper, we consider decentralized optimization problems where agents
have individual cost functions to minimize subject to subspace constraints that
require the minimizers across the network to lie in low-dimensional subspaces.
This constrained formulation includes consensus or single-task optimization as
special cases, and allows for more general task relatedness models such as
multitask smoothness and coupled optimization. In order to cope with
communication constraints, we propose and study an adaptive decentralized
strategy where the agents employ differential randomized quantizers to compress
their estimates before communicating with their neighbors. The analysis shows
that, under some general conditions on the quantization noise, and for
sufficiently small step-sizes , the strategy is stable both in terms of
mean-square error and average bit rate: by reducing , it is possible to
keep the estimation errors small (on the order of ) without increasing
indefinitely the bit rate as . Simulations illustrate the
theoretical findings and the effectiveness of the proposed approach, revealing
that decentralized learning is achievable at the expense of only a few bits
Recommended from our members
Advances in Compression using Probabilistic Models
The increasing demand for data transmission and storage necessitate the use of efficient compression methods. Compression algorithms work by mapping data to a more compact representation from which the original data can be recovered. To operate efficiently, they need to capture the characteristics of the data distribution, which can be difficult, especially for high-dimensional data.
One emerging solution lies in applying probabilistic machine learning to capture the data distribution in an unsupervised manner. Once a probabilistic model for the data is defined, variational inference can be used to infer its parameters from data. Variational inference is closely related to the optimal compression size, as stated by Hinton's bits-back argument: the evidence lower bound, the objective optimized by variational inference, corresponds to a lower bound on the optimal compression size of the average datapoint. However, current compression methods rely on variational inference merely as a heuristic, and they do not approach its postulated efficiency. In this thesis, we present principled and practical algorithms that get closer to this limit. After discussing our approach, we demonstrate its efficacy in image compression and model compression.
First, we focus on image compression, where we use a variational autoencoder to learn a mapping between the images and their unobserved, latent representations. We propose a stochastic coding scheme to encode the latent representation, from which the original image can be approximately reconstructed. Next, we look at the compression of deep learning models. We use variational inference to approximate the posterior distribution of the weights in a neural network, and apply our stochastic coding scheme to encode a weight configuration. Finally, we investigate a connection between variational inference and our compression algorithm. We show that a technique we used for compression can improve variational inference by generating samples from a highly flexible posterior approximation, without significantly increasing the computational costs
Anomaly detection and dynamic decision making for stochastic systems
Thesis (Ph.D.)--Boston UniversityThis dissertation focuses on two types of problems, both of which are related to systems with uncertainties.
The first problem concerns network system anomaly detection. We present several stochastic and deterministic methods for anomaly detection of networks whose normal behavior is not time-varying. Our methods cover most of the common techniques in the anomaly detection field. We evaluate all methods in a simulated network that consists of nominal data, three flow-level anomalies and one packet-level attack. Through analyzing the results, we summarize the advantages and the disadvantages of each method. As a next step, we propose two robust stochastic anomaly detection methods for networks whose normal behavior is time-varying. We develop a procedure for learning the underlying family of patterns that characterize a time-varying network.
This procedure first estimates a large class of patterns from network data and then refines it to select a representative subset. The latter part formulates the refinement problem using ideas from set covering via integer programming. Then we propose two robust methods, one model-free and one model-based, to evaluate whether a sequence of observations is drawn from the learned patterns. Simulation results show that the robust methods have significant advantages over the alternative stationary methods in time-varying networks. The final anomaly detection setting we consider targets the detection of botnets before they launch an attack. Our method analyzes the social graph of the nodes in a network and consists of two stages: (i) network anomaly detection based on large deviations theory and (ii) community detection based on a refined modularity measure. We apply our method on real-world botnet traffic and compare its performance with other methods.
The second problem considered by this dissertation concerns sequential decision mak- ings under uncertainty, which can be modeled by a Markov Decision Processes (MDPs). We focus on methods with an actor-critic structure, where the critic part estimates the gradient of the overall objective with respect to tunable policy parameters and the actor part optimizes a policy with respect to these parameters. Most existing actor- critic methods use Temporal Difference (TD) learning to estimate the gradient and steepest gradient ascent to update the policies. Our first contribution is to propose an actor-critic method that uses a Least Squares Temporal Difference (LSTD) method, which is known to converge faster than the TD methods. Our second contribution is to develop a new Newton-like actor-critic method that performs better especially for ill-conditioned problems. We evaluate our methods in problems motivated from robot motion control
- …