3,380 research outputs found
Energy Sharing for Multiple Sensor Nodes with Finite Buffers
We consider the problem of finding optimal energy sharing policies that
maximize the network performance of a system comprising of multiple sensor
nodes and a single energy harvesting (EH) source. Sensor nodes periodically
sense the random field and generate data, which is stored in the corresponding
data queues. The EH source harnesses energy from ambient energy sources and the
generated energy is stored in an energy buffer. Sensor nodes receive energy for
data transmission from the EH source. The EH source has to efficiently share
the stored energy among the nodes in order to minimize the long-run average
delay in data transmission. We formulate the problem of energy sharing between
the nodes in the framework of average cost infinite-horizon Markov decision
processes (MDPs). We develop efficient energy sharing algorithms, namely
Q-learning algorithm with exploration mechanisms based on the -greedy
method as well as upper confidence bound (UCB). We extend these algorithms by
incorporating state and action space aggregation to tackle state-action space
explosion in the MDP. We also develop a cross entropy based method that
incorporates policy parameterization in order to find near optimal energy
sharing policies. Through simulations, we show that our algorithms yield energy
sharing policies that outperform the heuristic greedy method.Comment: 38 pages, 10 figure
Differential Privacy for Relational Algebra: Improving the Sensitivity Bounds via Constraint Systems
Differential privacy is a modern approach in privacy-preserving data analysis
to control the amount of information that can be inferred about an individual
by querying a database. The most common techniques are based on the
introduction of probabilistic noise, often defined as a Laplacian parametric on
the sensitivity of the query. In order to maximize the utility of the query, it
is crucial to estimate the sensitivity as precisely as possible.
In this paper we consider relational algebra, the classical language for
queries in relational databases, and we propose a method for computing a bound
on the sensitivity of queries in an intuitive and compositional way. We use
constraint-based techniques to accumulate the information on the possible
values for attributes provided by the various components of the query, thus
making it possible to compute tight bounds on the sensitivity.Comment: In Proceedings QAPL 2012, arXiv:1207.055
A Coordination Language for Databases
We present a coordination language for the modeling of distributed database
applications. The language, baptized Klaim-DB, borrows the concepts of
localities and nets of the coordination language Klaim but re-incarnates the
tuple spaces of Klaim as databases. It provides high-level abstractions and
primitives for the access and manipulation of structured data, with integrity
and atomicity considerations. We present the formal semantics of Klaim-DB and
develop a type system that avoids potential runtime errors such as certain
evaluation errors and mismatches of data format in tables, which are monitored
in the semantics. The use of the language is illustrated in a scenario where
the sales from different branches of a chain of department stores are
aggregated from their local databases. Raising the abstraction level and
encapsulating integrity checks in the language primitives have benefited the
modeling task considerably
Hi-Val: Iterative Learning of Hierarchical Value Functions for Policy Generation
Task decomposition is effective in manifold applications where the global complexity of a problem makes planning and decision-making too demanding. This is true, for example, in high-dimensional robotics domains, where (1) unpredictabilities and modeling limitations typically prevent the manual specification of robust behaviors, and (2) learning an action policy is challenging due to the curse of dimensionality. In this work, we borrow the concept of Hierarchical Task Networks (HTNs) to decompose the learning procedure, and we exploit Upper Confidence Tree (UCT) search to introduce HOP, a novel iterative algorithm for hierarchical optimistic planning with learned value functions. To obtain better generalization and generate policies, HOP simultaneously learns and uses action values. These are used to formalize constraints within the search space and to reduce the dimensionality of the problem. We evaluate our algorithm both on a fetching task using a simulated 7-DOF KUKA light weight arm and, on a pick and delivery task with a Pioneer robot
Q-CP: Learning Action Values for Cooperative Planning
Research on multi-robot systems has demonstrated promising results in manifold applications and domains. Still, efficiently learning an effective robot behaviors is very difficult, due to unstructured scenarios, high uncertainties, and large state dimensionality (e.g. hyper-redundant and groups of robot). To alleviate this problem, we present Q-CP a cooperative model-based reinforcement learning algorithm, which exploits action values to both (1) guide the exploration of the state space and (2) generate effective policies. Specifically, we exploit Q-learning to attack the curse-of-dimensionality in the iterations of a Monte-Carlo Tree Search. We implement and evaluate Q-CP on different stochastic cooperative (general-sum) games: (1) a simple cooperative navigation problem among 3 robots, (2) a cooperation scenario between a pair of KUKA YouBots performing hand-overs, and (3) a coordination task between two mobile robots entering a door. The obtained results show the effectiveness of Q-CP in the chosen applications, where action values drive the exploration and reduce the computational demand of the planning process while achieving good performance
- …