Rollout algorithm based duty cycle control with joint optimisation of delay and energy efficiency for beacon-enabled IEEE 802.15.4 networks

Duty cycle control is applied in IEEE 802.15.4 medium access control (MAC) protocol to reduce energy consumption. A low duty cycle improves the energy efficiency but it reduces the available transmission time, thereby increases the end-to-end delay. Thus, it is a challenge issue to achieve a good trade-off between energy efficiency and delay. In this paper, we study a duty cycle control problem with the aim of minimising the joint-cost of energy consumption and end-to-end delay. By applying dynamic programming (DP), the optimal duty cycle control is derived. Furthermore, to ensure the feasibility of implementing the control on computation limited sensor devices, a low complexity rollout algorithm based duty cycle control (RADutyCon) is proposed. The joint-cost upper bound of the proposed RADutyCon is investigated. Simulation results show that RADutyCon can effectively reduces the joint-cost of energy consumption and end-to-end delay under various network traffic. In addition, RADutyCon achieves an exponential reduction of computation complexity compared with DP optimal control

Delay Sensitive Communications over Cognitive Radio Networks

Supporting the quality of service of unlicensed users in cognitive radio networks is very challenging, mainly due to dynamic resource availability because of the licensed users' activities. In this paper, we study the optimal admission control and channel allocation decisions in cognitive overlay networks in order to support delay sensitive communications of unlicensed users. We formulate it as a Markov decision process problem, and solve it by transforming the original formulation into a stochastic shortest path problem. We then propose a simple heuristic control policy, which includes a threshold-based admission control scheme and and a largest-delay-first channel allocation scheme, and prove the optimality of the largest-delay-first channel allocation scheme. We further propose an improved policy using the rollout algorithm. By comparing the performance of both proposed policies with the upper-bound of the maximum revenue, we show that our policies achieve close-to-optimal performance with low complexities.Comment: 11 pages, 8 figure

Efficient Domain Coverage for Vehicles with Second Order Dynamics via Multi-Agent Reinforcement Learning

Collaborative autonomous multi-agent systems covering a specified area have many potential applications, such as UAV search and rescue, forest fire fighting, and real-time high-resolution monitoring. Traditional approaches for such coverage problems involve designing a model-based control policy based on sensor data. However, designing model-based controllers is challenging, and the state-of-the-art classical control policy still exhibits a large degree of suboptimality. In this paper, we present a reinforcement learning (RL) approach for the multi-agent coverage problem involving agents with second-order dynamics. Our approach is based on the Multi-Agent Proximal Policy Optimization Algorithm (MAPPO). To improve the stability of the learning-based policy and efficiency of exploration, we utilize an imitation loss based on the state-of-the-art classical control policy. Our trained policy significantly outperforms the state-of-the-art. Our proposed network architecture includes incorporation of self attention, which allows a single-shot domain transfer of the trained policy to a large variety of domain shapes and number of agents. We demonstrate our proposed method in a variety of simulated experiments.Comment: This paper has been submitted to IEEE Robotics and Automation Letters. Includes 8 pages with 5 figure

Active Information Acquisition With Mobile Robots

The recent proliferation of sensors and robots has potential to transform fields as diverse as environmental monitoring, security and surveillance, localization and mapping, and structure inspection. One of the great technical challenges in these scenarios is to control the sensors and robots in order to extract accurate information about various physical phenomena autonomously. The goal of this dissertation is to provide a unified approach for active information acquisition with a team of sensing robots. We formulate a decision problem for maximizing relevant information measures, constrained by the motion capabilities and sensing modalities of the robots, and focus on the design of a scalable control strategy for the robot team. The first part of the dissertation studies the active information acquisition problem in the special case of linear Gaussian sensing and mobility models. We show that the classical principle of separation between estimation and control holds in this case. It enables us to reduce the original stochastic optimal control problem to a deterministic version and to provide an optimal centralized solution. Unfortunately, the complexity of obtaining the optimal solution scales exponentially with the length of the planning horizon and the number of robots. We develop approximation algorithms to manage the complexity in both of these factors and provide theoretical performance guarantees. Applications in gas concentration mapping, joint localization and vehicle tracking in sensor networks, and active multi-robot localization and mapping are presented. Coupled with linearization and model predictive control, our algorithms can even generate adaptive control policies for nonlinear sensing and mobility models. Linear Gaussian information seeking, however, cannot be applied directly in the presence of sensing nuisances such as missed detections, false alarms, and ambiguous data association or when some sensor observations are discrete (e.g., object classes, medical alarms) or, even worse, when the sensing and target models are entirely unknown. The second part of the dissertation considers these complications in the context of two applications: active localization from semantic observations (e.g, recognized objects) and radio signal source seeking. The complexity of the target inference problem forces us to resort to greedy planning of the sensor trajectories. Non-greedy closed-loop information acquisition with general discrete models is achieved in the final part of the dissertation via dynamic programming and Monte Carlo tree search algorithms. Applications in active object recognition and pose estimation are presented. The techniques developed in this thesis offer an effective and scalable approach for controlled information acquisition with multiple sensing robots and have broad applications to environmental monitoring, search and rescue, security and surveillance, localization and mapping, precision agriculture, and structure inspection

Sensor Path Planning for Emitter Localization

The localization of a radio frequency (RF) emitter is relevant in many military and civilian applications. The recent decade has seen a rapid progress in the development of small and mobile unmanned aerial vehicles (UAVs), which offer a way to perform emitter localization autonomously. The path a UAV travels influences the localization significantly, making path planning an important part of a mobile emitter localization system. The topic of this thesis is path planning for a UAV that uses bearing measurements to localize a stationary emitter. Using a directional antenna, the direction towards the target can be determined by the UAV rotating around its own vertical axis. During this rotation the UAV is required to remain at the same position, which induces a trade-off between movement and measurement that influences the optimal trajectories. This thesis derives a novel path planning algorithm for localizing an emitter with a UAV. It improves the current state of the art by providing a localization with defined accuracy in a shorter amount of time compared to other algorithms in simulations. The algorithm uses the policy rollout principle to perform a nonmyopic planning and to incorporate the uncertainty of the estimation process into its decision. The concept of an action selection algorithm for policy rollout is introduced, which allows the use of existing optimization algorithms to effectively search the action space. Multiple action selection algorithms are compared to optimize the speed of the path planning algorithm. Similarly, to reduce computational demand, an adaptive grid-based localizer has been developed. To evaluate the algorithm an experimental system has been built and the algorithm was tested on this system. Based on initial experiments, the path planning algorithm has been modified, including a minimal distance to the emitter and an outlier detection step. The resulting algorithm shows promising results in experimental flights

Relaying in the Internet of Things (IoT): A Survey

The deployment of relays between Internet of Things (IoT) end devices and gateways can improve link quality. In cellular-based IoT, relays have the potential to reduce base station overload. The energy expended in single-hop long-range communication can be reduced if relays listen to transmissions of end devices and forward these observations to gateways. However, incorporating relays into IoT networks faces some challenges. IoT end devices are designed primarily for uplink communication of small-sized observations toward the network; hence, opportunistically using end devices as relays needs a redesign of both the medium access control (MAC) layer protocol of such end devices and possible addition of new communication interfaces. Additionally, the wake-up time of IoT end devices needs to be synchronized with that of the relays. For cellular-based IoT, the possibility of using infrastructure relays exists, and noncellular IoT networks can leverage the presence of mobile devices for relaying, for example, in remote healthcare. However, the latter presents problems of incentivizing relay participation and managing the mobility of relays. Furthermore, although relays can increase the lifetime of IoT networks, deploying relays implies the need for additional batteries to power them. This can erode the energy efficiency gain that relays offer. Therefore, designing relay-assisted IoT networks that provide acceptable trade-offs is key, and this goes beyond adding an extra transmit RF chain to a relay-enabled IoT end device. There has been increasing research interest in IoT relaying, as demonstrated in the available literature. Works that consider these issues are surveyed in this paper to provide insight into the state of the art, provide design insights for network designers and motivate future research directions