Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Intelligent Design for Real Time Networked Multi-Agent Systems

Past decade has witnessed an unprecedented growth in reasearch for Unmanned Aerial Vehicles (UAVs) both in military and nonmilitary fronts. They have become ubiquitous in almost every military operations which includes domestic and overseas missions. With rapidly advancing technology, open source nature of the flight controllers, and significantly lesser costs than before, companies around the world are delving into UAV market as one of the upcoming lucrative investments. Companies like Amazon Inc., Dominos Pizza Inc. have had some successful test runs which again solidifies the research opportunities. Delivery services and recreational uses seems to have increased in the past 3-4 years which has let the Federal Aviation Administration to update their rules and regulations. Mapping, Surveying and search/rescue mission are some of the applications of UAVs that are most appealing. Making these applications airborne cuts the time and cost at considerable and affordable levels. Using UAVs for operations has advantages in both response time and need of manpower compared to piloted aricrafts. Obtaining prior information of a person/people in distress can become a deciding factor for a successful mission. It can help in making critical decision as which location or type of helicopter / vehicle to be used for extraction, equipment to bring and how many crew members that are needed. The idea here is to make this system of UAVs automated to coordinate with each other without human intervention (other than high level commands like takeoff and land). Researchers and Military experts have recognized the use of drones for search and rescue missions to be of utmost importance. Year 2016 saw a first of its kind UAV search and rescue symposium held in Nevada. The objective was to give a platform for UAV enthusiasts and researchers and share their experiences and concerns while using UAVs as first responders. The biggest drawback of using an aerial vehicle for inspection/search/rescue mission is its airborne time. The batteries used are big and heavy which increases the weight and decreases the flight time. One can go about solving this issue by using a swarm of UAVs which would inspect/search a given area in less amount of time. This has advantage in both response time and need for lesser man power.The main challenges for Multiple Drone Control (MDC) includes 1) Address the periodic sampling frequency issue of information of assets so as to maintain stability; 2) Optimize the communication channel while providing minimum Quality of Service (QoS); 3) Optimal control strategy which includes non-linearity in state space model; 4) Optimal control in presence of uncertainties; 5) Admitting new agents for dynamic agents in the Networked Multi-Agent System (MAS) Scenario.This dissertation aims at building a hardware and a software platform for communication of multiple UAVs upon which additional control algorithms can be implementated. It starts with building a DJI S1000 octacopter from the ground up. The components used are specified in the following sections. The idea here is to make a drone that can autonomously travel to specified location with safety features like geofencing and land on emergency situations. The user has to provide the necessary commands like GPS locations and takeoff/land commands via a Radio Controller (RC) remote. At any point of the flight, the UAV should be able to receive new commands from the ground control stations (GCS). After successful implementation, the UAV would not be restricted to the range of RC remote. It would be able to travel greater distances given the GPS signal remains operational in the field. This is possible at a global scale with limitation of only the batteries and flight time

Distributed Delay-Tolerant Strategies for Equality-Constraint Sum-Preserving Resource Allocation

This paper proposes two nonlinear dynamics to solve constrained distributed optimization problem for resource allocation over a multi-agent network. In this setup, coupling constraint refers to resource-demand balance which is preserved at all-times. The proposed solutions can address various model nonlinearities, for example, due to quantization and/or saturation. Further, it allows to reach faster convergence or to robustify the solution against impulsive noise or uncertainties. We prove convergence over weakly connected networks using convex analysis and Lyapunov theory. Our findings show that convergence can be reached for general sign-preserving odd nonlinearity. We further propose delay-tolerant mechanisms to handle general bounded heterogeneous time-varying delays over the communication network of agents while preserving all-time feasibility. This work finds application in CPU scheduling and coverage control among others. This paper advances the state-of-the-art by addressing (i) possible nonlinearity on the agents/links, meanwhile handling (ii) resource-demand feasibility at all times, (iii) uniform-connectivity instead of all-time connectivity, and (iv) possible heterogeneous and time-varying delays. To our best knowledge, no existing work addresses contributions (i)-(iv) altogether. Simulations and comparative analysis are provided to corroborate our contributions

On the Design of Campus Parking Systems with QoS guarantees

Parking spaces are resources that can be pooled together and shared, especially when there are complementary day-time and night-time users. We answer two design questions. First, given a quality of service requirement, how many spaces should be set aside as contingency during day-time for night-time users? Next, how can we replace the first-come-first-served access method by one that aims at optimal efficiency while keeping user preferences private