Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Multi-Robot Coalition Formation for Distributed Area Coverage

The problem of distributed area coverage using multiple mobile robots is an important problem in distributed multi-robot sytems. Multi-robot coverage is encountered in many real world applications, including unmanned search & rescue, aerial reconnaissance, robotic demining, inspection of engineering structures, and automatic lawn mowing. To achieve optimal coverage, robots should move in an efficient manner and reduce repeated coverage of the same region that optimizes a certain performance metric such as the amount of time or energy expended by the robots. This dissertation especially focuses on using mini-robots with limited capabilities, such as low speed of the CPU and limited storage of the memory, to fulfill the efficient area coverage task. Previous research on distributed area coverage use offline or online path planning algorithms to address this problem. Some of the existing approaches use behavior-based algorithms where each robot implements simple rules and the interaction between robots manifests in the global objective of overall coverage of the environment. Our work extends this line of research using an emergent, swarming based technique where robots use partial coverage histories from themselves as well as other robots in their vicinity to make local decisions that attempt to ensure overall efficient area coverage. We have then extended this technique in two directions. First, we have integreated the individual-robot, swarming-based technique for area coverage to teams of robots that move in formation to perform area coverage more efficiently than robots that move individually. Then we have used a team formation technique from coalition game theory, called Weighted Voting Game (WVG) to handle situations where a team moving in formation while performing area coverage has to dynamically reconfigure into sub-teams or merge with other teams, to continue the area coverage efficiently. We have validated our techniques by testing them on accurate models of e-puck robots in the Webots robot simulation platform, as well as on physical e-puck robots

Risk-Aware Planning for Sensor Data Collection

With the emergence of low-cost unmanned air vehicles, civilian and military organizations are quickly identifying new applications for affordable, large-scale collectives to support and augment human efforts via sensor data collection. In order to be viable, these collectives must be resilient to the risk and uncertainty of operating in real-world environments. Previous work in multi-agent planning has avoided planning for the loss of agents in environments with risk. In contrast, this dissertation presents a problem formulation that includes the risk of losing agents, the effect of those losses on the mission being executed, and provides anticipatory planning algorithms that consider risk. We conduct a thorough analysis of the effects of risk on path-based planning, motivating new solution methods. We then use hierarchical clustering to generate risk-aware plans for a variable number of agents, outperforming traditional planning methods. Next, we provide a mechanism for distributed negotiation of stable plans, utilizing coalitional game theory to provide cost allocation methods that we prove to be fair and stable. Centralized planning with redundancy is then explored, planning for parallel task completion to mitigate risk and provide further increased expected value. Finally, we explore the role of cost uncertainty as additional source of risk, using bi-objective optimization to generate sets of alternative plans. We demonstrate the capability of our algorithms on randomly generated problem instances, showing an improvement over traditional multi-agent planning methods as high as 500% on very large problem instances

Assessing optimal water quality monitoring network in road construction using integrated information-theoretic techniques

Author´s accepted manuscript.The environmental impacts of road construction on the aquatic environment necessitate the monitoring of receiving water quality. The main contribution of the paper is developing a feasible methodology for spatial optimization of the water quality monitoring network (WQMN) in surface water during road construction using the field data. First, using the Canadian Council of Ministers of the Environment (CCME) method, the water quality index (WQI) was computed in each potential monitoring station during construction. Then, the integrated form of the information-theoretic techniques consists of the transinformation entropy (TE), and the value of information (VOI) were calculated for the potential stations. To achieve the optimal WQMNs, the Non-dominated Sorting Genetic Algorithm II and III (NSGA-II, and III) based multi-objective optimization models were developed considering three objective functions, including i) minimizing the number of stations, ii) maximizing the VOI in the selected network, and iii) minimizing redundant information for the selected nodes. Finally, three multi-criteria decision-making models, including Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), Preference Ranking Organisation Method for Enrichment Evaluations (PROMETHEE), and Analytical Hierarchy Process (AHP) were utilized for choosing the best alternative among Pareto optimal solutions considering various weighing scenarios assigned to criteria. The applicability of the presented methodology was assessed in a 22 km long road construction site in southern Norway. The results deliver significant knowledge for decision-makers on establishing a robust WQMN in surface water during road construction projects.publishedVersio

Link Resource Adaptation for Multiantenna Bit-Interleaved Coded Multicarrier Systems

n/

Stochastic rule-based decision support system for reliability redundancy allocation problem

Reliability Redundancy Allocation (RRA) is one of the most important problems facing the managers to improve the systems performance. In the most RRA models, presented in the literature components’ reliability used to be assumed as an exact value in (0,1) interval, while various factors might affect components’ reliability and change it over time. Therefore, components reliability values should be considered as uncertain parameters. In this paper, by developing a discrete - continuous inference system, an optimization - oriented decision support system is proposed considering the components’ reliability as stochastic variables. Proposed DSS uses stochastic if - then rules to infer optimum or near optimum values for the decision variables as well as the objective function. Finally, In order to evaluate the efficiency of the proposed system, several examples are provided. Comparison of the inferred results with the optimal values shows the very good performance of the developed stochastic decision support system

Heterogeneous Sensor Signal Processing for Inference with Nonlinear Dependence

Inferring events of interest by fusing data from multiple heterogeneous sources has been an interesting and important topic in recent years. Several issues related to inference using heterogeneous data with complex and nonlinear dependence are investigated in this dissertation. We apply copula theory to characterize the dependence among heterogeneous data. In centralized detection, where sensor observations are available at the fusion center (FC), we study copula-based fusion. We design detection algorithms based on sample-wise copula selection and mixture of copulas model in different scenarios of the true dependence. The proposed approaches are theoretically justified and perform well when applied to fuse acoustic and seismic sensor data for personnel detection. Besides traditional sensors, the access to the massive amount of social media data provides a unique opportunity for extracting information about unfolding events. We further study how sensor networks and social media complement each other in facilitating the data-to-decision making process. We propose a copula-based joint characterization of multiple dependent time series from sensors and social media. As a proof-of-concept, this model is applied to the fusion of Google Trends (GT) data and stock/flu data for prediction, where the stock/flu data serves as a surrogate for sensor data. In energy constrained networks, local observations are compressed before they are transmitted to the FC. In these cases, conditional dependence and heterogeneity complicate the system design particularly. We consider the classification of discrete random signals in Wireless Sensor Networks (WSNs), where, for communication efficiency, only local decisions are transmitted. We derive the necessary conditions for the optimal decision rules at the sensors and the FC by introducing a hidden random variable. An iterative algorithm is designed to search for the optimal decision rules. Its convergence and asymptotical optimality are also proved. The performance of the proposed scheme is illustrated for the distributed Automatic Modulation Classification (AMC) problem. Censoring is another communication efficient strategy, in which sensors transmit only informative observations to the FC, and censor those deemed uninformative . We design the detectors that take into account the spatial dependence among observations. Fusion rules for censored data are proposed with continuous and discrete local messages, respectively. Their computationally efficient counterparts based on the key idea of injecting controlled noise at the FC before fusion are also investigated. In this thesis, with heterogeneous and dependent sensor observations, we consider not only inference in parallel frameworks but also the problem of collaborative inference where collaboration exists among local sensors. Each sensor forms coalition with other sensors and shares information within the coalition, to maximize its inference performance. The collaboration strategy is investigated under a communication constraint. To characterize the influence of inter-sensor dependence on inference performance and thus collaboration strategy, we quantify the gain and loss in forming a coalition by introducing the copula-based definitions of diversity gain and redundancy loss for both estimation and detection problems. A coalition formation game is proposed for the distributed inference problem, through which the information contained in the inter-sensor dependence is fully explored and utilized for improved inference performance

Optimization of systems reliability by metaheuristic approach

The application of metaheuristic approaches in addressing the reliability of systems through optimization is of greater interest to researchers and designers in recent years. Reliability optimization has become an essential part of the design and operation of largescale manufacturing systems. This thesis addresses the optimization of system-reliability for series–parallel systems to solve redundant, continuous, and combinatorial optimization problems in reliability engineering by using metaheuristic approaches (MAs). The problem is to select the best redundancy strategy, component, and redundancy level for each subsystem to maximize the system reliability under system-level constraints. This type of problem involves the selection of components with multiple choices and redundancy levels that yield the maximum benefits, and it is subject to the cost and weight constraints at the system level. These are very common and realistic problems faced in the conceptual design of numerous engineering systems. The development of efficient solutions to these problems is becoming progressively important because mechanical systems are becoming increasingly complex, while development plans are decreasing in size and reliability requirements are rapidly changing and becoming increasingly difficult to adhere to. An optimal design solution can be obtained very frequently and more quickly by using genetic algorithm redundancy allocation problems (GARAPs). In general, redundancy allocation problems (RAPs) are difficult to solve for real cases, especially in large-scale situations. In this study, the reliability optimization of a series–parallel by using a genetic algorithm (GA) and statistical analysis is considered. The approach discussed herein can be applied to address the challenges in system reliability that includes redundant numbers of carefully chosen modules, overall cost, and overall weight. Most related studies have focused only on the single-objective optimization of RAP. Multiobjective optimization has not yet attracted much attention. This research project examines the multiobjective situation by focusing on multiobjective formulation, which is useful in maximizing system reliability while simultaneously minimizing system cost and weight to solve the RAP. The present study applies a methodology for optimizing the reliability of a series–parallel system based on multiobjective optimization and multistate reliability by using a hybrid GA and a fuzzy function. The study aims to determine the strategy for selecting the degree of redundancy for every subsystem to exploit the general system reliability depending on the overall cost and weight limitations. In addition, the outcomes of the case study for optimizing the reliability of the series–parallel system are presented, and the relationships with previously investigated phenomena are presented to determine the performance of the GA under review. Furthermore, this study established a new metaheuristic-based technique for resolving multiobjective optimization challenges, such as the common reliability redundancy allocation problem. Additionally, a new simulation process was developed to generate practical tools for designing reliable series–parallel systems. Hence, metaheuristic methods were applied for solving such difficult and complex problems. In addition, metaheuristics provide a useful compromise between the amount of computation time required and the quality of the approximated solution space. The industrial challenges include the maximization of system reliability subject to limited system cost and weight, minimization of system weight subject to limited system cost and the system reliability requirements and increasing of quality components through optimization and system reliability. Furthermore, a real-life situation research on security control of a gas turbine in the overspeed state was explored in this study with the aim of verifying the proposed algorithm from the context of system optimization