Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Optimisation of Mobile Communication Networks - OMCO NET

The mini conference “Optimisation of Mobile Communication Networks” focuses on advanced methods for search and optimisation applied to wireless communication networks. It is sponsored by Research & Enterprise Fund Southampton Solent University. The conference strives to widen knowledge on advanced search methods capable of optimisation of wireless communications networks. The aim is to provide a forum for exchange of recent knowledge, new ideas and trends in this progressive and challenging area. The conference will popularise new successful approaches on resolving hard tasks such as minimisation of transmit power, cooperative and optimal routing

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Probabilistic approaches to the design of wireless ad hoc and sensor networks

The emerging wireless technologies has made ubiquitous wireless access a reality and enabled wireless systems to support a large variety of applications. Since the wireless self-configuring networks do not require infrastructure and promise greater flexibility and better coverage, wireless ad hoc and sensor networks have been under intensive research. It is believed that wireless ad hoc and sensor networks can become as important as the Internet. Just as the Internet allows access to digital information anywhere, ad hoc and sensor networks will provide remote interaction with the physical world. Dynamics of the object distribution is one of the most important features of the wireless ad hoc and sensor networks. This dissertation deals with several interesting estimation and optimization problems on the dynamical features of ad hoc and sensor networks. Many demands in application, such as reliability, power efficiency and sensor deployment, of wireless ad hoc and sensor network can be improved by mobility estimation and/or prediction. In this dissertation, we study several random mobility models, present a mobility prediction methodology, which relies on the analysis of the moving patterns of the mobile objects. Through estimating the future movement of objects and analyzing the tradeoff between the estimation cost and the quality of reliability, the optimization of tracking interval for sensor networks is presented. Based on the observation on the location and movement of objects, an optimal sensor placement algorithm is proposed by adaptively learn the dynamical object distribution. Moreover, dynamical boundary of mass objects monitored in a sensor network can be estimated based on the unsupervised learning of the distribution density of objects. In order to provide an accurate estimation of mobile objects, we first study several popular mobility models. Based on these models, we present some mobility prediction algorithms accordingly, which are capable of predicting the moving trajectory of objects in the future. In wireless self-configuring networks, an accurate estimation algorithm allows for improving the link reliability, power efficiency, reducing the traffic delay and optimizing the sensor deployment. The effects of estimation accuracy on the reliability and the power consumption have been studied and analyzed. A new methodology is proposed to optimize the reliability and power efficiency by balancing the trade-off between the quality of performance and estimation cost. By estimating and predicting the mass objects\u27 location and movement, the proposed sensor placement algorithm demonstrates a siguificant improvement on the detection of mass objects with nearmaximal detection accuracy. Quantitative analysis on the effects of mobility estimation and prediction on the accuracy of detection by sensor networks can be conducted with recursive EM algorithms. The future work includes the deployment of the proposed concepts and algorithms into real-world ad hoc and sensor networks

Hybrid performance modelling of opportunistic networks

We demonstrate the modelling of opportunistic networks using the process algebra stochastic HYPE. Network traffic is modelled as continuous flows, contact between nodes in the network is modelled stochastically, and instantaneous decisions are modelled as discrete events. Our model describes a network of stationary video sensors with a mobile ferry which collects data from the sensors and delivers it to the base station. We consider different mobility models and different buffer sizes for the ferries. This case study illustrates the flexibility and expressive power of stochastic HYPE. We also discuss the software that enables us to describe stochastic HYPE models and simulate them.Comment: In Proceedings QAPL 2012, arXiv:1207.055

Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and Applications

Wireless sensor networks monitor dynamic environments that change rapidly over time. This dynamic behavior is either caused by external factors or initiated by the system designers themselves. To adapt to such conditions, sensor networks often adopt machine learning techniques to eliminate the need for unnecessary redesign. Machine learning also inspires many practical solutions that maximize resource utilization and prolong the lifespan of the network. In this paper, we present an extensive literature review over the period 2002-2013 of machine learning methods that were used to address common issues in wireless sensor networks (WSNs). The advantages and disadvantages of each proposed algorithm are evaluated against the corresponding problem. We also provide a comparative guide to aid WSN designers in developing suitable machine learning solutions for their specific application challenges.Comment: Accepted for publication in IEEE Communications Surveys and Tutorial