Reinforcement Learning in Multiple-UAV Networks: Deployment and Movement Design

A novel framework is proposed for quality of experience (QoE)-driven deployment and dynamic movement of multiple unmanned aerial vehicles (UAVs). The problem of joint non-convex three-dimensional (3D) deployment and dynamic movement of the UAVs is formulated for maximizing the sum mean opinion score (MOS) of ground users, which is proved to be NP-hard. In the aim of solving this pertinent problem, a three-step approach is proposed for attaining 3D deployment and dynamic movement of multiple UAVs. Firstly, genetic algorithm based K-means (GAK-means) algorithm is utilized for obtaining the cell partition of the users. Secondly, Q-learning based deployment algorithm is proposed, in which each UAV acts as an agent, making their own decision for attaining 3D position by learning from trial and mistake. In contrast to conventional genetic algorithm based learning algorithms, the proposed algorithm is capable of training the direction selection strategy offline. Thirdly, Q-learning based movement algorithm is proposed in the scenario that the users are roaming. The proposed algorithm is capable of converging to an optimal state. Numerical results reveal that the proposed algorithms show a fast convergence rate after a small number of iterations. Additionally, the proposed Q-learning based deployment algorithm outperforms K-means algorithms and Iterative-GAKmean (IGK) algorithms with a low complexity

A Gentle Introduction to Deep Learning in Medical Image Processing

This paper tries to give a gentle introduction to deep learning in medical image processing, proceeding from theoretical foundations to applications. We first discuss general reasons for the popularity of deep learning, including several major breakthroughs in computer science. Next, we start reviewing the fundamental basics of the perceptron and neural networks, along with some fundamental theory that is often omitted. Doing so allows us to understand the reasons for the rise of deep learning in many application domains. Obviously medical image processing is one of these areas which has been largely affected by this rapid progress, in particular in image detection and recognition, image segmentation, image registration, and computer-aided diagnosis. There are also recent trends in physical simulation, modelling, and reconstruction that have led to astonishing results. Yet, some of these approaches neglect prior knowledge and hence bear the risk of producing implausible results. These apparent weaknesses highlight current limitations of deep learning. However, we also briefly discuss promising approaches that might be able to resolve these problems in the future.Comment: Accepted by Journal of Medical Physics; Final Version after revie

Coordination approaches and systems - part I : a strategic perspective

This is the first part of a two-part paper presenting a fundamental review and summary of research of design coordination and cooperation technologies. The theme of this review is aimed at the research conducted within the decision management aspect of design coordination. The focus is therefore on the strategies involved in making decisions and how these strategies are used to satisfy design requirements. The paper reviews research within collaborative and coordinated design, project and workflow management, and, task and organization models. The research reviewed has attempted to identify fundamental coordination mechanisms from different domains, however it is concluded that domain independent mechanisms need to be augmented with domain specific mechanisms to facilitate coordination. Part II is a review of design coordination from an operational perspective

Applications of Multi-Agent Slime Mould Computing

The giant single-celled slime mould Physarum polycephalum has inspired rapid develop- ments in unconventional computing substrates since the start of this century. This is primarily due to its simple component parts and the distributed nature of the computation which it approximates during its growth, foraging and adaptation to a changing environment. Slime mould functions as a living embodied computational material which can be influenced (or pro- grammed) by the placement of external stimuli. The goal of exploiting this material behaviour for unconventional computation led to the development of a multi-agent approach to the ap- proximation of slime mould behaviour. The basis of the model is a simple dynamical pattern formation mechanism which exhibits self-organised formation and subsequent adaptation of collective transport networks. The system exhibits emergent properties such as relaxation and minimisation and it can be considered as a virtual computing material, influenced by the external application of spatial concentration gradients. In this paper we give an overview of this multi-agent approach to unconventional computing. We describe its computational mechanisms and different generic application domains, together with concrete example ap- plications of material computation. We examine the potential exploitation of the approach for computational geometry, path planning, combinatorial optimisation, data smoothing and statistical applications

New technologies for urban designers: the VENUE project

In this report, we first outline the basic idea of VENUE. This involves developing digital tools froma foundation of geographic information systems (GIS) software which we then apply to urbandesign, a subject area and profession which has little tradition in using such tools. Our project wasto develop two types of tool, namely functional analysis based on embedding models of movementin local environments into GIS based on ideas from the field of space syntax; and secondlyfashioning these ideas in a wider digital context in which the entire range of GIS technologies werebrought to bear at the local scale. By local scale, we mean the representation of urban environmentsfrom about 1: 500 to around 1: 2500

6G White Paper on Machine Learning in Wireless Communication Networks

The focus of this white paper is on machine learning (ML) in wireless communications. 6G wireless communication networks will be the backbone of the digital transformation of societies by providing ubiquitous, reliable, and near-instant wireless connectivity for humans and machines. Recent advances in ML research has led enable a wide range of novel technologies such as self-driving vehicles and voice assistants. Such innovation is possible as a result of the availability of advanced ML models, large datasets, and high computational power. On the other hand, the ever-increasing demand for connectivity will require a lot of innovation in 6G wireless networks, and ML tools will play a major role in solving problems in the wireless domain. In this paper, we provide an overview of the vision of how ML will impact the wireless communication systems. We first give an overview of the ML methods that have the highest potential to be used in wireless networks. Then, we discuss the problems that can be solved by using ML in various layers of the network such as the physical layer, medium access layer, and application layer. Zero-touch optimization of wireless networks using ML is another interesting aspect that is discussed in this paper. Finally, at the end of each section, important research questions that the section aims to answer are presented

Surrogate-based toll optimization in a large-scale heterogeneously congested network

Toll optimization in a large-scale dynamic traffic network is typically characterized by an expensive-to-evaluate objective function. In this paper, we propose two toll level problems (TLPs) integrated with a large-scale simulation-based dynamic traffic assignment (DTA) model of Melbourne, Australia. The first TLP aims to control the pricing zone (PZ) through a time-varying joint distance and delay toll (JDDT) such that the network fundamental diagram (NFD) of the PZ does not enter the congested regime. The second TLP is built upon the first TLP by further considering the minimization of the heterogeneity of congestion distribution in the PZ. To solve the two TLPs, a computationally efficient surrogate-based optimization method, i.e., regressing kriging (RK) with expected improvement (EI) sampling, is applied to approximate the simulation input-output mapping, which can balance well between local exploitation and global exploration. Results show that the two optimal TLP solutions reduce the average travel time in the PZ (entire network) by 29.5% (1.4%) and 21.6% (2.5%), respectively. Reducing the heterogeneity of congestion distribution achieves higher network flows in the PZ and a lower average travel time or a larger total travel time saving in the entire network.Comment: 16 pages, 7 figure

A Review of Reinforcement Learning for Autonomous Building Energy Management

The area of building energy management has received a significant amount of interest in recent years. This area is concerned with combining advancements in sensor technologies, communications and advanced control algorithms to optimize energy utilization. Reinforcement learning is one of the most prominent machine learning algorithms used for control problems and has had many successful applications in the area of building energy management. This research gives a comprehensive review of the literature relating to the application of reinforcement learning to developing autonomous building energy management systems. The main direction for future research and challenges in reinforcement learning are also outlined.Comment: 17 pages, 3 figure

Energy and Latency Aware Application Mapping Algorithm & Optimization for Homogeneous 3D Network on Chip

Energy efficiency is one of the most critical issue in design of System on Chip. In Network On Chip (NoC) based system, energy consumption is influenced dramatically by mapping of Intellectual Property (IP) which affect the performance of the system. In this paper we test the antecedently extant proposed algorithms and introduced a new energy proficient algorithm stand for 3D NoC architecture. In addition a hybrid method has also been implemented using bioinspired optimization (particle swarm optimization) technique. The proposed algorithm has been implemented and evaluated on randomly generated benchmark and real life application such as MMS, Telecom and VOPD. The algorithm has also been tested with the E3S benchmark and has been compared with the existing algorithm (spiral and crinkle) and has shown better reduction in the communication energy consumption and shows improvement in the performance of the system. Comparing our work with spiral and crinkle, experimental result shows that the average reduction in communication energy consumption is 19% with spiral and 17% with crinkle mapping algorithms, while reduction in communication cost is 24% and 21% whereas reduction in latency is of 24% and 22% with spiral and crinkle. Optimizing our work and the existing methods using bio-inspired technique and having the comparison among them an average energy reduction is found to be of 18% and 24%.Comment: 15 pages, 11 figure, CCSEA 201