Observing the clouds : a survey and taxonomy of cloud monitoring

This research was supported by a Royal Society Industry Fellowship and an Amazon Web Services (AWS) grant. Date of Acceptance: 10/12/2014Monitoring is an important aspect of designing and maintaining large-scale systems. Cloud computing presents a unique set of challenges to monitoring including: on-demand infrastructure, unprecedented scalability, rapid elasticity and performance uncertainty. There are a wide range of monitoring tools originating from cluster and high-performance computing, grid computing and enterprise computing, as well as a series of newer bespoke tools, which have been designed exclusively for cloud monitoring. These tools express a number of common elements and designs, which address the demands of cloud monitoring to various degrees. This paper performs an exhaustive survey of contemporary monitoring tools from which we derive a taxonomy, which examines how effectively existing tools and designs meet the challenges of cloud monitoring. We conclude by examining the socio-technical aspects of monitoring, and investigate the engineering challenges and practices behind implementing monitoring strategies for cloud computing.Publisher PDFPeer reviewe

Real Time Control for Intelligent 6G Networks

The benefits of telemetry for optical networking have been shown in the literature, and several telemetry architectures have been defined. In general, telemetry data is collected from observation points in the devices and sent to a central system running besides the Software Defined Networking (SDN) controller. In this project, we try to develop a telemetry architecture that supports intelligent data aggregation and nearby data collection. Several frameworks and technologies have been explored to ensure that they fit well into the architecture's composition. A description of these different technologies is presented in this work, along with a comparison between their main features and downsides. Some intelligent techniques, aka. Algorithms have been stated and tested within architecture, showing their benefits by reducing the amount of data processed. In the design of this architecture, the main issues related to distributed systems have been faced, and some initial solutions have been proposed. In particular, several security solutions have been explored to deal with threats but also with scalability and performance issues, trying to find a balance between performance and security. Finally, two use cases are presented, showing a real implementation of the architecture that has been presented at conferences and validated within the project's development

Real-time big data processing for anomaly detection : a survey

The advent of connected devices and omnipresence of Internet have paved way for intruders to attack networks, which leads to cyber-attack, financial loss, information theft in healthcare, and cyber war. Hence, network security analytics has become an important area of concern and has gained intensive attention among researchers, off late, specifically in the domain of anomaly detection in network, which is considered crucial for network security. However, preliminary investigations have revealed that the existing approaches to detect anomalies in network are not effective enough, particularly to detect them in real time. The reason for the inefficacy of current approaches is mainly due the amassment of massive volumes of data though the connected devices. Therefore, it is crucial to propose a framework that effectively handles real time big data processing and detect anomalies in networks. In this regard, this paper attempts to address the issue of detecting anomalies in real time. Respectively, this paper has surveyed the state-of-the-art real-time big data processing technologies related to anomaly detection and the vital characteristics of associated machine learning algorithms. This paper begins with the explanation of essential contexts and taxonomy of real-time big data processing, anomalous detection, and machine learning algorithms, followed by the review of big data processing technologies. Finally, the identified research challenges of real-time big data processing in anomaly detection are discussed. © 2018 Elsevier Lt