On the design and performance evaluation of automatic traffic report generation systems with huge data volumes

In this paper, we analyze the performance issues involved in the generation of automated traffic reports for large IT infrastructures. Such reports allow the IT manager to proactively detect possible abnormal situations and roll out the corresponding corrective actions. With the ever‐increasing bandwidth of current networks, the design of automated traffic report generation systems is very challenging. In a first step, the huge volumes of collected traffic are transformed into enriched flow records obtained from diverse collectors and dissectors. Then, such flow records, along with time series obtained from the raw traffic, are further processed to produce a usable report. As will be shown, the data volume in flow records turns out to be very large as well and requires careful selection of the key performance indicators (KPIs) to be included in the report. In this regard, we discuss the use of high‐level languages versus low‐level approaches, in terms of speed and versatility. Furthermore, our design approach is targeted for rapid development in commodity hardware, which is essential to cost‐effectively tackle demanding traffic analysis scenarios. Actually, the paper shows feasibility of delivering a large number of KPIs, as will be detailed later, for several TBytes of traffic per day using a commodity hardware architecture and high‐level languagesThis work has been partially supported by the Spanish Ministry of Economy and Competitiveness and the European Regional Development Fund under the projects TRÁFICA (MINECO/FEDER TEC2015‐69417‐C2‐1‐R) and Procesado Inteligente de Tráfico (MINECO/FEDER TEC2015‐69417‐C2‐2‐

Building an Emulation Environment for Cyber Security Analyses of Complex Networked Systems

Computer networks are undergoing a phenomenal growth, driven by the rapidly increasing number of nodes constituting the networks. At the same time, the number of security threats on Internet and intranet networks is constantly growing, and the testing and experimentation of cyber defense solutions requires the availability of separate, test environments that best emulate the complexity of a real system. Such environments support the deployment and monitoring of complex mission-driven network scenarios, thus enabling the study of cyber defense strategies under real and controllable traffic and attack scenarios. In this paper, we propose a methodology that makes use of a combination of techniques of network and security assessment, and the use of cloud technologies to build an emulation environment with adjustable degree of affinity with respect to actual reference networks or planned systems. As a byproduct, starting from a specific study case, we collected a dataset consisting of complete network traces comprising benign and malicious traffic, which is feature-rich and publicly available

Reinforcement machine learning for predictive analytics in smart cities

The digitization of our lives cause a shift in the data production as well as in the required data management. Numerous nodes are capable of producing huge volumes of data in our everyday activities. Sensors, personal smart devices as well as the Internet of Things (IoT) paradigm lead to a vast infrastructure that covers all the aspects of activities in modern societies. In the most of the cases, the critical issue for public authorities (usually, local, like municipalities) is the efficient management of data towards the support of novel services. The reason is that analytics provided on top of the collected data could help in the delivery of new applications that will facilitate citizens’ lives. However, the provision of analytics demands intelligent techniques for the underlying data management. The most known technique is the separation of huge volumes of data into a number of parts and their parallel management to limit the required time for the delivery of analytics. Afterwards, analytics requests in the form of queries could be realized and derive the necessary knowledge for supporting intelligent applications. In this paper, we define the concept of a Query Controller ( QC ) that receives queries for analytics and assigns each of them to a processor placed in front of each data partition. We discuss an intelligent process for query assignments that adopts Machine Learning (ML). We adopt two learning schemes, i.e., Reinforcement Learning (RL) and clustering. We report on the comparison of the two schemes and elaborate on their combination. Our aim is to provide an efficient framework to support the decision making of the QC that should swiftly select the appropriate processor for each query. We provide mathematical formulations for the discussed problem and present simulation results. Through a comprehensive experimental evaluation, we reveal the advantages of the proposed models and describe the outcomes results while comparing them with a deterministic framework

Automatic Methodology for Multi-modal Trip Generation with Roadside LiDAR

Transportation planning based on historical data and methods has major limitations. Trip data canbe useful to increase the transportation safety of the specific sites and the process and programming purposes. One of the challenges in this regard is data collecting to gain an accurate analysis of land use development. The previous methods of data gathering such as human observational data counting and automatic methods like pneumatic tubes and video camera suffers some limitations that affect the accuracy of trip analysis which cause over mitigating or set some wrong rules and regulations. Light Detection and Ranging (LiDAR) sensing is a powerful tool that has been vastly used for mapping, safety, and medical applications. [1] Also, its application in transportation has drawn attention in recent years. However, LiDAR sense is yet to be further explored in trip generation. This study is an initial attempt to: 1) perform a LiDAR-based trip generation data gathering for a local area in midtown, Reno, and 2) analyze the resulting data based on the GIS software to develop a systematic plan for the case study and beyond

A survey on online monitoring approaches of computer-based systems

This report surveys forms of online data collection that are in current use (as well as being the subject of research to adapt them to changing technology and demands), and can be used as inputs to assessment of dependability and resilience, although they are not primarily meant for this use

CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research

Big data analytics:Computational intelligence techniques and application areas

Big Data has significant impact in developing functional smart cities and supporting modern societies. In this paper, we investigate the importance of Big Data in modern life and economy, and discuss challenges arising from Big Data utilization. Different computational intelligence techniques have been considered as tools for Big Data analytics. We also explore the powerful combination of Big Data and Computational Intelligence (CI) and identify a number of areas, where novel applications in real world smart city problems can be developed by utilizing these powerful tools and techniques. We present a case study for intelligent transportation in the context of a smart city, and a novel data modelling methodology based on a biologically inspired universal generative modelling approach called Hierarchical Spatial-Temporal State Machine (HSTSM). We further discuss various implications of policy, protection, valuation and commercialization related to Big Data, its applications and deployment