No NAT'd User left Behind: Fingerprinting Users behind NAT from NetFlow Records alone

It is generally recognized that the traffic generated by an individual connected to a network acts as his biometric signature. Several tools exploit this fact to fingerprint and monitor users. Often, though, these tools assume to access the entire traffic, including IP addresses and payloads. This is not feasible on the grounds that both performance and privacy would be negatively affected. In reality, most ISPs convert user traffic into NetFlow records for a concise representation that does not include, for instance, any payloads. More importantly, large and distributed networks are usually NAT'd, thus a few IP addresses may be associated to thousands of users. We devised a new fingerprinting framework that overcomes these hurdles. Our system is able to analyze a huge amount of network traffic represented as NetFlows, with the intent to track people. It does so by accurately inferring when users are connected to the network and which IP addresses they are using, even though thousands of users are hidden behind NAT. Our prototype implementation was deployed and tested within an existing large metropolitan WiFi network serving about 200,000 users, with an average load of more than 1,000 users simultaneously connected behind 2 NAT'd IP addresses only. Our solution turned out to be very effective, with an accuracy greater than 90%. We also devised new tools and refined existing ones that may be applied to other contexts related to NetFlow analysis

Seiðr: Dataplane Assisted Flow Classification Using ML

Real-time, high-speed flow classification is fundamental for network operation tasks, including reactive and proactive traffic engineering, anomaly detection and security enhancement. Existing flow classification solutions, however, do not allow operators to classify traffic based on fine-grained, temporal dynamics due to imprecise timing, often rely on sampled data, or only work with low traffic volumes and rates. In this paper, we present Seiðr, a classification solution that: (i) uses precision timing, (ii) has the ability to examine every packet on the network, (iii) classifies very high traffic volumes with high precision. To achieve this, Seiðr exploits the data aggregation and timestamping functionality of programmable dataplanes. As a concrete example, we present how Seiðr can be used together with Machine Learning algorithms (such as CNN, k -NN) to provide accurate, real-time and high-speed TCP congestion control classification, separating TCP BBR from its predecessors with over 88–96% accuracy and F1-score of 0.864-0.965, while only using 15.5 MiB of memory in the dataplane

VELOCITY : A NetFlow Based Optimized Geo-IP Lookup Tool

Title from PDF of title page, viewed on August 24, 2016Thesis advisor: Deepankar MedhiVitaIncludes bibliographical references (pages 36-38)Thesis (M.S.)--School of Computing and Engineering. University of Missouri--Kansas City, 2016It is a challenging task for network administrators to monitor their institution's network against undesirable behavior. While NetFlow is useful to gather flow-level data for any Internet connection, its feature is limited to traditional flow-level information such as source IP address, destination IP address, source port number, destination port number, and the protocol type. Thus, if we are to understand geographic dynamics of any flow connected to hosts at an institution from the outside world, it is not currently possible with NetFlow. To address for geo-location information of such flows, we developed the tool, VELOCITY. This tool allows to correlate IP addresses with geo-location information to visualize geo-location of incoming and outgoing flows. The VELOCITY tool consists of four different methods, with increasing order of efficiency of the methods. We found that Method 3 outperforms Methods 1 and 2 in case of filling database with geographical data for the first time. Method 4, which is an extension of Method 3, finds geographical information for IP addresses that are not present in the currently populated database, thereby providing a more optimized approach than Method 3 for incremental flow data. Furthermore, for visualization and near real time experience, we also developed a web application that displays geographical information of IP address of flows on Google maps.Introduction -- Literature survey -- Methods -- WEB application -- Results -- Conclusion -- Appendix A. Xidel -- Appendix B. GNU paralle

Reviewing Traffic ClassificationData Traffic Monitoring and Analysis

Traffic classification has received increasing attention in the last years. It aims at offering the ability to automatically recognize the application that has generated a given stream of packets from the direct and passive observation of the individual packets, or stream of packets, flowing in the network. This ability is instrumental to a number of activities that are of extreme interest to carriers, Internet service providers and network administrators in general. Indeed, traffic classification is the basic block that is required to enable any traffic management operations, from differentiating traffic pricing and treatment (e.g., policing, shaping, etc.), to security operations (e.g., firewalling, filtering, anomaly detection, etc.). Up to few years ago, almost any Internet application was using well-known transport layer protocol ports that easily allowed its identification. More recently, the number of applications using random or non-standard ports has dramatically increased (e.g. Skype, BitTorrent, VPNs, etc.). Moreover, often network applications are configured to use well-known protocol ports assigned to other applications (e.g. TCP port 80 originally reserved for Web traffic) attempting to disguise their presence. For these reasons, and for the importance of correctly classifying traffic flows, novel approaches based respectively on packet inspection, statistical and machine learning techniques, and behavioral methods have been investigated and are becoming standard practice. In this chapter, we discuss the main trend in the field of traffic classification and we describe some of the main proposals of the research community. We complete this chapter by developing two examples of behavioral classifiers: both use supervised machine learning algorithms for classifications, but each is based on different features to describe the traffic. After presenting them, we compare their performance using a large dataset, showing the benefits and drawback of each approac

Design and Development of a Framework for Traffic Management in a Global Manufacturing Enterprise: The American Standard Case Study

Managed Bandwidth Services (MBSs) use Quality of Service (QoS) guarantees to effectively control traffic flows and reduce network delay. In the past, the provision of MBS in a global manufacturing enterprise was a difficult task for network administrators. However, advances in recently emerging technologies, such as Multiprotocol Label Switching (MPLS), Generalized Multiprotocol Label Switching (GMPLS), Integrated Services (IntServ), Differentiated Services (DiffServ), and Constraint-based Routing (CBR), hold promise to make MBS implementation more manageable. QoS technologies, such as DiffServ and IntServ, offer the benefits of better application performance and delivery of reliable network service. As a consequence of network traffic loads, packet congestion and latency increases still exist and must be addressed by enterprises that intend to support an MBS solution. In this investigation, the author addressed an issue that is faced by many large manufacturing enterprises, i.e., the addition of latency and congestion sensitive traffic such as Voice-over-Internet Protocol (VoIP) to networks with limited bandwidth. The goal of this research was to provide global manufacturing enterprises with a model for bandwidth management in their offices and plants. This model was based on findings from a case study of traffic management at American Standard Companies

Monitorização de sistemas de informação críticos

Tese de mestrado integrado. Engenharia Informática e Computação. Universidade do Porto. Faculdade de Engenharia. 201

Toward Open and Programmable Wireless Network Edge

Increasingly, the last hop connecting users to their enterprise and home networks is wireless. Wireless is becoming ubiquitous not only in homes and enterprises but in public venues such as coffee shops, hospitals, and airports. However, most of the publicly and privately available wireless networks are proprietary and closed in operation. Also, there is little effort from industries to move forward on a path to greater openness for the requirement of innovation. Therefore, we believe it is the domain of university researchers to enable innovation through openness. In this thesis work, we introduce and defines the importance of open framework in addressing the complexity of the wireless network. The Software Defined Network (SDN) framework has emerged as a popular solution for the data center network. However, the promise of the SDN framework is to make the network open, flexible and programmable. In order to deliver on the promise, SDN must work for all users and across all networks, both wired and wireless. Therefore, we proposed to create new modules and APIs to extend the standard SDN framework all the way to the end-devices (i.e., mobile devices, APs). Thus, we want to provide an extensible and programmable abstraction of the wireless network as part of the current SDN-based solution. In this thesis work, we design and develop a framework, weSDN (wireless extension of SDN), that extends the SDN control capability all the way to the end devices to support client-network interaction capabilities and new services. weSDN enables the control-plane of wireless networks to be extended to mobile devices and allows for top-level decisions to be made from an SDN controller with knowledge of the network as a whole, rather than device centric configurations. In addition, weSDN easily obtains user application information, as well as the ability to monitor and control application flows dynamically. Based on the weSDN framework, we demonstrate new services such as application-aware traffic management, WLAN virtualization, and security management

USER PROFILING BASED ON NETWORK APPLICATION TRAFFIC MONITORING

There is increasing interest in identifying users and behaviour profiling from network traffic metadata for traffic engineering and security monitoring. However, user identification and behaviour profiling in real-time network management remains a challenge, as the activities and underlying interactions of network applications are constantly changing. User behaviour is also changing and adapting in parallel, due to changes in the online interaction environment. A major challenge is how to detect user activity among generic network traffic in terms of identifying the user and his/her changing behaviour over time. Another issue is that relying only on computer network information (Internet Protocol [IP] addresses) directly to identify individuals who generate such traffic is not reliable due to user mobility and IP mobility (resulting from the widespread use of the Dynamic Host Configuration Protocol [DHCP]) within a network. In this context, this project aims to identify and extract a set of features that may be adequate for use in identifying users based on their network application activity and timing resolution to describe user behaviour. The project also provides a procedure for traffic capturing and analysis to extract the required profiling parameters; the procedure includes capturing flow traffic and then performing statistical analysis to extract the required features. This will help network administrators and internet service providers to create user behaviour traffic profiles in order to make informed decisions about policing and traffic management and investigate various network security perspectives. The thesis explores the feasibility of user identification and behaviour profiling in order to be able to identify users independently of their IP address. In order to maintain privacy and overcome the issues associated with encryption (which exists on an increasing volume of network traffic), the proposed approach utilises data derived from generic flow network traffic (NetFlow information). A number of methods and techniques have been proposed in prior research for user identification and behaviour profiling from network traffic information, such as port-based monitoring and profiling, deep packet inspection (DPI) and statistical methods. However, the statistical methods proposed in this thesis are based on extracting relevant features from network traffic metadata, which are utilised by the research community to overcome the limitations that occur with port-based and DPI techniques. This research proposes a set of novel statistical timing features extracted by considering application-level flow sessions identified through Domain Name System (DNS) filtering criteria and timing resolution bins: one-hour time bins (0-23) and quarter- hour time bins (0-95). The novel time bin features are utilised to identify users by representing their 24-hour daily activities by analysing the application-level network traffic based on an automated technique. The raw network traffic is analysed based on the development of a features extraction process in terms of representing each user’s daily usage through a combination of timing features, including the flow session, timing and DNS filtering for the top 11 applications. In addition, media access control (MAC) and IP source mapping (in a truth table) is utilised to ensure that profiling is allocated to the correct host, even if the IP addresses change. The feature extraction process developed for this thesis focuses more on the user, rather than machine-to-machine traffic, and the research has sought to use this information to determine whether a behavioural profile could be developed to enable the identification of users. Network traffic was collected and processed using the aforementioned feature extraction process for 23 users for a period of 60 days (8 May-8 July 2018). The traffic was captured from the Centre for Cyber Security, Communications and Network Research (CSCAN) at the University of Plymouth. The results of identifying and profiling users from extracted timing features behaviour show that the system is capable of identifying users with an average true positive identification rate (TPIR) based on hourly time bin features for the whole population of ~86% and ~91% for individual users. Furthermore, the results show that the system has the ability to identify users based on quarter-hour time bin features, with an average TPIR of ~94% for the whole population and ~96% for the individual user.Royal Embassy of Saudi Arabia Cultural Burea

Arhitektura sistema za prepoznavanje nepravilnosti u mrežnom saobraćaju zasnovano na analizi entropije

With the steady increase in reliance on computer networks in all aspects of life, computers and other connected devices have become more vulnerable to attacks, which exposes them to many major threats, especially in recent years. There are different systems to protect networks from these threats such as firewalls, antivirus programs, and data encryption, but it is still hard to provide complete protection for networks and their systems from the attacks, which are increasingly sophisticated with time. That is why it is required to use intrusion detection systems (IDS) on a large scale to be the second line of defense for computer and network systems along with other network security techniques. The main objective of intrusion detection systems is used to monitor network traffic and detect internal and external attacks. Intrusion detection systems represent an important focus of studies today, because most protection systems, no matter how good they are, can fail due to the emergence of new (unknown/predefined) types of intrusions. Most of the existing techniques detect network intrusions by collecting information about known types of attacks, so-called signature-based IDS, using them to recognize any attempt of attack on data or resources. The major problem of this approach is its inability to detect previously unknown attacks, even if these attacks are derived slightly from the known ones (the so-called zero-day attack). Also, it is powerless to detect encryption-related attacks. On the other hand, detecting abnormalities concerning conventional behavior (anomaly-based IDS) exceeds the abovementioned limitations. Many scientific studies have tended to build modern and smart systems to detect both known and unknown intrusions. In this research, an architecture that applies a new technique for IDS using an anomaly-based detection method based on entropy is introduced. Network behavior analysis relies on the profiling of legitimate network behavior in order to efficiently detect anomalous traffic deviations that indicate security threats. Entropy-based detection techniques are attractive due to their simplicity and applicability in real-time network traffic, with no need to train the system with labelled data. Besides the fact that the NetFlow protocol provides only a basic set of information about network communications, it is very beneficial for identifying zero-day attacks and suspicious behavior in traffic structure. Nevertheless, the challenge associated with limited NetFlow information combined with the simplicity of the entropy-based approach is providing an efficient and sensitive mechanism to detect a wide range of anomalies, including those of small intensity. However, a recent study found of generic entropy-based anomaly detection reports its vulnerability to deceit by introducing spoofed data to mask the abnormality. Furthermore, the majority of approaches for further classification of anomalies rely on machine learning, which brings additional complexity. Previously highlighted shortcomings and limitations of these approaches open up a space for the exploration of new techniques and methodologies for the detection of anomalies in network traffic in order to isolate security threats, which will be the main subject of the research in this thesis. Abstract An architrvture for network traffic anomaly detection system based on entropy analysis Page vii This research addresses all these issues by providing a systematic methodology with the main novelty in anomaly detection and classification based on the entropy of flow count and behavior features extracted from the basic data obtained by the NetFlow protocol. Two new approaches are proposed to solve these concerns. Firstly, an effective protection mechanism against entropy deception derived from the study of changes in several entropy types, such as Shannon, Rényi, and Tsallis entropies, as well as the measurement of the number of distinct elements in a feature distribution as a new detection metric. The suggested method improves the reliability of entropy approaches. Secondly, an anomaly classification technique was introduced to the existing entropy-based anomaly detection system. Entropy-based anomaly classification methods were presented and effectively confirmed by tests based on a multivariate analysis of the entropy changes of several features as well as aggregation by complicated feature combinations. Through an analysis of the most prominent security attacks, generalized network traffic behavior models were developed to describe various communication patterns. Based on a multivariate analysis of the entropy changes by anomalies in each of the modelled classes, anomaly classification rules were proposed and verified through the experiments. The concept of the behavior features is generalized, while the proposed data partitioning provides greater efficiency in real-time anomaly detection. The practicality of the proposed architecture for the implementation of effective anomaly detection and classification system in a general real-world network environment is demonstrated using experimental data

Network attacks detection based on traffic flows analysis using hybrid machine learning algorithms

Razvoj savremenih mrežnih okruženja se zasniva na primeni različitih tehnologija, povezivanju sa drugim tehnološki drugačijim konceptima i obezbeđivanju njihove interoperabilnosti. Tako složeno mrežno okruženje je neprekidno izloženo različitim izazovima, pri čemu je obezbeđivanje sigurnosti servisa i podataka jedan od najvažnijih zadataka. Novi zahtevi za sisteme zaštite se zasnivaju na potrebi za efikasnim praćenjem i razumevanju karakteristika mrežnog saobraćaja, a uslovljeni su stalnim porastom broja korisnika i razvojem novih aplikacija. Razvoj rešenja u oblasti detekcije anomalija i napada je postao svojevrsni imperativ, imajući u vidu da se paralelno odvija intenzivni razvoj u oblasti sajber napada. Osim toga, promene mrežnog saobraćaja su postale sve dinamičnije, a kao poseban problem se izdvaja velika heterogenost primenjenih tehnologija i korisničkih uređaja. Iako dostupna literatura prepoznaje veliki broj radova koji se bave analizom tokova mrežnog saobraćaja za potrebe praćenja performansi i sigurnosnih aspekata mreža, mali je broj istraživanja koja se zasnivaju na procedurama generisanja i analize profila ponašanja mrežnog saobraćaja, odnosno specifičnih komunikacionih obrazaca. U tom smislu, analiza ponašanja mreže se u sve većoj meri oslanja na razumevanje normalnih ili prihvatljivih obrazaca ponašanja na osnovu kojih je moguće efikasno otkrivanje obrazaca anomalija. Za razliku od sistema za otkrivanje napada koji se zasnivaju na analizi sadržaja svakog pojedinačnog paketa (signature-based), ovaj pristup je izuzetno koristan za identifikaciju nepoznatih pretnji, napada nultog dana, sumnjivog ponašanja i za sveopšte poboljšavanje performansi mrežnih okruženja...The development of the modern network environments, their application, and the dynamics of their interoperability with other technologically different concepts, is based on the application and compatibility of different heterogeneous technologies. Such a complex network environment is constantly exposed to various operational challenges, where ensuring the security and safety of services and data represents one of the most important tasks. The constant increase in the number of users and the intensive development of new applications that require high bandwidth has defined new requirements for security systems, which are based on monitoring and effectively understanding network traffic characteristics. In the light of the increasingly intensive development in the field of cyberattacks, persistent dynamic changes in network traffic, as well as the increased heterogeneity of the used technologies and devices, the development of solutions in the field of anomaly and attack detection has become a kind of imperative. Although the available literature recognizes a large number of papers dealing with the analysis of network traffic flows for the needs of the monitoring of the performance and security aspects of networks, just a few studies are based on the procedures for generating network traffic behavior profiles, or specific communication patterns. In this sense, network behavior analysis relies on an understanding of normal or acceptable behavior patterns, which would allow for the effective detection of unusual, anomalous behavior patterns. Unlike the intrusion detection systems that are based on the packet payload or signature (signature-based), this approach is extremely useful not only for the identification of unknown threats, zero-day attacks, and suspicious behavior, but also for the improvement of the overall network performance..