Mnemonic Lossy Counting: An Efficient and Accurate Heavy-hitters Identification Algorithm

International audienceIdentifying heavy-hitter traffic flows efficiently and accurately is essential for Internet security, accounting and traffic engineering. However, finding all heavy-hitters might require large memory for storage of flows information that is incompatible with the usage of fast and small memory. Moreover, upcoming 100Gbps transmission rates make this recognition more challenging. How to improve the accuracy of heavy-hitters identification with limited memory space has become a critical issue. This paper presents a scalable algorithm named Mnemonic Lossy Counting (MLC) that improves the accuracy of heavy-hitters identification while having a reasonable time and space complexity. MLC algorithm holds potential candidate heavy-hitters in a historical information table. This table is used to obtain tighter error bounds on the estimated sizes of candidate heavy-hitters. We validate the MLC algorithm using real network traffic traces, and we compared its performance with two state-of-theart algorithms, namely Lossy Counting (LC) and Probabilistic Lossy Counting (PLC). The results reveal that: 1) with same set of parameters and memory usage, MLC achieves between 31.5% and 6.67% fewer false positives than LC and PLC. 2) MLC and LC have a zero false negative ratio, whereas 38% of the cases PLC has a non-zero false negatives and PLC can miss up to 4.4% of heavy-hitters. 3) MLC has a slightly lower memory cost than LC during the first few windows and its memory usage decreases with time, when PLC memory usage declines sharply. 4) MLC has similar runtime than LC, and smaller time than PLC

A SMART DATA APPROACH TO ANALYZE VEHICLE FLOWS

Abstract. In the logic of Smart Cities it is of fundamental importance to analyze the traffic situation through dedicated sensors and networks. According to this approach and through the potential of smart data is based this study. Improve prediction of traffic patterns by analyzing and counting vehicles in a virtualized scene in real time. In the past, the technique of hardware inductive coils was used that were dropped in the asphalt to exploit the principle of magnetic induction in order to verify the transit of vehicles. This technique is not able to classify vehicles or estimate their speed, unless using multiple inductive coils. The proposed system provides for the virtualization of an area of interest which requires a selection and mapping of the areas where the control areas are to be included. The "image detection" techniques allow us to classify the vehicles in transit. With the techniques of "machine learning" can to able to verify the flow, count the vehicles present in the scene and classify them by vehicle type in real time. The vehicle counting and classification data available in the cloud platform allow to model and update the main nodes of the network in order to improve the prediction and estimates of the best routes of the road network according to the degree of saturation of the flows and the length of the line of the graph. The model can also indicate additional information of an environmental nature in an ITS system present in the cloud

Distributed Collaborative Monitoring in Software Defined Networks

We propose a Distributed and Collaborative Monitoring system, DCM, with the following properties. First, DCM allow switches to collaboratively achieve flow monitoring tasks and balance measurement load. Second, DCM is able to perform per-flow monitoring, by which different groups of flows are monitored using different actions. Third, DCM is a memory-efficient solution for switch data plane and guarantees system scalability. DCM uses a novel two-stage Bloom filters to represent monitoring rules using small memory space. It utilizes the centralized SDN control to install, update, and reconstruct the two-stage Bloom filters in the switch data plane. We study how DCM performs two representative monitoring tasks, namely flow size counting and packet sampling, and evaluate its performance. Experiments using real data center and ISP traffic data on real network topologies show that DCM achieves highest measurement accuracy among existing solutions given the same memory budget of switches

Optimal Elephant Flow Detection

Monitoring the traffic volumes of elephant flows, including the total byte count per flow, is a fundamental capability for online network measurements. We present an asymptotically optimal algorithm for solving this problem in terms of both space and time complexity. This improves on previous approaches, which can only count the number of packets in constant time. We evaluate our work on real packet traces, demonstrating an up to X2.5 speedup compared to the best alternative.Comment: Accepted to IEEE INFOCOM 201

Identifying Key Sectors in the Regional Economy: A Network Analysis Approach Using Input-Output Data

By applying network analysis techniques to large input-output system, we identify key sectors in the local/regional economy. We overcome the limitations of traditional measures of centrality by using random-walk based measures, as an extension of Blochl et al. (2011). These are more appropriate to analyze very dense networks, i.e. those in which most nodes are connected to all other nodes. These measures also allow for the presence of recursive ties (loops), since these are common in economic systems (depending to the level of aggregation, most firms buy from and sell to other firms in the same industrial sector). The centrality measures we present are well suited for capturing sectoral effects missing from the usual output and employment multipliers. We also develop an R package (xtranat) for the processing of data from IMPLAN(R) models and for computing the newly developed measures