Distributed Collaborative Monitoring in Software Defined Networks

We propose a Distributed and Collaborative Monitoring system, DCM, with the following properties. First, DCM allow switches to collaboratively achieve flow monitoring tasks and balance measurement load. Second, DCM is able to perform per-flow monitoring, by which different groups of flows are monitored using different actions. Third, DCM is a memory-efficient solution for switch data plane and guarantees system scalability. DCM uses a novel two-stage Bloom filters to represent monitoring rules using small memory space. It utilizes the centralized SDN control to install, update, and reconstruct the two-stage Bloom filters in the switch data plane. We study how DCM performs two representative monitoring tasks, namely flow size counting and packet sampling, and evaluate its performance. Experiments using real data center and ISP traffic data on real network topologies show that DCM achieves highest measurement accuracy among existing solutions given the same memory budget of switches

Online Accumulation: Reconstruction of Worm Propagation Path

Abstract. Knowledge of the worm origin is necessary to forensic analysis, and knowledge of the initial causal flows supports diagnosis of how network defenses were breached. Fast and accurate online tracing network worm during its propagation, help to detect worm origin and the earliest infected nodes, and is essential for large-scale worm containment. This paper introduces the Accumulation Algorithm which can efficiently tracing worm origin and the initial propagation paths, and presents an improved online Accumulation Algorithm using sliding detection windows. We also analyzes and verifies their detection accuracy and containment efficacy through simulation experiments in large scale network. Results indicate that the online Accumulation Algorithm can accurately tracing worms and efficiently containing their propagation in an approximately real-time manner

Enabling event-triggered data plane monitoring

We propose a push-based approach to network monitoring that allows the detection, within the dataplane, of traffic aggregates. Notifications from the switch to the controller are sent only if required, avoiding the transmission or processing of unnecessary data. Furthermore, the dataplane iteratively refines the responsible IP prefixes, allowing the controller to receive information with a flexible granularity. We implemented our solution, Elastic Trie, in P4 and for two different FPGA devices. We evaluated it with packet traces from an ISP backbone. Our approach can spot changes in the traffic patterns and detect (with 95% of accuracy) either hierarchical heavy hitters with less than 8KB or superspreaders with less than 300KB of memory, respectively. Additionally, it reduces controller-dataplane communication overheads by up to two orders of magnitude with respect to state-of-the-art solutions

Coordinated sampling sans Origin-Destination identifiers: Algorithms and analysis

Abstract—Flow monitoring is used for a wide range of network management applications. Many such applications require that the monitoring infrastructure provide high flow coverage and support fine-grained network-wide objectives. Coordinated Sampling (cSamp) is a recent proposal that improves the monitoring capabilities of ISPs to address these demands. In this paper, we address a key deployment impediment for cSamp-like solutions–the need for routers to determine the Origin-Destination (OD) pair of each packet. In practice, however, this information is not available without expensive changes. We present a new framework called cSamp-T, in which each router uses only local information, instead of the OD-pair identifiers. Leveraging results from the theory of maximizing submodular set functions, cSamp-T provides near-ideal performance in maximizing the total flow coverage in the network. Further, with a small amount of targeted upgrades to a few routers, cSamp-T nearly optimally maximizes the minimum fractional coverage across all OD-pairs. We demonstrate these results on a range of real topologies. I

Enabling Efficient and General Subpopulation Analytics in Multidimensional Data Streams

Today’s large-scale services (e.g., video streaming platforms, data centers, sensor grids) need diverse real-time summary statistics across multiple subpopulations of multidimensional datasets. However, state-of-the-art frameworks do not offer general and accurate analytics in real time at reasonable costs. The root cause is the combinatorial explosion of data subpopulations and the diversity of summary statistics we need to monitor simultaneously. We present Hydra, an efficient framework for multidimensional analytics that presents a novel combination of using a “sketch of sketches” to avoid the overhead of monitoring exponentially-many subpopulations and universal sketching to ensure accurate estimates for multiple statistics. We build Hydra as an Apache Spark plugin and address practical system challenges to minimize overheads at scale. Across multiple real-world and synthetic multidimensional datasets, we show that Hydra can achieve robust error bounds and is an order of magnitude more efficient in terms of operational cost and memory footprint than existing frameworks (e.g., Spark, Druid) while ensuring interactive estimation times

Resilient communications in smart grids

Tese de mestrado, Segurança Informática, Universidade de Lisboa, Faculdade de Ciências, 2018As redes elétricas, algumas já centenárias, foram concebidas para uma realidade bastante diferente da actual. O facto de terem sido desenhadas para transportar e distribuir a energia de forma unidirecional, torna a infraestrutura rígida, causando problemas em termos de escalabilidade e dificulta a sua evolução. Conhecidas questões ambientais têm levado a que a geração de energia baseada em combustíveis fosseis seja substituída pela geração através de fontes de energia renováveis. Esta situação motivou a criação de incentivos ao investimento nas fontes de energia renováveis, o que levou a que cada vez mais consumidores apostem na microgeração. Estas alterações causaram uma mudança na forma como é feita a produção e distribuição de energia elétrica, com uma aposta crescente na interligação entre as várias fontes ao longo da infraestrutura, tornando a gestão destas redes uma tarefa extremamente complexa. Com o crescimento significativo de consumidores que também podem ser produtores, torna-se essencial uma coordenação cuidada na injeção de energia na rede. Este facto, aliado à crescente utilização de energia elétrica, faz com que a manutenção da estabilidade da rede seja um enorme desafio. As redes inteligentes, ou smart grids, propõem resolver muitos dos problemas que surgiram com esta alteração do paradigma de consumo/produção de energia elétrica. Os componentes da rede passam a comunicar uns com os outros, tornando a rede eléctrica bidirecional, facilitando assim a sua manutenção e gestão. A possibilidade de constante troca de informação entre todos os componentes que constituem a smart grid permite uma reação imediata relativamente às ações dos produtores e consumidores de energia elétrica. No entanto, com esta alteração de paradigma surgiram também muitos desafios. Nomeadamente, a necessidade de comunicação entre os equipamentos existentes nas smart grids leva a que as redes de comunicação tenham de cobrir grandes áreas. Essa complexidade aumenta quando a gestão necessita de ser feita ao nível de cada equipamento e não de forma global. Isto ´e devido ao facto de nas redes de comunicação tradicionais, o plano de controlo e o de dados estarem no mesmo equipamento, o que leva a que o seu controlo seja difícil e propício a erros. Este controlo descentralizado dificulta também a reorganização da rede quando ocorrem faltas pelo facto de não existir um dispositivo que tenha o conhecimento completo da rede. A adaptação rápida a faltas de forma a tornar a comunicação resiliente tem grande importância em redes sensíveis a latência como é o caso da smart grid, pelo que mecanismos eficientes de tolerância a faltas devem ser implementados. As redes definidas por software, ou Software Defined Networks (SDN), surgem como uma potencial solução para estes problemas. Através da separação entre o plano de controlo e o plano de dados, permite a centralização lógica do controlo da rede no controlador. Para tal, é necessário adicionar uma camada de comunicação entre o controlador e os dispositivos de rede, através de um protocolo como o Openflow. Esta separação reduz a complexidade da gestão da rede e a centralização lógica torna possível programar a rede de forma global, de modo a aplicar as políticas pretendidas. Estes fatores tornam a SDN uma soluçãoo interessante para utilizar em smart grids. Esta tese investiga formas de tornar a rede de comunicações empregue numa smart grid resiliente a faltas. Pelas vantagens mencionadas anteriormente, é usada uma solução baseada em SDN, sendo propostos dois módulos essenciais. O primeiro tem como objectivo a monitorização segura da rede, permitindo obter em tempo real métricas como largura de banda, latência e taxa de erro. O segundo módulo trata do roteamento e engenharia de tráfego, utilizando a informação fornecida pelo módulo de monitorização de forma a que os componentes da smart grid comuniquem entre si, garantindo que os requisitos das aplicações são cumpridos. Dada a criticidade da rede elétrica e a importância das comunicações na smart grid, os mecanismos desenvolvidos toleram faltas, quer de tipo malicioso, quer de tipo acidental.The evolution on how electricity is produced and consumed has made the management of power grids an extremely complex task. Today’s centenary power grids were not designed to fit a new reality where consumers can also be producers, or the impressive increase in consumption caused by more sophisticated and powerful appliances. Smart Grids have been prepared as a solution to cope with this problem, by supporting more sophisticated communications among all the components, allowing the grid to react quickly to changes in both consumption or production of energy. On the other hand, resorting to information and communication technologies (ICT) brings some challenges, namely, managing network devices at this scale and assuring that the strict communication requirements are fulfilled is a dauting task. Software Defined Networks (SDN) can address some of these problems by separating the control and data planes, and logically centralizing network control in a controller. The centralised control has the ability to observe the current state of the network from a vantage point, and programatically react based on that view, making the management task substantially easier. In this thesis we provide a solution for a resilient communications network for Smart Grids based on SDN. As Smart Grids are very sensitive to network issues, such as latency and packet loss, it is important to detect and react to any fault in a timely manner. To achieve this we propose and develop two core modules, a network monitor and a routing and traffic engineering module. The first is a solution for monitoring with the goal to obtain a global view of the current state of the network. The solution is secure, allowing malicious attempts to subvert this module to be detected in a timely manner. This information is then used by the second module to make routing decisions. The routing and traffic engineering module ensures that the communications among the smart grid components are possible and fulfils the strict requirements of the Smart Grid

On countermeasures of worm attacks over the Internet

Worm attacks have always been considered dangerous threats to the Internet since they can infect a large number of computers and consequently cause large-scale service disruptions and damage. Thus, research on modeling worm attacks, and defenses against them, have become vital to the field of computer and network security. This dissertation intends to systematically study two classes of countermeasures against worm attacks, known as traffic-based countermeasure and non-traffic based countermeasure. Traffic-based countermeasures are those whose means are limited to monitoring, collecting, and analyzing the traffic generated by worm attacks. Non-traffic based countermeasures do not have such limitations. For the traffic-based countermeasures, we first consider the worm attack that adopts feedback loop-control mechanisms which make its overall propagation traffic behavior similar to background non-worm traffic and circumvent the detection. We also develop a novel spectrumbased scheme to achieve highly effective detection performance against such attacks. We then consider worm attacks that perform probing traffic in a stealthy manner to obtain the location infrastructure of a defense system and introduce an information-theoretic based framework to obtain the limitations of such attacks and develop corresponding countermeasures. For the non-traffic based countermeasures, we first consider new unseen worm attacks and develop the countermeasure based on mining the dynamic signature of worm programs’ run-time execution. We then consider a generic worm attack that dynamically changes its propagation patterns and develops integrated countermeasures based on the attacker’s contradicted objectives. Lastly, we consider the real-world system setting with multiple incoming worm attacks that collaborate by sharing the history of their interactions with the defender and develop a generic countermeasure based on establishing the defender’s reputation of toughness in its repeated interactions with multiple incoming attackers to optimize the long-term defense performance. This dissertation research has broad impacts on Internet worm research since this work is fundamental, practical and extensible. Our developed framework can be used by researchers to understand key features of other forms of new worm attacks and develop countermeasures against them

Enabling efficient and general subpopulation analytics in multidimensional data streams

Today's large-scale services ( e.g. , video streaming platforms, data centers, sensor grids) need diverse real-time summary statistics across multiple subpopulations of multidimensional datasets. However, state-of-the-art frameworks do not offer general and accurate analytics in real time at reasonable costs. The root cause is the combinatorial explosion of data subpopulations and the diversity of summary statistics we need to monitor simultaneously. We present Hydra, an efficient framework for multidimensional analytics that presents a novel combination of using a "sketch of sketches" to avoid the overhead of monitoring exponentially-many subpopulations and universal sketching to ensure accurate estimates for multiple statistics. We build Hydra as an Apache Spark plugin and address practical system challenges to minimize overheads at scale. Across multiple real-world and synthetic multidimensional datasets, we show that Hydra can achieve robust error bounds and is an order of magnitude more efficient in terms of operational cost and memory footprint than existing frameworks (e.g., Spark, Druid) while ensuring interactive estimation times.Red Hat; CNS-2107086 - National Science Foundation; CNS-2106946 - National Science Foundation; CNS-2132643 - National Science FoundationPublished versio