Characterizing the IoT ecosystem at scale

Internet of Things (IoT) devices are extremely popular with home, business, and industrial users. To provide their services, they typically rely on a backend server in- frastructure on the Internet, which collectively form the IoT Ecosystem. This ecosys- tem is rapidly growing and offers users an increasing number of services. It also has been a source and target of significant security and privacy risks. One notable exam- ple is the recent large-scale coordinated global attacks, like Mirai, which disrupted large service providers. Thus, characterizing this ecosystem yields insights that help end-users, network operators, policymakers, and researchers better understand it, obtain a detailed view, and keep track of its evolution. In addition, they can use these insights to inform their decision-making process for mitigating this ecosystem’s security and privacy risks. In this dissertation, we characterize the IoT ecosystem at scale by (i) detecting the IoT devices in the wild, (ii) conducting a case study to measure how deployed IoT devices can affect users’ privacy, and (iii) detecting and measuring the IoT backend infrastructure. To conduct our studies, we collaborated with a large European Internet Service Provider (ISP) and a major European Internet eXchange Point (IXP). They rou- tinely collect large volumes of passive, sampled data, e.g., NetFlow and IPFIX, for their operational purposes. These data sources help providers obtain insights about their networks, and we used them to characterize the IoT ecosystem at scale. We start with IoT devices and study how to track and trace their activity in the wild. We developed and evaluated a scalable methodology to accurately detect and monitor IoT devices with limited, sparsely sampled data in the ISP and IXP. Next, we conduct a case study to measure how a myriad of deployed devices can affect the privacy of ISP subscribers. Unfortunately, we found that the privacy of a substantial fraction of IPv6 end-users is at risk. We noticed that a single device at home that encodes its MAC address into the IPv6 address could be utilized as a tracking identifier for the entire end-user prefix—even if other devices use IPv6 privacy extensions. Our results showed that IoT devices contribute the most to this privacy leakage. Finally, we focus on the backend server infrastructure and propose a methodology to identify and locate IoT backend servers operated by cloud services and IoT vendors. We analyzed their IoT traffic patterns as observed in the ISP. Our analysis sheds light on their diverse operational and deployment strategies. The need for issuing a priori unknown network-wide queries against large volumes of network flow capture data, which we used in our studies, motivated us to develop Flowyager. It is a system built on top of existing traffic capture utilities, and it relies on flow summarization techniques to reduce (i) the storage and transfer cost of flow captures and (ii) query response time. We deployed a prototype of Flowyager at both the IXP and ISP.Internet-of-Things-Geräte (IoT) sind aus vielen Haushalten, Büroräumen und In- dustrieanlagen nicht mehr wegzudenken. Um ihre Dienste zu erbringen, nutzen IoT- Geräte typischerweise auf eine Backend-Server-Infrastruktur im Internet, welche als Gesamtheit das IoT-Ökosystem bildet. Dieses Ökosystem wächst rapide an und bie- tet den Nutzern immer mehr Dienste an. Das IoT-Ökosystem ist jedoch sowohl eine Quelle als auch ein Ziel von signifikanten Risiken für die Sicherheit und Privatsphäre. Ein bemerkenswertes Beispiel sind die jüngsten groß angelegten, koordinierten globa- len Angriffe wie Mirai, durch die große Diensteanbieter gestört haben. Deshalb ist es wichtig, dieses Ökosystem zu charakterisieren, eine ganzheitliche Sicht zu bekommen und die Entwicklung zu verfolgen, damit Forscher, Entscheidungsträger, Endnutzer und Netzwerkbetreibern Einblicke und ein besseres Verständnis erlangen. Außerdem können alle Teilnehmer des Ökosystems diese Erkenntnisse nutzen, um ihre Entschei- dungsprozesse zur Verhinderung von Sicherheits- und Privatsphärerisiken zu verbes- sern. In dieser Dissertation charakterisieren wir die Gesamtheit des IoT-Ökosystems indem wir (i) IoT-Geräte im Internet detektieren, (ii) eine Fallstudie zum Einfluss von benutzten IoT-Geräten auf die Privatsphäre von Nutzern durchführen und (iii) die IoT-Backend-Infrastruktur aufdecken und vermessen. Um unsere Studien durchzuführen, arbeiten wir mit einem großen europäischen Internet- Service-Provider (ISP) und einem großen europäischen Internet-Exchange-Point (IXP) zusammen. Diese sammeln routinemäßig für operative Zwecke große Mengen an pas- siven gesampelten Daten (z.B. als NetFlow oder IPFIX). Diese Datenquellen helfen Netzwerkbetreibern Einblicke in ihre Netzwerke zu erlangen und wir verwendeten sie, um das IoT-Ökosystem ganzheitlich zu charakterisieren. Wir beginnen unsere Analysen mit IoT-Geräten und untersuchen, wie diese im Inter- net aufgespürt und verfolgt werden können. Dazu entwickelten und evaluierten wir eine skalierbare Methodik, um IoT-Geräte mit Hilfe von eingeschränkten gesampelten Daten des ISPs und IXPs präzise erkennen und beobachten können. Als Nächstes führen wir eine Fallstudie durch, in der wir messen, wie eine Unzahl von eingesetzten Geräten die Privatsphäre von ISP-Nutzern beeinflussen kann. Lei- der fanden wir heraus, dass die Privatsphäre eines substantiellen Teils von IPv6- Endnutzern bedroht ist. Wir entdeckten, dass bereits ein einzelnes Gerät im Haus, welches seine MAC-Adresse in die IPv6-Adresse kodiert, als Tracking-Identifikator für das gesamte Endnutzer-Präfix missbraucht werden kann — auch wenn andere Geräte IPv6-Privacy-Extensions verwenden. Unsere Ergebnisse zeigten, dass IoT-Geräte den Großteil dieses Privatsphäre-Verlusts verursachen. Abschließend fokussieren wir uns auf die Backend-Server-Infrastruktur und wir schla- gen eine Methodik zur Identifizierung und Lokalisierung von IoT-Backend-Servern vor, welche von Cloud-Diensten und IoT-Herstellern betrieben wird. Wir analysier- ten Muster im IoT-Verkehr, der vom ISP beobachtet wird. Unsere Analyse gibt Auf- schluss über die unterschiedlichen Strategien, wie IoT-Backend-Server betrieben und eingesetzt werden. Die Notwendigkeit a-priori unbekannte netzwerkweite Anfragen an große Mengen von Netzwerk-Flow-Daten zu stellen, welche wir in in unseren Studien verwenden, moti- vierte uns zur Entwicklung von Flowyager. Dies ist ein auf bestehenden Netzwerkverkehrs- Tools aufbauendes System und es stützt sich auf die Zusammenfassung von Verkehrs- flüssen, um (i) die Kosten für Archivierung und Transfer von Flow-Daten und (ii) die Antwortzeit von Anfragen zu reduzieren. Wir setzten einen Prototypen von Flowyager sowohl im IXP als auch im ISP ein

Flow monitoring in software-defined networks: finding the accuracy/performance tradeoffs

In OpenFlow-based Software-Defined Networks, obtaining flow-level measurements, similar to those provided by NetFlow/IPFIX, is challenging as it requires to install an entry per flow in the flow tables. This approach does not scale well as the number of entries in the flow tables is limited and small. Moreover, labeling the flows with the application that generates the traffic would greatly enrich these reports, as it would provide very valuable information for network performance and security among others. In this paper, we present a scalable flow monitoring solution fully compatible with current off-the-shelf OpenFlow switches. Measurements are maintained in the switches and are asynchronously sent to a SDN controller. Additionally, flows are classified using a combination of DPI and Machine Learning (ML) techniques with special focus on the identification of web and encrypted traffic. For the sake of scalability, we designed two different traffic sampling methods depending on the OpenFlow features available in the switches. We implemented our monitoring solution within OpenDaylight and evaluated it in a testbed with Open vSwitch, using also a number of DPI and ML tools to find the best tradeoff between accuracy and performance. Our experimental results using real-world traffic show that the measurement and classification systems are accurate and the cost to deploy them is significantly reduced.Peer ReviewedPostprint (author's final draft

Sampling techniques applied to anomalous events detection

Dissertação de mestrado integrado em Engenharia InformáticaNowadays, one of the major worries about a network is security. Since the network has become the big platform it is, the number of attacks or attempts to steal information or just harm someone or something is getting bigger to handle or harder to find. Sampling techniques help to solve these problems as they are used to reduce the scope of the analysis, as well as the resources needed to perform it. By using sample techniques to search and find the attacks in the network traffic it will become easier to detect attacks and keep the network secure. As will be seen in the following sections, joining sampling and security is not an easy task to do. Questions such as, what are the best techniques to be used, what are the best methods to be implemented, are inevitable when using sampling. However, sampling can bring more advantages than disadvantages. Besides that, depending on the chosen measurement method, sampling technique or algorithm performed to analyse the samples, the results can change a lot according to the target for the technique. To achieve results for evaluation, a Network-based Intrusion Detection System (NIDS) will be used to identify anomalous events present in the samples.Hoje em dia, uma das maiores preocupações com uma rede é a segurança. Como a rede se tornou a grande plataforma que é, o número de ataques ou tentativas de roubar informações ou apenas prejudicar alguém ou algo está cada vez maior ou mais difícil de encontrar. As téc nicas de amostragem ajudam a resolver esses problemas visto que são utilizadas para reduzir o escopo da análise assim como os recursos necessários para realizar a mesma. Usando técnicas de amostra para procurar e localizar os ataques no tráfego da rede, facilita prevenir ataques e manter a rede segura. Como será constatado nas próximas secções, juntar amostragem e segurança não é uma tarefa fácil. Questões como, quais são as melhores técnicas a serem utilizadas, quais os melhores métodos a serem implementados, são inevitáveis aquando da utilização de amostragem. Contudo, amostragem pode trazer mais vantagens do que desvan tagens. Além disso, dependendo do método de medição escolhido, técnica de amostragem ou algoritmo usado para analisar as amostras, os resultados podem variar muito consoante o alvo da técnica. Para alcançar resultados para avaliação vai ser utilizado um Network-based Intrusion Detection System (NIDS) de forma a identificar os eventos anómalos presentes nas amostragens

The Cloud Strikes Back: Investigating the Decentralization of IPFS

Interplanetary Filesystem (IPFS) is one of the largest peer-to-peer filesystems in operation. The network is the default storage layer for Web3 and is being presented as a solution to the centralization of the web. In this paper, we present a large-scale, multi-modal measurement study of the IPFS network. We analyze the topology, the traffic, the content providers and the entry points from the classical Internet. Our measurements show significant centralization in the IPFS network and a high share of nodes hosted in the cloud. We also shed light on the main stakeholders in the ecosystem. We discuss key challenges that might disrupt continuing efforts to decentralize the Web and highlight multiple properties that are creating pressures toward centralization

Big Data Analytics for Flow-based Anomaly Detection in High-Speed Networks

The Cisco VNI Complete Forecast Highlights clearly states that the Internet traffic is growing in three different directions, Volume, Velocity, and Variety, bringing computer network into the big data era. At the same time, sophisticated network attacks are growing exponentially. Such growth making the existing signature-based security tools, like firewall and traditional intrusion detection systems, ineffective against new kind of attacks or variations of known attacks. In this dissertation, we propose an unsupervised method for network anomaly detection. This method is able to detect unknown and new malicious activities in high-speed network traffic. Our method uses an innovative detection algorithm able to identify the hosts responsible for anomalous flows by using a new statistical feature related to traffic flow. This feature is defined as the ratio between the number of flows generated by a host and the number of flows it receives. We evaluate our method with real backbone traffic traces from the Measurement and Analysis on the WIDE Internet (MAWI) archive. Furthermore, we compare the results of our method with MAWILab archive, a database that assists researchers to evaluate their traffic anomaly detection methods. The results point out that our method achieves an average positive prediction rate (i.e. Precision) of 90\% outperforming the four MAWILab detection methods in terms of false negative rate. We deploy three cluster configurations to evaluate the horizontal and vertical scalability performance of the proposed architecture and our method shows outstanding performance in terms of response time