Byzantine fault-tolerant agreement protocols for wireless Ad hoc networks

Tese de doutoramento, Informática (Ciências da Computação), Universidade de Lisboa, Faculdade de Ciências, 2010.The thesis investigates the problem of fault- and intrusion-tolerant consensus in resource-constrained wireless ad hoc networks. This is a fundamental problem in distributed computing because it abstracts the need to coordinate activities among various nodes. It has been shown to be a building block for several other important distributed computing problems like state-machine replication and atomic broadcast. The thesis begins by making a thorough performance assessment of existing intrusion-tolerant consensus protocols, which shows that the performance bottlenecks of current solutions are in part related to their system modeling assumptions. Based on these results, the communication failure model is identified as a model that simultaneously captures the reality of wireless ad hoc networks and allows the design of efficient protocols. Unfortunately, the model is subject to an impossibility result stating that there is no deterministic algorithm that allows n nodes to reach agreement if more than n2 omission transmission failures can occur in a communication step. This result is valid even under strict timing assumptions (i.e., a synchronous system). The thesis applies randomization techniques in increasingly weaker variants of this model, until an efficient intrusion-tolerant consensus protocol is achieved. The first variant simplifies the problem by restricting the number of nodes that may be at the source of a transmission failure at each communication step. An algorithm is designed that tolerates f dynamic nodes at the source of faulty transmissions in a system with a total of n 3f + 1 nodes. The second variant imposes no restrictions on the pattern of transmission failures. The proposed algorithm effectively circumvents the Santoro- Widmayer impossibility result for the first time. It allows k out of n nodes to decide despite dn 2 e(nk)+k2 omission failures per communication step. This algorithm also has the interesting property of guaranteeing safety during arbitrary periods of unrestricted message loss. The final variant shares the same properties of the previous one, but relaxes the model in the sense that the system is asynchronous and that a static subset of nodes may be malicious. The obtained algorithm, called Turquois, admits f < n 3 malicious nodes, and ensures progress in communication steps where dnf 2 e(n k f) + k 2. The algorithm is subject to a comparative performance evaluation against other intrusiontolerant protocols. The results show that, as the system scales, Turquois outperforms the other protocols by more than an order of magnitude.Esta tese investiga o problema do consenso tolerante a faltas acidentais e maliciosas em redes ad hoc sem fios. Trata-se de um problema fundamental que captura a essência da coordenação em actividades envolvendo vários nós de um sistema, sendo um bloco construtor de outros importantes problemas dos sistemas distribuídos como a replicação de máquina de estados ou a difusão atómica. A tese começa por efectuar uma avaliação de desempenho a protocolos tolerantes a intrusões já existentes na literatura. Os resultados mostram que as limitações de desempenho das soluções existentes estão em parte relacionadas com o seu modelo de sistema. Baseado nestes resultados, é identificado o modelo de falhas de comunicação como um modelo que simultaneamente permite capturar o ambiente das redes ad hoc sem fios e projectar protocolos eficientes. Todavia, o modelo é restrito por um resultado de impossibilidade que afirma não existir algoritmo algum que permita a n nós chegaram a acordo num sistema que admita mais do que n2 transmissões omissas num dado passo de comunicação. Este resultado é válido mesmo sob fortes hipóteses temporais (i.e., em sistemas síncronos) A tese aplica técnicas de aleatoriedade em variantes progressivamente mais fracas do modelo até ser alcançado um protocolo eficiente e tolerante a intrusões. A primeira variante do modelo, de forma a simplificar o problema, restringe o número de nós que estão na origem de transmissões faltosas. É apresentado um algoritmo que tolera f nós dinâmicos na origem de transmissões faltosas em sistemas com um total de n 3f + 1 nós. A segunda variante do modelo não impõe quaisquer restrições no padrão de transmissões faltosas. É apresentado um algoritmo que contorna efectivamente o resultado de impossibilidade Santoro-Widmayer pela primeira vez e que permite a k de n nós efectuarem progresso nos passos de comunicação em que o número de transmissões omissas seja dn 2 e(n k) + k 2. O algoritmo possui ainda a interessante propriedade de tolerar períodos arbitrários em que o número de transmissões omissas seja superior a . A última variante do modelo partilha das mesmas características da variante anterior, mas com pressupostos mais fracos sobre o sistema. Em particular, assume-se que o sistema é assíncrono e que um subconjunto estático dos nós pode ser malicioso. O algoritmo apresentado, denominado Turquois, admite f < n 3 nós maliciosos e assegura progresso nos passos de comunicação em que dnf 2 e(n k f) + k 2. O algoritmo é sujeito a uma análise de desempenho comparativa com outros protocolos na literatura. Os resultados demonstram que, à medida que o número de nós no sistema aumenta, o desempenho do protocolo Turquois ultrapassa os restantes em mais do que uma ordem de magnitude.FC

Paxos Made Wireless: Consensus in the Air

Many applications in low-power wireless networks require complex coordination between their members. Swarms of robots or sensors and actuators in industrial closed-loop control need to coordinate within short periods of time to execute tasks. Failing to agree on a common decision can cause substantial consequences, like system failures and threats to human life. Such applications require consensus algorithms to enable coordination. While consensus has been studied for wired networks decades ago, with, for example, Paxos and Raft, it remains an open problem in multi-hop low-power wireless networks due to the limited resources available and the high cost of established solutions.This paper presents Wireless Paxos, a fault-tolerant, network-wide consensus primitive for low-power wireless networks. It is a new flavor of Paxos, the most-used consensus protocol today, and is specifically designed to tackle the challenges of low-power wireless networks. By building on top of concurrent transmissions, it provides low-latency, high reliability, and guarantees on the consensus. Our results show that Wireless Paxos requires only 289 ms to complete a consensus between 188 nodes in testbed experiments. Furthermore, we show that Wireless Paxos\ua0stays consistent even when injecting controlled failures

Echo-CGC: A Communication-Efficient Byzantine-Tolerant Distributed Machine Learning Algorithm in Single-Hop Radio Network

In the past few years, many Byzantine-tolerant distributed machine learning (DML) algorithms have been proposed in the point-to-point communication model. In this paper, we focus on a popular DML framework - the parameter server computation paradigm and iterative learning algorithms that proceed in rounds, e.g., [Gupta and Vaidya, 2020; El-Mhamdi et al., 2020; Chen et al., 2017]. One limitation of prior algorithms in this domain is the high communication complexity. All the Byzantine-tolerant DML algorithms that we are aware of need to send n d-dimensional vectors from worker nodes to the parameter server in each round, where n is the number of workers and d is the number of dimensions of the feature space (which may be in the order of millions). In a wireless network, power consumption is proportional to the number of bits transmitted. Consequently, it is extremely difficult, if not impossible, to deploy these algorithms in power-limited wireless devices. Motivated by this observation, we aim to reduce the communication complexity of Byzantine-tolerant DML algorithms in the single-hop radio network [Alistarh et al., 2010; Bhandari and Vaidya, 2005; Koo, 2004]. Inspired by the CGC filter developed by Gupta and Vaidya, PODC 2020 [Gupta and Vaidya, 2020], we propose a gradient descent-based algorithm, Echo-CGC. Our main novelty is a mechanism to utilize the broadcast properties of the radio network to avoid transmitting the raw gradients (full d-dimensional vectors). In the radio network, each worker is able to overhear previous gradients that were transmitted to the parameter server. Roughly speaking, in Echo-CGC, if a worker "agrees" with a combination of prior gradients, it will broadcast the "echo message" instead of the its raw local gradient. The echo message contains a vector of coefficients (of size at most n) and the ratio of the magnitude between two gradients (a float). In comparison, the traditional approaches need to send n local gradients in each round, where each gradient is typically a vector in a ultra-high dimensional space (d ? n). The improvement on communication complexity of our algorithm depends on multiple factors, including number of nodes, number of faulty workers in an execution, and the cost function. We numerically analyze the improvement, and show that with a large number of nodes, Echo-CGC reduces 80% of the communication under standard assumptions

Intrusion Tolerant Routing Protocols for Wireless Sensor Networks

This MSc thesis is focused in the study, solution proposal and experimental evaluation of security solutions for Wireless Sensor Networks (WSNs). The objectives are centered on intrusion tolerant routing services, adapted for the characteristics and requirements of WSN nodes and operation behavior. The main contribution addresses the establishment of pro-active intrusion tolerance properties at the network level, as security mechanisms for the proposal of a reliable and secure routing protocol. Those properties and mechanisms will augment a secure communication base layer supported by light-weigh cryptography methods, to improve the global network resilience capabilities against possible intrusion-attacks on the WSN nodes. Adapting to WSN characteristics, the design of the intended security services also pushes complexity away from resource-poor sensor nodes towards resource-rich and trustable base stations. The devised solution will construct, securely and efficiently, a secure tree-structured routing service for data-dissemination in large scale deployed WSNs. The purpose is to tolerate the damage caused by adversaries modeled according with the Dolev-Yao threat model and ISO X.800 attack typology and framework, or intruders that can compromise maliciously the deployed sensor nodes, injecting, modifying, or blocking packets, jeopardizing the correct behavior of internal network routing processing and topology management. The proposed enhanced mechanisms, as well as the design and implementation of a new intrusiontolerant routing protocol for a large scale WSN are evaluated by simulation. For this purpose, the evaluation is based on a rich simulation environment, modeling networks from hundreds to tens of thousands of wireless sensors, analyzing different dimensions: connectivity conditions, degree-distribution patterns, latency and average short-paths, clustering, reliability metrics and energy cost

Intrusion tolerant routing with data consensus in wireless sensor networks

Dissertação para obtenção do Grau de Mestre em Engenharia InformáticaWireless sensor networks (WSNs) are rapidly emerging and growing as an important new area in computing and wireless networking research. Applications of WSNs are numerous, growing, and ranging from small-scale indoor deployment scenarios in homes and buildings to large scale outdoor deployment settings in natural, industrial, military and embedded environments. In a WSN, the sensor nodes collect data to monitor physical conditions or to measure and pre-process physical phenomena, and forward that data to special computing nodes called Syncnodes or Base Stations (BSs). These nodes are eventually interconnected, as gateways, to other processing systems running applications. In large-scale settings, WSNs operate with a large number of sensors – from hundreds to thousands of sensor nodes – organised as ad-hoc multi-hop or mesh networks, working without human supervision. Sensor nodes are very limited in computation, storage, communication and energy resources. These limitations impose particular challenges in designing large scale reliable and secure WSN services and applications. However, as sensors are very limited in their resources they tend to be very cheap. Resilient solutions based on a large number of nodes with replicated capabilities, are possible approaches to address dependability concerns, namely reliability and security requirements and fault or intrusion tolerant network services. This thesis proposes, implements and tests an intrusion tolerant routing service for large-scale dependable WSNs. The service is based on a tree-structured multi-path routing algorithm, establishing multi-hop and multiple disjoint routes between sensors and a group of BSs. The BS nodes work as an overlay, processing intrusion tolerant data consensus over the routed data. In the proposed solution the multiple routes are discovered, selected and established by a self-organisation process. The solution allows the WSN nodes to collect and route data through multiple disjoint routes to the different BSs, with a preventive intrusion tolerance approach, while handling possible Byzantine attacks and failures in sensors and BS with a pro-active recovery strategy supported by intrusion and fault tolerant data-consensus algorithms, performed by the group of Base Stations

Coordination and Self-Adaptive Communication Primitives for Low-Power Wireless Networks

The Internet of Things (IoT) is a recent trend where objects are augmented with computing and communication capabilities, often via low-power wireless radios. The Internet of Things is an enabler for a connected and more sustainable modern society: smart grids are deployed to improve energy production and consumption, wireless monitoring systems allow smart factories to detect faults early and reduce waste, while connected vehicles coordinate on the road to ensure our safety and save fuel. Many recent IoT applications have stringent requirements for their wireless communication substrate: devices must cooperate and coordinate, must perform efficiently under varying and sometimes extreme environments, while strict deadlines must be met. Current distributed coordination algorithms have high overheads and are unfit to meet the requirements of today\u27s wireless applications, while current wireless protocols are often best-effort and lack the guarantees provided by well-studied coordination solutions. Further, many communication primitives available today lack the ability to adapt to dynamic environments, and are often tuned during their design phase to reach a target performance, rather than be continuously updated at runtime to adapt to reality.In this thesis, we study the problem of efficient and low-latency consensus in the context of low-power wireless networks, where communication is unreliable and nodes can fail, and we investigate the design of a self-adaptive wireless stack, where the communication substrate is able to adapt to changes to its environment. We propose three new communication primitives: Wireless Paxos brings fault-tolerant consensus to low-power wireless networking, STARC is a middleware for safe vehicular coordination at intersections, while Dimmer builds on reinforcement learning to provide adaptivity to low-power wireless networks. We evaluate in-depth each primitive on testbed deployments and we provide an open-source implementation to enable their use and improvement by the community

New Distributed Byzantine Fault Detection & Data Integrity Scheme for WANET

Wireless ad-hoc networks (WANET) with multi-hop communication are subject to a variety of faults and attacks, and detecting the source of any fault is highly important to maintain the quality of service, confidentiality, and reliability of an entire network operation. Intermediate byzantine nodes in WANET could subvert the system by altering sensitive routed information unintentionally due to many reasons such as power depletion, software bug, malware, and environmental obstacles. This thesis highlights some of the research studies done in the area of distributed fault detection (DFD) and proposes a solution to detect Byzantine behavior cooperatively. The present research will focus on designing a scalable distributed fault detection (DFD) algorithm to detect byzantine nodes who permanently try to distort or reroute information while relaying a message from one node to another, complimentary to that, a symmetric distributed cryptography scheme will be employed to continuously validates the data integrity of a routed message. The main hypothesis of the research is that if a wireless ad-hoc network is been divided into N number of groups (classes) with relatively equal number of members, each group of nodes can cooperatively protect the network from every other group. Practically, each group of nodes will be assigned to a distinct shared key; nodes with similar group assignment shall guard the integrity of a routing path by incorporating their own secret message authentication code (MAC) that can be only validated by nodes belonging to the same group contributing to the same routing path. If a node from Group(i) detects a tampering event, it should either store and delay a fault report or embed a fault report to the same routed message and forward it to the Master Node (Destination) if applicable. Further report message overhead optimization has been devised to reduce the energy cost. Moreover, the empirical results have shown that the more reported evidence the master node can collect, the more accuracy of detection can be reached based on an incremental stream of evidence that contains information about both healthy and unhealthy nodes; so that every healthy report type can justify the unhealthy false report. The heuristic simulation based study considered many different aspects of the system for evaluation such as detection accuracy, fault model, the optimal number of classes, energy consumption, the impact of mobility, and network lifetime. The iGraph network simulation tool has been employed for visualization and graph manipulation, whereas, Python programming language has been utilized in conjunction to implement and simulate the DFD algorithm and generate the results

Resource-Efficient Communication in the Presence of Adversaries

This dissertation presents algorithms for achieving communication in the presence of adversarial attacks in large, decentralized, resource-constrained networks. We consider abstract single-hop communication settings where a set of senders wishes to directly communicate with a set of receivers . These results are then extended to provide resource-efficient, multi-hop communication in wireless sensor networks (WSNs), where energy is critically scarce, and peer-to-peer (P2P) networks, where bandwidth and computational power are limited. Our algorithms are provably correct in the face of attacks by a computationally bounded adversary who seeks to disrupt communication between correct participants. The first major result in this dissertation addresses a general scenario involving single-hop communication in a time-slotted network where a single sender in wishes to transmit a message to a single receiver in . The two players share a communication channel; however, there exists an adversary who aims to prevent the transmission of by periodically blocking this channel. There are costs to send, receive or block on the channel, and we ask: How much do the two players need to spend relative to the adversary in order to guarantee transmission of the message? This problem abstracts many types of conflict in information networks, and the associated costs represent an expenditure of network resources. We show that it is significantly more costly for the adversary to block than for the two players to achieve communication. Specifically, if the cost to send, receive and block in a slot are fixed constants, and the adversary spends a total of slots to try to block the message, then both the sender and receiver must be active in only O(ᵠ⁻¹ + 1) slots in expectation to transmit , where φ = (1+ √5)/2 is the golden ratio. Surprisingly, this result holds even if (1) the value of is unknown to either player; (2) the adversary knows the algorithms of both players, but not their random bits; and (3) the adversary is able to launch attacks using total knowledge of past actions of both players. Finally, these results are applied to two concrete problems. First, we consider jamming attacks in WSNs and address the fundamental task of propagating from a single device to all others in a WSN in the presence of faults; this is the problem of reliable broadcast. Second, we examine how our algorithms can mitigate application-level distributed denial-of-service attacks in wired client-server scenarios. The second major result deals with a single-hop communication problem where now consists of multiple senders and there is still a single receiver who wishes to obtain a message . However, many of the senders (strictly less than half) can be faulty, failing to send or sending incorrect messages. While the majority of the senders possess , rather than listening to all of and majority filtering on the received data, we desire an algorithm that allows the single receiver to decide on in a more efficient manner. To investigate this scenario, we define and devise algorithms for a new data streaming problem called the Bad Santa problem which models the selection dilemma faced by the receiver. With our results for the Bad Santa problem, we consider the problem of energy-efficient reliable broadcast. All previous results on reliable broadcast require devices to spend significant time in the energy-expensive receiving state which is a critical problem in WSNs where devices are typically battery powered. In a popular WSN model, we give a reliable broadcast protocol that achieves optimal fault tolerance (i.e., tolerates the maximum number of faults in this WSN model) and improves over previous results by achieving an expected quadratic decrease in the cost to each device. For the case where the number of faults is within a (1-∊)-factor of the optimal fault tolerance, for any constant ∊>0, we give a reliable broadcast protocol that improves further by achieving an expected (roughly) exponential decrease in the cost to each device. The third and final major result of this dissertation addresses single-hop communication where and both consist of multiple peers that need to communicate in an attack-resistant P2P network. There are several analytical results on P2P networks that can tolerate an adversary who controls a large number of peers and uses them to disrupt network functionality. Unfortunately, in such systems, operations such as data retrieval and message sending incur significant communication costs. Here, we employ cryptographic techniques to define two protocols both of which are more efficient than existing solutions. For a network of peers, our first protocol is deterministic with O(log²) message complexity and our second protocol is randomized with expected O(log ) message complexity; both improve over all previous results. The hidden constants and setup costs for our protocols are small and no trusted third party is required. Finally, we present an analysis showing that our protocols are practical for deployment under significant churn and adversarial behaviour

Byzantine Resilient Protocol for the IoT

Wireless sensor networks, often adhering to a single gateway architecture, constitute the communication backbone for many modern cyber-physical systems. Consequently, faulttolerance in CPS becomes a challenging task, especially when accounting for failures (potentially malicious) that incapacitate the gateway or disrupt the nodes-gateway communication, not to mention the energy, timeliness, and security constraints demanded by CPS domains. This paper aims at ameliorating the fault-tolerance of WSN based CPS to increase system and data availability. To this end, we propose a replicated gateway architecture augmented with energy-efficient real-time Byzantineresilient data communication protocols. At the sensors level, we introduce FT-TSTP, a geographic routing protocol capable of delivering messages in an energy-efficient and timely manner to multiple gateways, even in the presence of voids caused by faulty and malicious sensor nodes. At the gateway level, we propose a multi-gateway synchronization protocol, which we call ByzCast, that delivers timely correct data to CPS applications, despite the failure or maliciousness of a number of gateways. We show, through extensive simulations, that our protocols provide better system robustness yielding an increased system and data availability while meeting CPS energy, timeliness, and security demands

When Distributed Consensus Meets Wireless Connected Autonomous Systems: A Review and A DAG-based Approach

The connected and autonomous systems (CAS) and auto-driving era is coming into our life. To support CAS applications such as AI-driven decision-making and blockchain-based smart data management platform, data and message exchange/dissemination is a fundamental element. The distributed message broadcast and forward protocols in CAS, such as vehicular ad hoc networks (VANET), can suffer from significant message loss and uncertain transmission delay, and faulty nodes might disseminate fake messages to confuse the network. Therefore, the consensus mechanism is essential in CAS with distributed structure to guaranteed correct nodes agree on the same parameter and reach consistency. However, due to the wireless nature of CAS, traditional consensus cannot be directly deployed. This article reviews several existing consensus mechanisms, including average/maximum/minimum estimation consensus mechanisms that apply on quantity, Byzantine fault tolerance consensus for request, state machine replication (SMR) and blockchain, as well as their implementations in CAS. To deploy wireless-adapted consensus, we propose a Directed Acyclic Graph (DAG)-based message structure to build a non-equivocation data dissemination protocol for CAS, which has resilience against message loss and unpredictable forwarding latency. Finally, we enhance this protocol by developing a two-dimension DAG-based strategy to achieve partial order for blockchain and total order for the distributed service model SMR