VIoLET: A Large-scale Virtual Environment for Internet of Things

IoT deployments have been growing manifold, encompassing sensors, networks, edge, fog and cloud resources. Despite the intense interest from researchers and practitioners, most do not have access to large-scale IoT testbeds for validation. Simulation environments that allow analytical modeling are a poor substitute for evaluating software platforms or application workloads in realistic computing environments. Here, we propose VIoLET, a virtual environment for defining and launching large-scale IoT deployments within cloud VMs. It offers a declarative model to specify container-based compute resources that match the performance of the native edge, fog and cloud devices using Docker. These can be inter-connected by complex topologies on which private/public networks, and bandwidth and latency rules are enforced. Users can configure synthetic sensors for data generation on these devices as well. We validate VIoLET for deployments with > 400 devices and > 1500 device-cores, and show that the virtual IoT environment closely matches the expected compute and network performance at modest costs. This fills an important gap between IoT simulators and real deployments.Comment: To appear in the Proceedings of the 24TH International European Conference On Parallel and Distributed Computing (EURO-PAR), August 27-31, 2018, Turin, Italy, europar2018.org. Selected as a Distinguished Paper for presentation at the Plenary Session of the conferenc

A State-Based Proactive Approach To Network Isolation Verification In Clouds

The multi-tenancy nature of public clouds usually leads to cloud tenants' concerns over network isolation around their virtual resources. Verifying network isolation in clouds faces unique challenges. The sheer size of virtual infrastructures paired with the self-serviced nature of clouds means the verification will likely have a high complexity and yet its results may become obsolete in seconds. Moreover, the _ne-grained and distributed network access control (e.g., per-VM security group rules) typical to virtual cloud infrastructures means the verification must examine not only the events but also the current state of the infrastructures. In this thesis, we propose VMGuard, a state-based proactive approach for efficiently verifying large-scale virtual infrastructures against network isolation policies. Informally, our key idea is to proactively trigger the verification based on predicted events and their simulated impact upon the current state, such that we can have the best of both worlds, i.e., the efficiency of a proactive approach and the effectiveness of state-based verification. We implement and evaluate VMGuard based on OpenStack, and our experiments with both real and synthetic data demonstrate the performance and efficiency

Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management

As users of big data applications expect fresh results, we witness a new breed of stream processing systems (SPS) that are designed to scale to large numbers of cloud-hosted machines. Such systems face new challenges: (i) to benefit from the pay-as-you-go model of cloud computing, they must scale out on demand, acquiring additional virtual machines (VMs) and parallelising operators when the workload increases; (ii) failures are common with deployments on hundreds of VMs - systems must be fault-tolerant with fast recovery times, yet low per-machine overheads. An open question is how to achieve these two goals when stream queries include stateful operators, which must be scaled out and recovered without affecting query results. Our key idea is to expose internal operator state explicitly to the SPS through a set of state management primitives. Based on them, we describe an integrated approach for dynamic scale out and recovery of stateful operators. Externalised operator state is checkpointed periodically by the SPS and backed up to upstream VMs. The SPS identifies individual operator bottlenecks and automatically scales them out by allocating new VMs and partitioning the check-pointed state. At any point, failed operators are recovered by restoring checkpointed state on a new VM and replaying unprocessed tuples. We evaluate this approach with the Linear Road Benchmark on the Amazon EC2 cloud platform and show that it can scale automatically to a load factor of L=350 with 50 VMs, while recovering quickly from failures. Copyright © 2013 ACM

Analyzing challenging aspects of IPv6 over IPv4

The exponential expansion of the Internet has exhausted the IPv4 addresses provided by IANA. The new IP edition, i.e. IPv6 introduced by IETF with new features such as a simplified packet header, a greater address space, a different address sort, improved encryption, powerful section routing, and stronger QoS. ISPs are slowly seeking to migrate from current IPv4 physical networks to new generation IPv6 networks. ‎The move from actual IPv4 to software-based IPv6 is very sluggish, since billions of computers across the globe use IPv4 addresses. The configuration and actions of IP4 and IPv6 protocols are distinct. Direct correspondence between IPv4 and IPv6 is also not feasible. In terms of the incompatibility problems, all protocols can co-exist throughout the transformation for a few years. Compatibility, interoperability, and stability are key concerns between IP4 and IPv6 protocols. After the conversion of the network through an IPv6, the move causes several issues for ISPs. The key challenges faced by ISPs are packet traversing, routing scalability, performance reliability, and protection. Within this study, we meticulously analyzed a detailed overview of all aforementioned issues during switching into ipv6 network