Move Fast and Meet Deadlines: Fine-grained Real-time Stream Processing with Cameo

Resource provisioning in multi-tenant stream processing systems faces the dual challenges of keeping resource utilization high (without over-provisioning), and ensuring performance isolation. In our common production use cases, where streaming workloads have to meet latency targets and avoid breaching service-level agreements, existing solutions are incapable of handling the wide variability of user needs. Our framework called Cameo uses fine-grained stream processing (inspired by actor computation models), and is able to provide high resource utilization while meeting latency targets. Cameo dynamically calculates and propagates priorities of events based on user latency targets and query semantics. Experiments on Microsoft Azure show that compared to state-of-the-art, the Cameo framework: i) reduces query latency by 2.7X in single tenant settings, ii) reduces query latency by 4.6X in multi-tenant scenarios, and iii) weathers transient spikes of workload

Usability heuristics for fast crime data anonymization in resource-constrained contexts

This thesis considers the case of mobile crime-reporting systems that have emerged as an effective and efficient data collection method in low and middle-income countries. Analyzing the data, can be helpful in addressing crime. Since law enforcement agencies in resource-constrained context typically do not have the expertise to handle these tasks, a cost-effective strategy is to outsource the data analytics tasks to third-party service providers. However, because of the sensitivity of the data, it is expedient to consider the issue of privacy. More specifically, this thesis considers the issue of finding low-intensive computational solutions to protecting the data even from an "honest-but-curious" service provider, while at the same time generating datasets that can be queried efficiently and reliably. This thesis offers a three-pronged solution approach. Firstly, the creation of a mobile application to facilitate crime reporting in a usable, secure and privacy-preserving manner. The second step proposes a streaming data anonymization algorithm, which analyses reported data based on occurrence rate rather than at a preset time on a static repository. Finally, in the third step the concept of using privacy preferences in creating anonymized datasets was considered. By taking into account user preferences the efficiency of the anonymization process is improved upon, which is beneficial in enabling fast data anonymization. Results from the prototype implementation and usability tests indicate that having a usable and covet crime-reporting application encourages users to declare crime occurrences. Anonymizing streaming data contributes to faster crime resolution times, and user privacy preferences are helpful in relaxing privacy constraints, which makes for more usable data from the querying perspective. This research presents considerable evidence that the concept of a three-pronged solution to addressing the issue of anonymity during crime reporting in a resource-constrained environment is promising. This solution can further assist the law enforcement agencies to partner with third party in deriving useful crime pattern knowledge without infringing on users' privacy. In the future, this research can be extended to more than one low-income or middle-income countries

Real-time Adaptive Sensor Attack Detection and Recovery in Autonomous Cyber-physical Systems

Cyber-Physical Systems (CPS) tightly couple information technology with physical processes, which rises new vulnerabilities such as physical attacks that are beyond conventional cyber attacks.Attackers may non-invasively compromise sensors and spoof the controller to perform unsafe actions. This issue is even emphasized with the increasing autonomy in CPS. While this fact has motivated many defense mechanisms against sensor attacks, a clear vision of the timing and usability (or the false alarm rate) of attack detection still remains elusive. Existing works tend to pursue an unachievable goal of minimizing the detection delay and false alarm rate at the same time, while there is a clear trade-off between the two metrics. Instead, this dissertation argues that attack detection should bias different metrics (detection delay and false alarm) when a system sits in different states. For example, if the system is close to unsafe states, reducing the detection delay is preferable to lowering the false alarm rate, and vice versa. This dissertation proposes two real-time adaptive sensor attack detection frameworks. The frameworks can dynamically adapt the detection delay and false alarm rate so as to meet a detection deadline and improve usability according to different system statuses. We design and implement the proposed frameworks and validate them using realistic sensor data of automotive CPS to demonstrate its efficiency and efficacy. Further, this dissertation proposes \textit{Recovery-by-Learning}, a data-driven attack recovery framework that restores CPS from sensor attacks. The importance of attack recovery is emphasized by the need to mitigate the attack\u27s impact on a system and restore it to continue functioning. We propose a double sliding window-based checkpointing protocol to remove compromised data and keep trustful data for state estimation. Together, the proposed solutions enable a holistic attack resilient solution for automotive cyber-physical systems

Metrics and Algorithms for Processing Multiple Continuous Queries

Data streams processing is an emerging research area that is driven by the growing need for monitoring applications. A monitoring application continuously processes streams of data for interesting, significant, or anomalous events. Such applications include tracking the stock market, real-time detection of diseaseoutbreaks, and environmental monitoring via sensor networks.Efficient employment of those monitoring applications requires advanced data processing techniques that can support the continuous processing of unbounded rapid data streams. Such techniques go beyond the capabilities of the traditional store-then-query Data BaseManagement Systems. This need has led to a new data processing paradigm and created a new generation of data processing systems,supporting continuous queries (CQ) on data streams.Primary emphasis in the development of first generation Data Stream Management Systems (DSMSs) was given to basic functionality. However, in order to support large-scale heterogeneous applications that are envisioned for subsequent generations of DSMSs, greater attention willhave to be paid to performance issues. Towards this, this thesis introduces new algorithms and metrics to the current design of DSMSs.This thesis identifies a collection of quality ofservice (QoS) and quality of data (QoD) metrics that are suitable for a wide range of monitoring applications. The establishment of well-defined metrics aids in the development of novel algorithms that are optimal with respect to a particular metric. Our proposed algorithms exploit the valuable chances for optimization that arise in the presence of multiple applications. Additionally, they aim to balance the trade-off between the DSMS's overall performance and the performance perceived by individual applications. Furthermore, we provide efficient implementations of the proposed algorithms and we also extend them to exploit sharing in optimized multi-query plans and multi-stream CQs. Finally, we experimentally show that our algorithms consistently outperform the current state of the art

Long-Term Stable Communication in Centrally Scheduled Low-Power Wireless Networks

With the emergence of the Internet of Things (IoT), more devices are connected than ever before. Most of these communicate wirelessly, forming Wireless Sensor Networks. In recent years, there has been a shift from personal networks, like Smart Home, to industrial networks. Industrial networks monitor pipelines or handle the communication between robots in factories. These new applications form the Industrial Internet of Things (IIoT). Many industrial applications have high requirements for communication, higher than the requirements of common IoT networks. Communications must stick to hard deadlines to avoid harm, and they must be highly reliable as skipping information is not a viable option when communicating critical information. Moreover, communication has to remain reliable over longer periods of time. As many sensor locations do not offer a power source, the devices have to run on battery and thus have to be power efficient. Current systems offer solutions for some of these requirements. However, they especially lack long-term stable communication that can dynamically adapt to changes in the wireless medium.In this thesis, we study the problem of stable and reliable communication in centrally scheduled low-power wireless networks. This communication ought to be stable when it can dynamically adapt to changes in the wireless medium while keeping latency at a minimum. We design and investigate approaches to solve the problem of low to high degrees of interference in the wireless medium. We propose three solutions to overcome interference: MASTER with Sliding Windows brings dynamic numbers of retransmissions to centrally scheduled low-power wireless networks, OVERTAKE allows to skip nodes affected by interference along the path, and AUTOBAHN combines opportunistic routing and synchronous transmissions with the Time-Slotted Channel Hopping (TSCH) MAC protocol to overcome local wide-band interference with the lowest possible latency. We evaluate our approaches in detail on testbed deployments and provide open-source implementations of the protocols to enable others to build their work upon them

Resource provisioning and scheduling algorithms for hybrid workflows in edge cloud computing

In recent years, Internet of Things (IoT) technology has been involved in a wide range of application domains to provide real-time monitoring, tracking and analysis services. The worldwide number of IoT-connected devices is projected to increase to 43 billion by 2023, and IoT technologies are expected to engaged in 25% of business sector. Latency-sensitive applications in scope of intelligent video surveillance, smart home, autonomous vehicle, augmented reality, are all emergent research directions in industry and academia. These applications are required connecting large number of sensing devices to attain the desired level of service quality for decision accuracy in a sensitive timely manner. Moreover, continuous data stream imposes processing large amounts of data, which adds a huge overhead on computing and network resources. Thus, latency-sensitive and resource-intensive applications introduce new challenges for current computing models, i.e, batch and stream. In this thesis, we refer to the integrated application model of stream and batch applications as a hybrid work ow model. The main challenge of the hybrid model is achieving the quality of service (QoS) requirements of the two computation systems. This thesis provides a systemic and detailed modeling for hybrid workflows which describes the internal structure of each application type for purposes of resource estimation, model systems tuning, and cost modeling. For optimizing the execution of hybrid workflows, this thesis proposes algorithms, techniques and frameworks to serve resource provisioning and task scheduling on various computing systems including cloud, edge cloud and cooperative edge cloud. Overall, experimental results provided in this thesis demonstrated strong evidences on the responsibility of proposing different understanding and vision on the applications of integrating stream and batch applications, and how edge computing and other emergent technologies like 5G networks and IoT will contribute on more sophisticated and intelligent solutions in many life disciplines for more safe, secure, healthy, smart and sustainable society