Towards Scalable, Cloud Based, Confidential Data Stream Processing

Increasing data availability, velocity, variability, and size have lead to the development of new data processing paradigms that offer users different ways to process and manage data specific to their needs. One such paradigm is data stream processing, as managed by Data Stream Processing Systems (DSPS). In contrast to traditional database management systems wherein data is stationary and queries are transient, in stream processing systems, data is transient and queries are stationary (that is, continuous and long running). In such systems, users are expecting to process temporal data, where data is only considered for some period of time, and discarded after. Often, as with many other software applications, those who employ such systems will outsource computation to third party computation platforms such as Amazon, IBM, or Google. The use of third parties not only outsources computation, but it outsources hardware and software maintenance costs as well, relieving the user from having to incur these costs themselves. Moreover, when a user outsources their DSPS, they often have some service level agreement that places guarantees on service availability and uptime. Given the above benefits to outsourcing computation, it is clearly desirable for a user to outsource their DSPS computation. Such outsourcing, however, may violate the privacy constraints of the those who provide the data stream. Specifically, they may not wish to share their plaintext data with a third-party that they may not trust. This leads to an interesting dichotomy between the desire of the user to outsource as much of their computation as possible and the desire of the data stream providers to keep their data private and avoid leaking data to a third-party system. Current work that explores linking the two poles of this dichotomy either limits the expressiveness of supported queries, requires the data provider to trust the third-party systems, or incurs computational or monetary overheads prohibitive for the querier. In this dissertation, we explore the methods for shrinking the gap between the poles of this dichotomy and overcome the limitation of the state-of-the art systems by providing data providers and queriers with efficient access control enforcement on untrusted third party systems over encrypted data. Specifically, we introduce our system PolyStream for executing queries on encrypted data using computation-enabling encryption, with an online key management system. We further introduce Sanctuary to provide computation on any data on third-party systems using trusted hardware. Finally we introduce Shoal, our query optimizer that considers the heterogeneous nature of streaming systems at optimization time to improve query performance when access controls are enforced on the streaming data. Through the union of the contributions of this dissertation, we show that considering access controls at optimization time can lead to better utilization, performance, and protection for streaming data

A security-and quality-aware system architecture for Internet of Things

Internet of Things (IoT) is characterized, at the system level, by high diversity with respect to enabling technologies and supported services. IoT also assumes to deal with a huge amount of heterogeneous data generated by devices, transmitted by the underpinning infrastructure and processed to support value-added services. In order to provide users with valuable output, the IoT architecture should guarantee the suitability and trustworthiness of the processed data. This is a major requirement of such systems in order to guarantee robustness and reliability at the service level. In this paper, we introduce a novel IoT architecture able to support security, privacy and data quality guarantees, thereby effectively boosting the diffusion of IoT services

Secure Shared Continuous Query Processing

Data Stream Management Systems (DSMSs) are being used in diverse application domains (e.g., stock trading), however, the need for processing data securely is becoming critical to several stream applications (e.g., patient monitoring). In this paper, we introduce a novel three stage (pre-processing, query processing, and post-processing) framework to enforce access control in DSMSs. As opposed to existing systems, our framework allows continuous queries to be shared when they have same or different privileges, does not modify the query plans, introduces no new security operators, and checks a tuple only once irrespective of the number of active continuous queries. In addition, it does not affect the DSMS quality of service improvement mechanisms as query plans are not modified. We discuss the prototype implementation using the MavStream Data Stream Management System. Finally, we discuss experimental evaluations to demonstrate the low overhead and feasibility of our approach

Challenges and Opportunities in Applying Semantics to Improve Access Control in the Field of Internet of Things

The increased number of IoT devices results in continuously generated massive amounts of raw data. Parts of this data are private and highly sensitive as they reflect owner’s behavior, obligations, habits, and preferences. In this paper, we point out that flexible and comprehensive access control policies are “a must” in the IoT domain. The Semantic Web technologies can address many of the challenges that the IoT access control is facing with today. Therefore, we analyze the current state of the art in this area and identify the challenges and opportunities for improved access control in a semantically enriched IoT environment. Applying semantics to IoT access control opens a lot of opportunities, such as semantic inference and reasoning, easy data sharing, data trading, new approaches to authentication, security policies based on a natural language and enhances the interoperability using a common ontology

Data stream processing meets the Advanced Metering Infrastructure: possibilities, challenges and applications

Distribution of electricity is changing.Energy production is increasingly distributed, weather dependent and located in the distribution network, close to consumers.Energy consumption is increasing throughout society and the electrification of transportation is driving distribution networks closer to the limits.Operating the networks closer to their limits also increases the risk for faults.Continuous monitoring of the distribution network closest to the customers is needed in order to mitigate this risk.The Advanced Metering Infrastructure introduced smart meters throughout the distribution network.Data stream processing is a computing paradigm that offers low latency results from analysis on large volumes of the data.This thesis investigates the possibilities and challenges for continuous monitoring that are created when the Advanced Metering Infrastructure and data stream processing meet.The challenges that are addressed in the thesis are efficient processing of unordered (also called out-of-order) data and efficient usage of the computational resources present in the Advanced Metering Infrastructure.Contributions towards more efficient processing of out-of-order data are made with eChIDNA and TinTiN. Both are systems that utilize knowledge about smart meter data to directly produce results where possible and storing only data that is relevant for late data in order to produce updated results when such late data arrives. eChIDNA is integrated in the streaming query itself, while TinTiN is a streaming middleware that can be applied to streaming queries in order to make them resilient against out-of-order data.Eventual determinism is defined in order to formally investigate the deterministic properties of output produced by such systems.Contributions towards efficient usage of the computational resources of the Advanced Metering Infrastructure are made with the application LoCoVolt.LoCoVolt implements a monitoring algorithm that can run on equipment that is localized in the communication infrastructure of the Advanced Metering Infrastructure and can take advantage of the overlap between the communication and distribution networks.All contributions are evaluated on hardware that is available in current AMI systems, using large scale data obtained from a real production AMI

A Data-Descriptive Feedback Framework for Data Stream Management Systems

Data Stream Management Systems (DSMSs) provide support for continuous query evaluation over data streams. Data streams provide processing challenges due to their unbounded nature and varying characteristics, such as rate and density fluctuations. DSMSs need to adapt stream processing to these changes within certain constraints, such as available computational resources and minimum latency requirements in producing results. The proposed research develops an inter-operator feedback framework, where opportunities for run-time adaptation of stream processing are expressed in terms of descriptions of substreams and actions applicable to the substreams, called feedback punctuations. Both the discovery of adaptation opportunities and the exploitation of these opportunities are performed in the query operators. DSMSs are also concerned with state management, in particular, state derived from tuple processing. The proposed research also introduces the Contracts Framework, which provides execution guarantees about state purging in continuous query evaluation for systems with and without inter-operator feedback. This research provides both theoretical and design contributions. The research also includes an implementation and evaluation of the feedback techniques in the NiagaraST DSMS, and a reference implementation of the Contracts Framework