Methods to Improve Applicability and Efficiency of Distributed Data-Centric Compute Frameworks

The success of modern applications depends on the insights they collect from their data repositories. Data repositories for such applications currently exceed exabytes and are rapidly increasing in size, as they collect data from varied sources - web applications, mobile phones, sensors and other connected devices. Distributed storage and data-centric compute frameworks have been invented to store and analyze these large datasets. This dissertation focuses on extending the applicability and improving the efficiency of distributed data-centric compute frameworks

Performance optimization and energy efficiency of big-data computing workflows

Next-generation e-science is producing colossal amounts of data, now frequently termed as Big Data, on the order of terabyte at present and petabyte or even exabyte in the predictable future. These scientific applications typically feature data-intensive workflows comprised of moldable parallel computing jobs, such as MapReduce, with intricate inter-job dependencies. The granularity of task partitioning in each moldable job of such big data workflows has a significant impact on workflow completion time, energy consumption, and financial cost if executed in clouds, which remains largely unexplored. This dissertation conducts an in-depth investigation into the properties of moldable jobs and provides an experiment-based validation of the performance model where the total workload of a moldable job increases along with the degree of parallelism. Furthermore, this dissertation conducts rigorous research on workflow execution dynamics in resource sharing environments and explores the interactions between workflow mapping and task scheduling on various computing platforms. A workflow optimization architecture is developed to seamlessly integrate three interrelated technical components, i.e., resource allocation, job mapping, and task scheduling. Cloud computing provides a cost-effective computing platform for big data workflows where moldable parallel computing models are widely applied to meet stringent performance requirements. Based on the moldable parallel computing performance model, a big-data workflow mapping model is constructed and a workflow mapping problem is formulated to minimize workflow makespan under a budget constraint in public clouds. This dissertation shows this problem to be strongly NP-complete and designs i) a fully polynomial-time approximation scheme for a special case with a pipeline-structured workflow executed on virtual machines of a single class, and ii) a heuristic for a generalized problem with an arbitrary directed acyclic graph-structured workflow executed on virtual machines of multiple classes. The performance superiority of the proposed solution is illustrated by extensive simulation-based results in Hadoop/YARN in comparison with existing workflow mapping models and algorithms. Considering that large-scale workflows for big data analytics have become a main consumer of energy in data centers, this dissertation also delves into the problem of static workflow mapping to minimize the dynamic energy consumption of a workflow request under a deadline constraint in Hadoop clusters, which is shown to be strongly NP-hard. A fully polynomial-time approximation scheme is designed for a special case with a pipeline-structured workflow on a homogeneous cluster and a heuristic is designed for the generalized problem with an arbitrary directed acyclic graph-structured workflow on a heterogeneous cluster. This problem is further extended to a dynamic version with deadline-constrained MapReduce workflows to minimize dynamic energy consumption in Hadoop clusters. This dissertation proposes a semi-dynamic online scheduling algorithm based on adaptive task partitioning to reduce dynamic energy consumption while meeting performance requirements from a global perspective, and also develops corresponding system modules for algorithm implementation in the Hadoop ecosystem. The performance superiority of the proposed solutions in terms of dynamic energy saving and deadline missing rate is illustrated by extensive simulation results in comparison with existing algorithms, and further validated through real-life workflow implementation and experiments using the Oozie workflow engine in Hadoop/YARN systems

A Blockchain Framework for Patient-Centered Health Records and Exchange (HealthChain): Evaluation and Proof-of-Concept Study

Background: Blockchain has the potential to disrupt the current modes of patient data access, accumulation, contribution, exchange, and control. Using interoperability standards, smart contracts, and cryptographic identities, patients can securely exchange data with providers and regulate access. The resulting comprehensive, longitudinal medical records can significantly improve the cost and quality of patient care for individuals and populations alike. Objective: This work presents HealthChain, a novel patient-centered blockchain framework. The intent is to bolster patient engagement, data curation, and regulated dissemination of accumulated information in a secure, interoperable environment. A mixed-block blockchain is proposed to support immutable logging and redactable patient blocks. Patient data are generated and exchanged through Health Level-7 Fast Healthcare Interoperability Resources, allowing seamless transfer with compliant systems. In addition, patients receive cryptographic identities in the form of public and private key pairs. Public keys are stored in the blockchain and are suitable for securing and verifying transactions. Furthermore, the envisaged system uses proxy re-encryption (PRE) to share information through revocable, smart contracts, ensuring the preservation of privacy and confidentiality. Finally, several PRE improvements are offered to enhance performance and security. Methods: The framework was formulated to address key barriers to blockchain adoption in health care, namely, information security, interoperability, data integrity, identity validation, and scalability. It supports 16 configurations through the manipulation of 4 modes. An open-source, proof-of-concept tool was developed to evaluate the performance of the novel patient block components and system configurations. To demonstrate the utility of the proposed framework and evaluate resource consumption, extensive testing was performed on each of the 16 configurations over a variety of scenarios involving a variable number of existing and imported records. Results: The results indicate several clear high-performing, low-bandwidth configurations, although they are not the strongest cryptographically. Of the strongest models, one’s anticipated cumulative record size is shown to influence the selection. Although the most efficient algorithm is ultimately user specific, Advanced Encryption Standard–encrypted data with static keys, incremental server storage, and no additional server-side encryption are the fastest and least bandwidth intensive, whereas proxy re-encrypted data with dynamic keys, incremental server storage, and additional server-side encryption are the best performing of the strongest configurations. Conclusions: Blockchain is a potent and viable technology for patient-centered access to and exchange of health information. By integrating a structured, interoperable design with patient-accumulated and generated data shared through smart contracts into a universally accessible blockchain, HealthChain presents patients and providers with access to consistent and comprehensive medical records. Challenges addressed include data security, interoperability, block storage, and patient-administered data access, with several configurations emerging for further consideration regarding speed and security

Renegotiation and recursion in Bitcoin contracts

BitML is a process calculus to express smart contracts that can be run on Bitcoin. One of its current limitations is that, once a contract has been stipulated, the participants cannot renegotiate its terms: this prevents expressing common financial contracts, where funds have to be added by participants at run-time. In this paper, we extend BitML with a new primitive for contract renegotiation. At the same time, the new primitive can be used to write recursive contracts, which was not possible in the original BitML. We show that, despite the increased expressiveness, it is still possible to execute BitML on standard Bitcoin, preserving the security guarantees of BitML.Comment: Full version of the paper presented at COORDINATION 202

Data Protection for the Internet of Things

The Internet of Things (abbreviated: “IoT”) is acknowledged as one of the most important disruptive technologies with more than 16 billion devices forecasted to interact autonomously by 2020. The idea is simple, devices will help to measure the status of physical objects. The devices, containing sensors and actuators, are so small that they can be integrated or attached to any object in order to measure that object and possibly change its status accordingly. A process or work flow is then able to interact with those devices and to control the objects physically. The result is the collection of massive data in a ubiquitous form. This data can be analysed to gain new insights, a benefit propagated by the “Big Data” and “Smart Data” paradigms. While governments, cities and industries are heavily involved in the Internet of Things, society’s privacy awareness and the concerns over data protection in IoT increase steadily. The scale of the collection, processing and dissemination of possibly private information in the Internet of Things has long begun to raise privacy concerns. The problem is a fundamental one, it is the massive data collection that benefits the investment on IoT, while it contradicts the interest on data minimization coming from privacy advocates. And the challenges go even further, while privacy is an actively researched topic with a mature variety of privacy preserving mechanisms, legal studies and surveillance studies in specific contexts, investigations of how to apply this concepts in the constrained environment of IoT have merely begun. Thus the objective of this thesis is threefold and tackles several topics, looking at them in a differentiated way and later bringing them together for one of the first, (more) complete pictures of privacy in IoT. The first starting point is the throughout study of stakeholders, impact areas and proposals on an architectural reference model for IoT. At the time of this writing, IoT was adversed heavily by several companies, products and even governments, creating a blurred picture of what IoT really is. This thesis surveys stakeholders, scenarios, architecture paradigms and definitions to find a working definition for IoT which adequately describes the intersection between all of the aforementioned topics. In a further step, the definition is applied exemplary on two scenarios to identify the common building blocks of those scenarios and of IoT in general. The building blocks are then verified against a similar approach by the IoT-A and Rerum projects and unified to an IoT domain model. This approach purposefully uses notions and paradigms provided in related scientific work and European projects in order to benefit from existing efforts and to achieve a common understanding. In this thesis, the observation of so called cyber-physical properties of IoT leads to the conclusion that IoT proposals miss a core concept of physical interaction in the “real world”. Accordingly, this thesis takes a detour to jurisdiction and identifies ownership and possession as a main concept of “human-to-object” relationships. The analysis of IoT building blocks ends with an enhanced IoT domain model. The next step breaks down “privacy by design”. Notably hereby is that privacy by design has been well integrated in to the new European General Data Protection Regulation (GDPR). This regulation heavily affects IoT and thus serves as the main source of privacy requirements. Gürses et al.’s privacy paradigm (privacy as confidentiality, privacy as control and privacy as practice) is used for the breakdown, preceded by a survey of relevant privacy proposals, where relevancy was measured upon previously identified IoT impact areas and stakeholders. Independently from IoT, this thesis shows that privacy engineering is a task that still needs to be well understood. A privacy development lifecycle was therefore sketched as a first step in this direction. Existing privacy technologies are part of the survey. Current research is summed up to show that while many schemes exist, few are adequate for actual application in IoT due to their high energy or computational consumption and high implementation costs (most notably caused by the implementation of special arithmetics). In an effort to give a first direction on possible new privacy enhancing technologies for IoT, new technical schemes are presented, formally verified and evaluated. The proposals comprise schemes, among others, on relaxed integrity protection, privacy friendly authentication and authorization as well as geo-location privacy. The schemes are presented to industry partners with positive results. This technologies have thus been published in academia and as intellectual property items. This thesis concludes by bringing privacy and IoT together. The final result is a privacy enhanced IoT domain model accompanied by a set of assumptions regarding stakeholders, economic impacts, economic and technical constraints as well as formally verified and evaluated proof of concept technologies for privacy in IoT. There is justifiable interest in IoT as it helps to tackle many future challenges found in several impact areas. At the same time, IoT impacts the stakeholders that participate in those areas, creating the need for unification of IoT and privacy. This thesis shows that technical and economic constraints do not impede such a process, although the process has merely begun

Building the Hyperconnected Society- Internet of Things Research and Innovation Value Chains, Ecosystems and Markets

This book aims to provide a broad overview of various topics of Internet of Things (IoT), ranging from research, innovation and development priorities to enabling technologies, nanoelectronics, cyber-physical systems, architecture, interoperability and industrial applications. All this is happening in a global context, building towards intelligent, interconnected decision making as an essential driver for new growth and co-competition across a wider set of markets. It is intended to be a standalone book in a series that covers the Internet of Things activities of the IERC – Internet of Things European Research Cluster from research to technological innovation, validation and deployment.The book builds on the ideas put forward by the European Research Cluster on the Internet of Things Strategic Research and Innovation Agenda, and presents global views and state of the art results on the challenges facing the research, innovation, development and deployment of IoT in future years. The concept of IoT could disrupt consumer and industrial product markets generating new revenues and serving as a growth driver for semiconductor, networking equipment, and service provider end-markets globally. This will create new application and product end-markets, change the value chain of companies that creates the IoT technology and deploy it in various end sectors, while impacting the business models of semiconductor, software, device, communication and service provider stakeholders. The proliferation of intelligent devices at the edge of the network with the introduction of embedded software and app-driven hardware into manufactured devices, and the ability, through embedded software/hardware developments, to monetize those device functions and features by offering novel solutions, could generate completely new types of revenue streams. Intelligent and IoT devices leverage software, software licensing, entitlement management, and Internet connectivity in ways that address many of the societal challenges that we will face in the next decade

Regulation for E-payment Systems - Analytical Approaches Beyond Private Ordering

Technology-driven payment instruments and services are facilitating the development of e-commerce; however, security concerns beleaguer their implementation, particularly in developing countries. This article considers the limits of private ordering in the regulation of e-payment systems. It uses Nigeria to exemplify a developing country that is increasingly pushing for the adoption of a regulatory framework for e-payment systems based on private ordering. It argues that, although technical standards and self-regulation by the financial industry are important, law is an essential regulatory mechanism that is largely absent. The article proposes that law be used as a mechanism to set and compel compliance with technical and industry standards, thus building trust, catering to public interest concerns and legitimizing the regulatory process

Proof-of-Prestige: A Useful Work Reward System for Unverifiable Tasks

As cryptographic tokens and altcoins are increasingly being built to serve as utility tokens, the notion of useful work consensus protocols is becoming ever more important. With useful work consensus protocols, users get rewards after they have carried out some specific tasks useful for the network. While in some cases the proof of some utility or service can be provided, the majority of tasks are impossible to verify reliably. To deal with such cases, we design “Proof-of-Prestige” (PoP)—a reward system that can run directly on Proof-of-Stake (PoS) blockchains or as a smart contract on top of Proof-of-Work (PoW) blockchains. PoP introduces “prestige,” which is a volatile resource that, in contrast to coins, regenerates over time. Prestige can be gained by performing useful work, spent when benefiting from services, and directly translates to users minting power. Our scheme allows us to reliably reward decentralized workers while keeping the system free for the end-users. PoP is resistant against Sybil and collusion attacks and can be used with a vast range of unverifiable tasks. We build a simulator to assess the cryptoeconomic behavior of the system and deploy a full prototype of a content dissemination platform rewarding its participants. We implement the blockchain component on both Ethereum (PoW) and Cosmos (PoS), provide a mobile application, and connect it with our scheme with a negligible memory footprint. Finally, we adapt a fair exchange protocol allowing us to atomically exchange files for rewards also in scenarios where not all the parties have Internet connectivity. Our evaluation shows that even for large Ethereum traces, PoP introduces sub-millisecond computational overhead for miners in Cosmos and less than 0.013$ smart contract invocation cost for users in Ethereum

Task Packing: Efficient task scheduling in unbalanced parallel programs to maximize CPU utilization

Load imbalance in parallel systems can be generated by external factors to the currently running applications like operating system noise or the underlying hardware like a heterogeneous cluster. HPC applications working on irregular data structures can also have difficulties to balance their computations across the parallel tasks. In this article we extend, improve and evaluate more deeply the Task Packing mechanism proposed in a previous work. The main idea of the mechanism is to concentrate the idle cycles of unbalanced applications in such a way that one or more CPUs are freed from execution. To achieve this, CPUs are stressed with just useful work of the parallel application tasks, provided performance is not degraded. The packing is solved by an algorithm based on the Knapsack problem, in a minimum number of CPUs and using oversubscription. We design and implement a more efficient version of such mechanism. To that end, we perform the Task Packing “in place”, taking advantage of idle cycles generated at synchronization points of unbalanced applications. Evaluations are carried out on a heterogeneous platform using FT and miniFE benchmarks. Results showed that our proposal generates low overhead. In addition the amount of freed CPUs are related to a load imbalance metric which can be used as a prediction for it.Peer ReviewedPostprint (author's final draft

Rights, Water, and Guardians: How Rights of Nature Movements are Reshaping our Current Environmental Ethics and What These Policies Need to be Successful

Giving legal rights to nature is no longer a fringe idea in international environmental law. Rights of Nature movements have gained traction in countries around the world, including Ecuador, Australia, India, Aotearoa New Zealand, and the United States. The act of organizing to recognize legal rights and legal personhood for nature represents a philosophical, moral, and political shift from previous anthropocentric values. Through two case studies in Aotearoa New Zealand and the United States, this thesis examines the policy language and the context and history that led to their creation. The Te Awa Tupua (Whanganui River Claims Settlement) Act and the Lake Erie Bill of Rights are two examples of movements and policies that created legal rights for a natural entity, a river, and a lake, respectively. My analysis of these two unique case studies illustrates some of the elements necessary for such policies to be implemented and enforced effectively: careful consideration of the local community and existing systems, a collaboration between marginalized groups and legislatures, and chosen leaders to oversee implementation and guardianship of the entity. Using the text of the legislation, court cases, press releases, and images, I analyze the impacts, both philosophical and practical, of these salient political and environmental movements