Context-aware task scheduling in distributed computing systems

These days, the popularity of technologies such as machine learning, augmented reality, and big data analytics is growing dramatically. This leads to a higher demand of computational power not only for IT professionals but also for ordinary device users who benefit from new applications. At the same time, the computational performance of end-user devices increases to meet the demands of these resource-hungry applications. As a result, there is a coexistence of a huge demand of computational power on the one side and a large pool of computational resources on the other side. Bringing these two sides together is the idea of computational resource sharing systems which allow applications to forward computationally intensive workload to remote resources. This technique is often used in cloud computing where customers can rent computational power. However, we argue that not only cloud resources can be used as offloading targets. Rather, idle CPU cycles from end-user administered devices at the edge of the network can be spontaneously leveraged as well. Edge devices, however, are not only heterogeneous in their hardware and software capabilities, they also do not provide any guarantees in terms of reliability or performance. Does it mean that either the applications that require further guarantees or the unpredictable resources need to be excluded from such a sharing system? In this thesis, we propose a solution to this problem by introducing the Tasklet system, our approach for a computational resource sharing system. The Tasklet system supports computation offloading to arbitrary types of devices, including stable cloud instances as well as unpredictable end-user owned edge resources. Therefore, the Tasklet system is structured into multiple layers. The lowest layer is a best-effort resource sharing system which provides lightweight task scheduling and execution. Here, best-effort means that in case of a failure, the task execution is dropped and that tasks are allocated to resources randomly. To provide execution guarantees such as a reliable or timely execution, we add a Quality of Computation (QoC) layer on top of the best-effort execution layer. The QoC layer enforces the guarantees for applications by using a context-aware task scheduler which monitors the available resources in the computing environment and performs the matchmaking between resources and tasks based on the current state of the system. As edge resources are controlled by individuals, we consider the fact that these users need to be able to decide with whom they want to share their resources and for which price. Thus, we add a social layer on top of the system that allows users to establish friendship connections which can then be leveraged for social-aware task allocation and accounting of shared computation

Analysis Of Aircraft Arrival Delay And Airport On-time Performance

While existing grid environments cater to specific needs of a particular user community, we need to go beyond them and consider general-purpose large-scale distributed systems consisting of large collections of heterogeneous computers and communication systems shared by a large user population with very diverse requirements. Coordination, matchmaking, and resource allocation are among the essential functions of large-scale distributed systems. Although deterministic approaches for coordination, matchmaking, and resource allocation have been well studied, they are not suitable for large-scale distributed systems due to the large-scale, the autonomy, and the dynamics of the systems. We have to seek for nondeterministic solutions for large-scale distributed systems. In this dissertation we describe our work on a coordination service, a matchmaking service, and a macro-economic resource allocation model for large-scale distributed systems. The coordination service coordinates the execution of complex tasks in a dynamic environment, the matchmaking service supports finding the appropriate resources for users, and the macro-economic resource allocation model allows a broker to mediate resource providers who want to maximize their revenues and resource consumers who want to get the best resources at the lowest possible price, with some global objectives, e.g., to maximize the resource utilization of the system

Sharing and the City

The sharing of public infrastructure, the exchange of small services, and the traditional cup of sugar borrowed from the neighbor are practices intrinsic to most urban agglomerations. In the digital age, these sharing initiatives are facilitated by online platforms such as Feastly, Peerby, and HomeExchange. These platforms allow city residents to share the idle capacity of some of their assets (e.g., clothing, tools, or a spare bedroom) with other residents living in close proximity to them, or with tourists looking for accommodation. While these practices can be justified by efficiency and sustainability concerns, some of them appear to be in conflict with longstanding regulations on local transportation, food safety, zoning, taxation, and short-term accommodation. This Article explores urban peer-to-peer sharing practices from a comparative perspective and discusses how a number of large cities in Europe, the United States, and Asia are currently addressing the regulatory challenges inherent to sharing platforms. We argue that cities should rethink the irregulations in light of this new form of urban sharing. The legal literature has thus far conveyed an incomplete image of the sharing economy by focusing on controversial platforms such as Uber and their ongoing lawsuits. In this Article, we reestablish the historical, economic, and legal meaning of genuine urban sharing. First, this Article distinguishes between genuinely collaborative initiatives that promote the sharing of underutilized assets (e.g., spare guestrooms) and non-collaborative platforms that are not driven by sustainable consumption (e.g., Uber). Second, it provides an overview of the economic and geographic sharing potential of cities and discusses how outdated regulations might restrict it. Third, drawing on the experience of the so-called sharing cities (e.g., Seoul), it suggests a new legal framework for the regulation of genuine sharing practices. In this context, we argue that cities should, in some cases, experiment with the regulation of sustainable sharing initiatives in order to gather more information as to their benefits or risks, and, in other cases, engage in collaborative decision-making by negotiating the content of new legal provisions and policies with digital platforms

Incentive-driven QoS in peer-to-peer overlays

A well known problem in peer-to-peer overlays is that no single entity has control over the software, hardware and configuration of peers. Thus, each peer can selfishly adapt its behaviour to maximise its benefit from the overlay. This thesis is concerned with the modelling and design of incentive mechanisms for QoS-overlays: resource allocation protocols that provide strategic peers with participation incentives, while at the same time optimising the performance of the peer-to-peer distribution overlay. The contributions of this thesis are as follows. First, we present PledgeRoute, a novel contribution accounting system that can be used, along with a set of reciprocity policies, as an incentive mechanism to encourage peers to contribute resources even when users are not actively consuming overlay services. This mechanism uses a decentralised credit network, is resilient to sybil attacks, and allows peers to achieve time and space deferred contribution reciprocity. Then, we present a novel, QoS-aware resource allocation model based on Vickrey auctions that uses PledgeRoute as a substrate. It acts as an incentive mechanism by providing efficient overlay construction, while at the same time allocating increasing service quality to those peers that contribute more to the network. The model is then applied to lagsensitive chunk swarming, and some of its properties are explored for different peer delay distributions. When considering QoS overlays deployed over the best-effort Internet, the quality received by a client cannot be adjudicated completely to either its serving peer or the intervening network between them. By drawing parallels between this situation and well-known hidden action situations in microeconomics, we propose a novel scheme to ensure adherence to advertised QoS levels. We then apply it to delay-sensitive chunk distribution overlays and present the optimal contract payments required, along with a method for QoS contract enforcement through reciprocative strategies. We also present a probabilistic model for application-layer delay as a function of the prevailing network conditions. Finally, we address the incentives of managed overlays, and the prediction of their behaviour. We propose two novel models of multihoming managed overlay incentives in which overlays can freely allocate their traffic flows between different ISPs. One is obtained by optimising an overlay utility function with desired properties, while the other is designed for data-driven least-squares fitting of the cross elasticity of demand. This last model is then used to solve for ISP profit maximisation

Idle Computing Resources as Micro-Currencies -- Bartering CPU Time for Online Content

In this paper we present a brokerage system for using idle computing resources as a micro-currency for Web services hardly quantifyable with real currencies as downloading stock charts or accessing a search engine. The Locust (LOw cost Computing Utilizing Skimmed idle Time) system acts as a Broker for micro-payable services by demanding a part of the user's idle computing resources in exchange for accessing the service. We introduce the price/market model of Locust enabling the collection, aggregation and trade of resource surpluses or lacks between coupled sub-markets and eventually leading to an electronic resource market focused on the mainstream Internet user as supplier of idle computing time. We further describe Locust's inter-operable, secure and ubiquitious Web and Java based infrastructure to make micro-paying with CPU cycles for online content literally as easy as surfing a Web page with embedded banner advertisements

DRIVE: A Distributed Economic Meta-Scheduler for the Federation of Grid and Cloud Systems

The computational landscape is littered with islands of disjoint resource providers including commercial Clouds, private Clouds, national Grids, institutional Grids, clusters, and data centers. These providers are independent and isolated due to a lack of communication and coordination, they are also often proprietary without standardised interfaces, protocols, or execution environments. The lack of standardisation and global transparency has the effect of binding consumers to individual providers. With the increasing ubiquity of computation providers there is an opportunity to create federated architectures that span both Grid and Cloud computing providers effectively creating a global computing infrastructure. In order to realise this vision, secure and scalable mechanisms to coordinate resource access are required. This thesis proposes a generic meta-scheduling architecture to facilitate federated resource allocation in which users can provision resources from a range of heterogeneous (service) providers. Efficient resource allocation is difficult in large scale distributed environments due to the inherent lack of centralised control. In a Grid model, local resource managers govern access to a pool of resources within a single administrative domain but have only a local view of the Grid and are unable to collaborate when allocating jobs. Meta-schedulers act at a higher level able to submit jobs to multiple resource managers, however they are most often deployed on a per-client basis and are therefore concerned with only their allocations, essentially competing against one another. In a federated environment the widespread adoption of utility computing models seen in commercial Cloud providers has re-motivated the need for economically aware meta-schedulers. Economies provide a way to represent the different goals and strategies that exist in a competitive distributed environment. The use of economic allocation principles effectively creates an open service market that provides efficient allocation and incentives for participation. The major contributions of this thesis are the architecture and prototype implementation of the DRIVE meta-scheduler. DRIVE is a Virtual Organisation (VO) based distributed economic metascheduler in which members of the VO collaboratively allocate services or resources. Providers joining the VO contribute obligation services to the VO. These contributed services are in effect membership “dues” and are used in the running of the VOs operations – for example allocation, advertising, and general management. DRIVE is independent from a particular class of provider (Service, Grid, or Cloud) or specific economic protocol. This independence enables allocation in federated environments composed of heterogeneous providers in vastly different scenarios. Protocol independence facilitates the use of arbitrary protocols based on specific requirements and infrastructural availability. For instance, within a single organisation where internal trust exists, users can achieve maximum allocation performance by choosing a simple economic protocol. In a global utility Grid no such trust exists. The same meta-scheduler architecture can be used with a secure protocol which ensures the allocation is carried out fairly in the absence of trust. DRIVE establishes contracts between participants as the result of allocation. A contract describes individual requirements and obligations of each party. A unique two stage contract negotiation protocol is used to minimise the effect of allocation latency. In addition due to the co-op nature of the architecture and the use of secure privacy preserving protocols, DRIVE can be deployed in a distributed environment without requiring large scale dedicated resources. This thesis presents several other contributions related to meta-scheduling and open service markets. To overcome the perceived performance limitations of economic systems four high utilisation strategies have been developed and evaluated. Each strategy is shown to improve occupancy, utilisation and profit using synthetic workloads based on a production Grid trace. The gRAVI service wrapping toolkit is presented to address the difficulty web enabling existing applications. The gRAVI toolkit has been extended for this thesis such that it creates economically aware (DRIVE-enabled) services that can be transparently traded in a DRIVE market without requiring developer input. The final contribution of this thesis is the definition and architecture of a Social Cloud – a dynamic Cloud computing infrastructure composed of virtualised resources contributed by members of a Social network. The Social Cloud prototype is based on DRIVE and highlights the ease in which dynamic DRIVE markets can be created and used in different domains

Incentives and Two-Sided Matching - Engineering Coordination Mechanisms for Social Clouds

The Social Cloud framework leverages existing relationships between members of a social network for the exchange of resources. This thesis focuses on the design of coordination mechanisms to address two challenges in this scenario. In the first part, user participation incentives are studied. In the second part, heuristics for two-sided matching-based resource allocation are designed and evaluated

Silicon's Second World: Scarcity, Political Indifference and Innovation in Czechoslovak Computing, 1964-1994

How societies invent, adopt, adapt, distribute and innovate with computers is an important puzzle for historians of technology, economists, educators and government planners alike. This dissertation examines the developmental path of Czechoslovakia from when its premier computer scientist, Antonín Svoboda, emigrated in 1964 to slightly beyond state dissolution in 1993. An industrialized consumer society with little to consume, as Jaroslav Švelch noted, Czechoslovakia illustrates both the still-understudied history of computing in state socialist societies and the global story of innovation and adaptation in liminal spaces that provide human capital and emerging markets for the West. An alternate modernity emerged in what Martin Müller calls the 'Global East,' constituted by users living in scarcity, skeptical of state and capital power and maintaining the countercultural community values articulated by exponents like Stewart Brand, Ted Nelson and Buckminster Fuller. This work contributes to the ongoing turn in the history of technology away from Silicon Valley-centered narratives of invention toward the maintenance, adaptation and second-order innovation better representative of technological encounters globally. Czech and Slovak computer users are the focus: Their social origins, personal politics, creativity and negotiated autonomy framed the shape of computing in their country. Their stories are told often by themselves-in extensive oral interviews with key scientists, prominent dissidents and black marketeers-and in the pages of their community's magazines, journals and newsletters, in television interviews, in their jokes and ribald songs. Their voices are part of a global chorus of hobbyism, tinkering, maintenance and technological communities informed by scholars like Jaroslav Švelch, Melanie Swalwell, Honghong Tinn, Helena Durnová, Patryk Wasiak, Ksenia Tatarchenko and Nathan Ensmenger