Context-aware task scheduling in distributed computing systems

These days, the popularity of technologies such as machine learning, augmented reality, and big data analytics is growing dramatically. This leads to a higher demand of computational power not only for IT professionals but also for ordinary device users who benefit from new applications. At the same time, the computational performance of end-user devices increases to meet the demands of these resource-hungry applications. As a result, there is a coexistence of a huge demand of computational power on the one side and a large pool of computational resources on the other side. Bringing these two sides together is the idea of computational resource sharing systems which allow applications to forward computationally intensive workload to remote resources. This technique is often used in cloud computing where customers can rent computational power. However, we argue that not only cloud resources can be used as offloading targets. Rather, idle CPU cycles from end-user administered devices at the edge of the network can be spontaneously leveraged as well. Edge devices, however, are not only heterogeneous in their hardware and software capabilities, they also do not provide any guarantees in terms of reliability or performance. Does it mean that either the applications that require further guarantees or the unpredictable resources need to be excluded from such a sharing system? In this thesis, we propose a solution to this problem by introducing the Tasklet system, our approach for a computational resource sharing system. The Tasklet system supports computation offloading to arbitrary types of devices, including stable cloud instances as well as unpredictable end-user owned edge resources. Therefore, the Tasklet system is structured into multiple layers. The lowest layer is a best-effort resource sharing system which provides lightweight task scheduling and execution. Here, best-effort means that in case of a failure, the task execution is dropped and that tasks are allocated to resources randomly. To provide execution guarantees such as a reliable or timely execution, we add a Quality of Computation (QoC) layer on top of the best-effort execution layer. The QoC layer enforces the guarantees for applications by using a context-aware task scheduler which monitors the available resources in the computing environment and performs the matchmaking between resources and tasks based on the current state of the system. As edge resources are controlled by individuals, we consider the fact that these users need to be able to decide with whom they want to share their resources and for which price. Thus, we add a social layer on top of the system that allows users to establish friendship connections which can then be leveraged for social-aware task allocation and accounting of shared computation

A cross-stack, network-centric architectural design for next-generation datacenters

This thesis proposes a full-stack, cross-layer datacenter architecture based on in-network computing and near-memory processing paradigms. The proposed datacenter architecture is built atop two principles: (1) utilizing commodity, off-the-shelf hardware (i.e., processor, DRAM, and network devices) with minimal changes to their architecture, and (2) providing a standard interface to the programmers for using the novel hardware. More specifically, the proposed datacenter architecture enables a smart network adapter to collectively compress/decompress data exchange between distributed DNN training nodes and assist the operating system in performing aggressive processor power management. It also deploys specialized memory modules in the servers, capable of performing general-purpose computation and network connectivity. This thesis unlocks the potentials of hardware and operating system co-design in architecting application-transparent, near-data processing hardware for improving datacenter's performance, energy efficiency, and scalability. We evaluate the proposed datacenter architecture using a combination of full-system simulation, FPGA prototyping, and real-system experiments

SDN-enabled Workload Offloading Schemes for IoT Video Analytics Applications

Increasing demand for using IoT applications, such as video analytics, leverages the importance of developing an architecture to meet the requirements in terms of the latency, reliability, and energy consumption. IoT video cameras combined with the power of machine learning algorithms introduce real-time video analytics applications that can be used in diverse domains, such as security surveillance, sports, and retail stores. However, processing captured video frames using machine learning algorithms needs resources that are beyond the capability of these IoT devices. IoT task offloading is a new paradigm to aim IoT applications to deliver processing intensive applications to their users. IoT devices, which have limited resources by nature, offload their tasks to more powerful servers, i.e., edge/cloud servers . Nonetheless, selecting an appropriate destination for offloading the tasks is the first incoming problem for the IoT task offloading. There are some criteria which needs to be considered when it comes to IoT task offloading, for example transmission latency, queuing delay, as well as processing latency. Although edge servers have limited resources compared to cloud servers, the end-to-end latency for sending the packets to the edge servers is less than the cloud servers. On the other hand, because of the limited available resources in the edge servers, distributing the offloaded tasks between these devices is necessary to avoid overloaded servers. Considering the above mentioned facts, in this thesis, we present load-balancing algorithms benefits from Software Defined Networking (SDN) to distribute offloaded tasks to reduce the chance of using overloaded servers and processing latency of offloaded packets of IoT video analytics applications. Taking into account the aforementioned facts, we propose a scoring metric to balance the incoming offloaded packets between edge servers. The introduced algorithm takes advantage of underlying SDN to collect information about the load of each edge server in the network. Then, the SDN controller uses the scoring metric and sorts the edge servers accordingly. The offloaded task will be directed to the edge server with the lowest processing load to avoid overloaded edge servers. Since the number of IoT devices in the network is not predictable, increasing number of IoT devices will lead to overloaded edge servers. Hence, offloading a part of the IoT tasks to the cloud server might be a better option, even though the packets should pass through the core network. In this regard, we developed a hierarchical edge/cloud system for IoT task offloading. We modeled each of edge/cloud servers by M/M/1 queue model. By benefiting from SDN as an underlying network, the SDN calculates the processing latency and transmission latency to edge and cloud servers, and decides the best destination in terms of the minimum latency that directs the offloaded tasks to one of the desired servers. We have conducted extensive performance evaluation to demonstrate the out-performance of the developed solutions compared with other related approaches in terms of total experienced latency and load distribution between the available servers. The results are comprehensively discussed in their related chapters to clarify the performance of the developed solution