Checkpointing as a Service in Heterogeneous Cloud Environments

A non-invasive, cloud-agnostic approach is demonstrated for extending existing cloud platforms to include checkpoint-restart capability. Most cloud platforms currently rely on each application to provide its own fault tolerance. A uniform mechanism within the cloud itself serves two purposes: (a) direct support for long-running jobs, which would otherwise require a custom fault-tolerant mechanism for each application; and (b) the administrative capability to manage an over-subscribed cloud by temporarily swapping out jobs when higher priority jobs arrive. An advantage of this uniform approach is that it also supports parallel and distributed computations, over both TCP and InfiniBand, thus allowing traditional HPC applications to take advantage of an existing cloud infrastructure. Additionally, an integrated health-monitoring mechanism detects when long-running jobs either fail or incur exceptionally low performance, perhaps due to resource starvation, and proactively suspends the job. The cloud-agnostic feature is demonstrated by applying the implementation to two very different cloud platforms: Snooze and OpenStack. The use of a cloud-agnostic architecture also enables, for the first time, migration of applications from one cloud platform to another.Comment: 20 pages, 11 figures, appears in CCGrid, 201

CyberGuarder: a virtualization security assurance architecture for green cloud computing

Cloud Computing, Green Computing, Virtualization, Virtual Security Appliance, Security Isolation

CyberLiveApp: a secure sharing and migration approach for live virtual desktop applications in a cloud environment

In recent years we have witnessed the rapid advent of cloud computing, in which the remote software is delivered as a service and accessed by users using a thin client over the Internet. In particular, the traditional desktop application can execute in the remote virtual machines without re-architecture providing a personal desktop experience to users through remote display technologies. However, existing cloud desktop applications mainly achieve isolation environments using virtual machines (VMs), which cannot adequately support application-oriented collaborations between multiple users and VMs. In this paper, we propose a flexible collaboration approach, named CyberLiveApp, to enable live virtual desktop applications sharing based on a cloud and virtualization infrastructure. The CyberLiveApp supports secure application sharing and on-demand migration among multiple users or equipment. To support VM desktop sharing among multiple users, a secure access mechanism is developed to distinguish view privileges allowing window operation events to be tracked to compute hidden window areas in real time. A proxy-based window filtering mechanism is also proposed to deliver desktops to different users. To support application sharing and migration between VMs, we use the presentation streaming redirection mechanism and VM cloning service. These approaches have been preliminary evaluated on an extended MetaVNC. Results of evaluations have verified that these approaches are effective and useful

Transparent Orchestration of Task-based Parallel Applications in Containers Platforms

This paper presents a framework to easily build and execute parallel applications in container-based distributed computing platforms in a user-transparent way. The proposed framework is a combination of the COMP Superscalar (COMPSs) programming model and runtime, which provides a straightforward way to develop task-based parallel applications from sequential codes, and containers management platforms that ease the deployment of applications in computing environments (as Docker, Mesos or Singularity). This framework provides scientists and developers with an easy way to implement parallel distributed applications and deploy them in a one-click fashion. We have built a prototype which integrates COMPSs with different containers engines in different scenarios: i) a Docker cluster, ii) a Mesos cluster, and iii) Singularity in an HPC cluster. We have evaluated the overhead in the building phase, deployment and execution of two benchmark applications compared to a Cloud testbed based on KVM and OpenStack and to the usage of bare metal nodes. We have observed an important gain in comparison to cloud environments during the building and deployment phases. This enables better adaptation of resources with respect to the computational load. In contrast, we detected an extra overhead during the execution, which is mainly due to the multi-host Docker networking.This work is partly supported by the Spanish Government through Programa Severo Ochoa (SEV-2015-0493), by the Spanish Ministry of Science and Technology through TIN2015-65316 project, by the Generalitat de Catalunya under contracts 2014-SGR-1051 and 2014-SGR-1272, and by the European Union through the Horizon 2020 research and innovation program under grant 690116 (EUBra-BIGSEA Project). Results presented in this paper were obtained using the Chameleon testbed supported by the National Science Foundation.Peer ReviewedPostprint (author's final draft

Measuring and Managing Answer Quality for Online Data-Intensive Services

Online data-intensive services parallelize query execution across distributed software components. Interactive response time is a priority, so online query executions return answers without waiting for slow running components to finish. However, data from these slow components could lead to better answers. We propose Ubora, an approach to measure the effect of slow running components on the quality of answers. Ubora randomly samples online queries and executes them twice. The first execution elides data from slow components and provides fast online answers; the second execution waits for all components to complete. Ubora uses memoization to speed up mature executions by replaying network messages exchanged between components. Our systems-level implementation works for a wide range of platforms, including Hadoop/Yarn, Apache Lucene, the EasyRec Recommendation Engine, and the OpenEphyra question answering system. Ubora computes answer quality much faster than competing approaches that do not use memoization. With Ubora, we show that answer quality can and should be used to guide online admission control. Our adaptive controller processed 37% more queries than a competing controller guided by the rate of timeouts.Comment: Technical Repor

Urban management revolution: intelligent management systems for ubiquitous cities

A successful urban management support system requires an integrated approach. This integration includes bringing together economic, socio-cultural and urban development with a well orchestrated transparent and open decision making mechanism. The paper emphasises the importance of integrated urban management to better tackle the climate change, and to achieve sustainable urban development and sound urban growth management. This paper introduces recent approaches on urban management systems, such as intelligent urban management systems, that are suitable for ubiquitous cities. The paper discusses the essential role of online collaborative decision making in urban and infrastructure planning, development and management, and advocates transparent, fully democratic and participatory mechanisms for an effective urban management system that is particularly suitable for ubiquitous cities. This paper also sheds light on some of the unclear processes of urban management of ubiquitous cities and online collaborative decision making, and reveals the key benefits of integrated and participatory mechanisms in successfully constructing sustainable ubiquitous cities