VM Migration for Secure Out-of-band Remote Management with Nested Virtualization

Infrastructure-as-a-Service clouds provide out-of-band remote management of the systems in virtual machines (VMs). This management method enables users to manage their systems even on several types of failures inside VMs. In this method, users access virtual devices of their VMs, but virtual devices are not sufficiently protected against untrusted cloud operators. For secure out-of-band remote management, previous work securely runs shadow devices outside an untrusted virtualized system using nested virtualization. However, the states of shadow devices are lost during VM migration. In this paper, we propose USShadow for continuing secure out-of-band remote management after VM migration. USShadow enables the migration manager inside the virtualized system to transparently and securely save and restore the states of shadow devices outside it. We have implemented USShadow, which supports Xen and KVM as virtualized systems. Then, we confirmed that USShadow could continue virtual serial console and that the migration overhead was negligible.2020 IEEE 13th International Conference on Cloud Computing (CLOUD 2020), 18–24 October, 2020, Beijing, China（オンライン開催に変更

Low-Cost and Fast Failure Recovery Using In-VM Containers in Clouds

Recently, various services are provided using virtual machines (VMs) in clouds. Therefore, it is necessary to prepare for system failures of VMs, hosts running VMs, and even data centers, e.g., using active/standby clustering. However, a trade-off exists between the maintenance cost for additional VMs and the recovery time in traditional techniques. For example, hot standby can rapidly fail over to the secondary system on a system failure, but the secondary system has to always run the same number of VMs as the primary system. In contrast, cold standby does not need to run VMs until a system failure, but it has to boot VMs on failure recovery. In this paper, we propose VCRecovery, which is the system for achieving both low-cost and fast failure recovery. VCRecovery consolidates services using containers inside VMs (in-VM containers) in the secondary system. For hot standby, it can reduce the maintenance cost by using only a smaller number of VMs in the secondary system. For cold standby, it can reduce the recovery time by quickly booting in-VM containers. If a VM is overloaded after the recovery, VCRecovery can migrate several in-VM containers to other VMs. To synchronize storage between VMs in the primary system and in-VM containers in the secondary system, it efficiently performs minimum file-based synchronization based on software packages. We have implemented VCRecovery using LXD and Zabbix and examined the performance.IEEE 17th International Conference on Dependable, Autonomic and Secure Computing / IEEE 17th International Conference on Pervasive Intelligence and Computing / IEEE 5th International Conference on Cloud and Big Data Computing / IEEE 4th Cyber Science and Technology Congress(DASC-PICom-CBDCom-CyberSciTech 2019), August 5-8 2019, Fukuoka, Japa

MigSGX: A Migration Mechanism for Containers Including SGX Applications

Recently, containers are widely used to process big data in clouds. To prevent information leakage from containers, applications in containers can protect sensitive information using enclaves provided by Intel SGX. The memory of enclaves is encrypted by a CPU using its internal keys. However, the execution of SGX applications cannot be continued after the container running those applications is migrated. This is because enclave memory cannot be correctly decrypted at the destination host. This paper proposes MigSGX for enabling the continuous execution of SGX applications after container migration. Since the states of enclaves cannot be directly accessed from the outside, MigSGX securely invokes each enclave and makes it dump and load its state. Atthe dump time, each enclave re-encrypts its state using a CPU-independent key to protect sensitive information. For space- and time-efficiency, MigSGX saves and restores a large amount of enclave memory in a pipelined manner. We have implemented MigSGX in the Intel SGX SDK and CRIU and showed that pipelining could improve migration performance by up to 52%. The memory necessary for migration was reduced only to 0.15%.UCC \u2721: 2021 IEEE/ACM 14th International Conference on Utility and Cloud Computing, December 6 - 9, 2021, Leicester, United Kingdo

Efficient Migration of Large-memory VMs Using Private Virtual Memory

Recently, Infrastructure-as-a-Service clouds provide virtual machines (VMs) with a large amount of memory. Such large-memory VMs can be migrated to other hosts on host maintenance, but it is costly to always preserve hosts with sufficient free memory as the destination of VM migration. Using virtual memory in destination hosts is a possible solution, but the performance of VM migration largely degrades because traditional general-purpose virtual memory causes frequent paging during the migration. This paper proposes VMemDirect, which achieves efficient migration of large-memory VMs using private virtual memory. VMemDirect creates private swap space for each VM on fast NVMe SSDs. Then it transfers likely accessed memory data to physical memory and the other data to the private swap space directly. This direct memory transfer can completely avoid paging during VM migration. We have implemented VMemDirect in KVM and showed that the performance of VM migration and the migrated VM was improved dramatically.11th International Conference on Intelligent Networking and Collaborative Systems(INCoS 2019), September 5-7, 2019, Oita, Japa

Flexible and Efficient Partial Migration of Split-memory VMs

Recently, virtual machines (VMs) with a large amount of memory are being widely used. For flexible migration of such large-memory VMs without large hosts, split migration has been proposed. It transfers VM fragments to multiple smaller hosts and runs a split-memory VM across those hosts with remote paging. However, the traditional method cannot migrate a split-memory VM efficiently because it always migrates the entire VM. In addition, it has to gather all the VM fragments to one host and transfer them from that host. To address these issues, this paper proposes flexible and efficient partial migration of split-memory VMs. In particular, subst migration migrates only part of a split-memory VM to enable the maintenance of some of the hosts running the VM. Merge migration efficiently consolidates VM fragments distributed across multiple hosts into one host by directly transferring a VM fragment from each host. Even if a split-memory VM itself causes remote paging during such partial migration, the consistency of the VM is maintained by retransferring and invalidating target memory. We have implemented partial migration in KVM and showed its efficiency.2020 IEEE 13th International Conference on Cloud Computing (CLOUD 2020), 18–24 October, 2020, Beijing, China（オンライン開催に変更

Secure VM management with strong user binding in semi-trusted clouds

In Infrastructure-as-a-Service (IaaS) clouds, remote users access provided virtual machines (VMs) via the management server. The management server is managed by cloud operators, but not all the cloud operators are trusted in semi-trusted clouds. They can execute arbitrary management commands to users’ VMs and redirect users’ commands to malicious VMs. We call the latter attack the VM redirection attack. The root cause is that the binding of remote users to their VMs is weak. In other words, it is difficult to enforce the execution of only users’ management commands to their VMs. In this paper, we propose UVBond for strongly binding users to their VMs to address this issue. UVBond boots user’s VM by decrypting its encrypted disk inside the trusted hypervisor. Then it issues a VM descriptor to securely identify that VM. To bridge the semantic gap between high-level management commands and low-level hypercalls, UVBond uses hypercall automata, which accept the sequences of hypercalls issued by commands. We have implemented UVBond in Xen and created hypercall automata for various management commands. Using UVBond, we confirmed that a VM descriptor and hypercall automata prevented insider attacks and that the overhead was not large in remote VM management

Fine-grained Autoscaling with In-VM Containers and VM Introspection

Clouds often provides a mechanism called autoscaling to deal with load increases of services running in virtual machines (VMs). When a VM is overloaded, scale-out is performed and automatically increases the number of VMs. However, when multiple services run in one VM, the entire VM is always scaled out even if only one service is over-utilized. In this case, only an over-utilized service should be scaled out, but it is not easy for clouds to accurately monitor the resource usage of services inside VMs. This paper proposes Ciel, which runs each service in a container created inside a VM for separation of services and enables fine-grained autoscaling of VMs. Using VM introspection, Ciel accurately monitors the resource usage of each in-VM container from the outside of a VM in a non-intrusive manner. If it detects an overloaded in-VM container, it creates a new VM of minimum size and boots only the container that needs to be scaled out in the VM. This can minimize both the cost of the VM and the time taken for scale-out. We have implemented Ciel using Xen and Docker and showed the effectiveness.2020 IEEE/ACM 13th International Conference on Utility and Cloud Computing (UCC 2020), 7–10 December, 2020, Leicester, UK（オンライン開催に変更

Consistent Offline Update of Suspended Virtual Machines in Clouds

In Infrastructure-as-a-Service clouds, there exist many virtual machines (VMs) that are not used for a long time. For such VMs, many vulnerabilities are often found in installed software while VMs are suspended. If security updates are applied to such VMs after the VMs are resumed, the VMs easily suffer from attacks via the Internet. To solve this problem, offline update of VMs has been proposed, but some approaches have to permit cloud administrators to resume users\u27 VMs. The others are applicable only to completely stopped VMs and often corrupt virtual disks if they are applied to suspended VMs. In addition, it is sometimes difficult to accurately emulate security updates offline. In this paper, we propose OUassister, which enables consistent offline update of suspended VMs. OUassister emulates security updates of VMs offline in a non-intrusive manner and applies the emulation results to the VMs online. This separation prevents virtual disks of even suspended VMs from being corrupted. For more accurate emulation of security updates, OUassister provides an emulation environment using a technique called VM introspection. Using this environment, it automatically extracts updated files and executed scripts. We have implemented OUassister in Xen and confirmed that the time for critical online update was largely reduced.IEEE 17th International Conference on Dependable, Autonomic and Secure Computing / IEEE 17th International Conference on Pervasive Intelligence and Computing / IEEE 5th International Conference on Cloud and Big Data Computing / IEEE 4th Cyber Science and Technology Congress(DASC-PICom-CBDCom-CyberSciTech 2019), August 5-8 2019, Fukuoka, Japa

Secure Offloading of Intrusion Detection Systems from VMs with Intel SGX

Virtual machines (VMs) inside clouds need to be monitored using intrusion detection systems (IDS). Since host-based IDS can be easily disabled by intruders, IDS offloading with VM introspection (VMI) is used to securely run IDS outside a target VM. However, offloaded IDS can be still attacked because it runs on top of a vulnerable operating system (OS). Various systems have been proposed to protect offloaded IDS, but no systems provide an appropriate execution environment to IDS. This paper proposes SGmonitor for enabling the secure execution of IDS offloaded from VMs inside clouds using Intel SGX. SGmonitor executes IDS in SGX enclaves and preserves confidentiality and integrity. It provides secure VMI for memory and storage by using encryption and integrity checking. To make the development of offloaded IDS easier, it provides the in-kernel API to in-enclave IDS and enables transparent access to OS data in VMs. We have implemented SGmonitor in Xen with SGX support and showed that the overhead of in-enclave IDS was 31% in compensation for much stronger security.2021 IEEE 14th International Conference on Cloud Computing (CLOUD), September 5-10, 2021, Chicago, IL, US

Prevention of a DoS Attack with Copy-on-write in the Overlay Filesystem

Recently, containers are widely used for lightweight virtualization. A container usually uses a disk image that stacks a thin writable layer on top of a read-only image layer. For this layering, a filesystem called OverlayFS is often used. To modify a file in the read-only lower layer, OverlayFS first copies the entire file to the upper layer and then writes requested data to it. This copy-on-write suspends a container for a long time and consumes the disk space of the upper layer when the size of the target file is large. If large files are intentionally modified by attackers, a potential denial-of-service (DoS) attack can be mounted. This paper proposes a new filesystem, called TranslayFS, based on OverlayFS to prevent this type of DoS attack. TranslayFS creates only a special file called a sparse file in the upper layer when a container modifies a file in the lower layer for the first time. Using this file, it holds only modified file data in the upper layer without copying the entire file. It returns the modified part of the file from the upper layer and the unmodified part from the lower layer. We have implemented TranslayFS in the Linux kernel and confirmed that TranslayFS could dramatically reduce the latency in the first write to a file, so that the DoS attack was not possible.19th IEEE International Conference on Dependable, Autonomic & Secure Computing (DASC 2021), October 25-28, 2021, Virtual Conferenc