Exploiting Rateless Codes in Cloud Storage Systems

devices (virtual disks) that can be directly accessed and used as if they were raw physical disks. In this paper we devise ENIGMA, an architecture for the back-end of BLCS systems able to provide adequate levels of access and transfer performance, availability, integrity, and confidentiality, for the data it stores. ENIGMA exploits LT rateless codes to store fragments of sectors on storage nodes organized in clusters. We quantitatively evaluate how the various ENIGMA system parameters affect the performance, availability, integrity, and confidentiality of virtual disks. These evaluations are carried out by using both analytical modeling (for availability, integrity, and confidentiality) and discrete event simulation (for performance), and by considering a set of realistic operational scenarios. Our results indicate that it is possible to simultaneously achieve all the objectives set forth for BLCS systems by using ENIGMA, and that a careful choice of the various system parameters is crucial to achieve a good compromise among them. Moreover, they also show that LT coding-based BLCS systems outperform traditional BLCS systems in all the aspects mentioned before

運用Open Source提供圖形化虛擬網路實驗室之管理

[[abstract]]本研究嘗試運用開放源碼之圖形化軟體針對淡江大學既有虛擬網路實驗室提供更友善的操作介面，讓授課老師設計實驗單元、實驗室管理人員與學生實作皆能有更友善的使用環境。系統實作採用ProxmoxVE結合KVM、OpenVZ與Vyatta虛擬防火牆，提供圖形操作介面取代Command Line操作，從而降低使用上的進入障礙。使得更多人願意使用虛擬實驗室進行教學與實習，普及實驗室使用率。本文亦展示了使用圖形化操作介面來部署網路實習單元的過程。[[conferencetype]]國際[[conferencetkucampus]]淡水校園[[iscallforpapers]]Y[[conferencelocation]]新北市淡水區, 台

Autoscaling Hadoop Clusters

Pilve arvutused on viimaste aastate jooksul palju kõneainet pakkunud. Alates sellest, et tegemist ei ole millegi muuga kui virtualiseerimine ilusa nimega, kuni selleni, et tulevik on pilve arvutuste p aralt. Juba 4 aastat on virtuaalsed serverid, andmehoidlad, andmebaasid ja muud infrastruktuuri elemendid olnud k attesaadavad veebiteenustena. Antud töös me ehitame ise sklaleeruva MapReduce platvormi, mis baseerub vabalähtekoodiga tarkvara Apache Hadoop projektil. Antud platvorm skaleerib end ise, vastavalt serverite koormatusele k aivitab uusi servereid, et kiirendada arvutusprotsessi.Cloud computing, specifically Infrastructure as a Service model provides us with the facilities to provision new servers at will and increase the computing power of a cluster almost in real time. This provisioning and deprovisioning of servers can happen automatically based on some performance metrics of the cluster. We introduce a framework of autoscaling clusters in the private and public cloud ecosystem using the Eucalyptus and AWS software stack and use MapReduce as the service provided by the cluster

Scalable Architecture for Integrated Batch and Streaming Analysis of Big Data

Thesis (Ph.D.) - Indiana University, Computer Sciences, 2015As Big Data processing problems evolve, many modern applications demonstrate special characteristics. Data exists in the form of both large historical datasets and high-speed real-time streams, and many analysis pipelines require integrated parallel batch processing and stream processing. Despite the large size of the whole dataset, most analyses focus on specific subsets according to certain criteria. Correspondingly, integrated support for efficient queries and post- query analysis is required. To address the system-level requirements brought by such characteristics, this dissertation proposes a scalable architecture for integrated queries, batch analysis, and streaming analysis of Big Data in the cloud. We verify its effectiveness using a representative application domain - social media data analysis - and tackle related research challenges emerging from each module of the architecture by integrating and extending multiple state-of-the-art Big Data storage and processing systems. In the storage layer, we reveal that existing text indexing techniques do not work well for the unique queries of social data, which put constraints on both textual content and social context. To address this issue, we propose a flexible indexing framework over NoSQL databases to support fully customizable index structures, which can embed necessary social context information for efficient queries. The batch analysis module demonstrates that analysis workflows consist of multiple algorithms with different computation and communication patterns, which are suitable for different processing frameworks. To achieve efficient workflows, we build an integrated analysis stack based on YARN, and make novel use of customized indices in developing sophisticated analysis algorithms. In the streaming analysis module, the high-dimensional data representation of social media streams poses special challenges to the problem of parallel stream clustering. Due to the sparsity of the high-dimensional data, traditional synchronization method becomes expensive and severely impacts the scalability of the algorithm. Therefore, we design a novel strategy that broadcasts the incremental changes rather than the whole centroids of the clusters to achieve scalable parallel stream clustering algorithms. Performance tests using real applications show that our solutions for parallel data loading/indexing, queries, analysis tasks, and stream clustering all significantly outperform implementations using current state-of-the-art technologies

Supporting Cloud Computing with the Virtual Block Store System

The fast development of cloud computing systems stimulates the needs for a standalone block storage system to provide persistent block storage services to virtual machines maintained by clouds. This paper presents the Virtual Block Store (VBS) System, a standalone block storage system built on the basis of LVM, iSCSI, and Xen hypervisor, which can provide basic block storage services such as volume creation and attachment. The concept and functional interface of VBS are based on Amazon Elastic Block Store (EBS) service; moreover, VBS works independently with an existing LVM volume server and Xen nodes, and thus can be easily extended to support other types of volume servers and virtual machine managers, or integrated with various cloud computing systems. Preliminary I/O benchmark results are presented and analyzed, indicating that a VBS volume can provide throughput that is similar to an ATA over Ethernet virtual device. 1