Detecting corrupted pages in M replicated large files

A file in a distributed database system is replicated on M sites and may contain corrupted pages. Abdel-Ghaffar and El Abbadi gave a detection scheme assuming that the number of corrupted pages f < M l 2. We replace this assumption by a much weaker one, that, for each page, the majority of copies are correct. Our schemes are based on the structure of the Reed-Solomon code, as proposed by Abdel-Ghaffar and El Abbadi for M= 2. © 1997 IEEE.published_or_final_versio

Snarl : entangled merkle trees for improved file availability and storage utilization

In cryptographic decentralized storage systems, files are split into chunks and distributed across a network of peers. These storage systems encode files using Merkle trees, a hierarchical data structure that provides integrity verification and lookup services. A Merkle tree maps the chunks of a file to a single root whose hash value is the file's content-address. A major concern is that even minor network churn can result in chunks becoming irretrievable due to the hierarchical dependencies in the Merkle tree. For example, chunks may be available but can not be found if all peers storing the root fail. Thus, to reduce the impact of churn, a decentralized replication process typically stores each chunk at multiple peers. However, we observe that this process reduces the network's storage utilization and is vulnerable to cascading failures as some chunks are replicated 10X less than others. We propose Snarl, a novel storage component that uses a variation of alpha entanglement codes to add user-controlled redundancy to address these problems. Our contributions are summarized as follows: 1) the design of an entangled Merkle tree, a resilient data structure that reduces the impact of hierarchical dependencies, and 2) the Snarl prototype to improve file availability and storage utilization in a real-world storage network. We evaluate Snarl using various failure scenarios on a large cluster running the Ethereum Swarm network. Our evaluation shows that Snarl increases storage utilization by 5X in Swarm with improved file availability. File recovery is bandwidth-efficient and uses less than 2X chunks on average in scenarios with up to 50% of total chunk loss.publishedVersio

Improving capacity-performance tradeoffs in the storage tier

Data-set sizes are growing. New techniques are emerging to organize and analyze these data-sets. There is a key access pattern emerging with these new techniques, large sequential file accesses. The trend toward bigger files exists to help amortize the cost of data accesses from the storage layer, as many workloads are recognized to be I/O bound. The storage layer is widely recognized as the slowest layer in the system. This work focuses on the tradeoff one can make with that storage capacity to improve system performance. ^ Capacity can be leveraged for improved availability or improved performance. This tradeoff is key in the storage layer, as this allows for data loss prevention and bandwidth aggregation. Typically these tradeoffs do not allow much choice with regard to capacity use. This work will leverage replication as the enabling mechanism to improve the capacity-performance tradeoff in the storage tier, while still providing for availability. ^ This capacity-performance tradeoff can be made at both the local and distributed file system level. I propose two techniques that allow for an improved tradeoff of capacity. The local file system can be employed on scale-out or scale-up infrastructures to improve performance. The distributed file system is targeted at distributed frameworks, such as MapReduce, to improve the cluster performance. The local file system design is MorphStore, and the distributed file system is BoostDFS. ^ MorphStore is a file system that significantly improves performance when accessing large files by using two innovations. MorphStore combines (a) load-adaptive I/O access scheduling to dynamically optimize throughput (aggregation), and (b) utility-xiii driven replication to best use capacity for performance. Additionally, adaptive-access scheduling can be utilized to optimize scheduling of requests (for throughput) on systems with a large number of storage devices. Replication is utilized to make available high utility files and then optimize throughput of these high utility files based on system load. ^ BoostDFS is a distributed file system that allows a better capacity-performance tradeoff via inter-node file replication. BoostDFS is built on the observation that distributed file systems currently inter-node replication for availability, but provide no mechanism to further improve performance. Replication for availability provides diminishing returns on performance, this is due to saturation of locality. BoostDFS exploits the common by improving I/O performance of these local tasks. This is done via intra-node replication by leveraging MorphStore as the local file system. This technique allows for capacity to be traded for availability as well as performance, with a small capacity overhead under constant availability. ^ Both MorphStore and BoostDFS utilize replication. Replication allows for both bandwidth aggregation and availability, This work primarily focuses on the performance utility of replication, but does not sacrifice availability in the process. These techniques provide an improved capacity-performance tradeoff while allowing the desired level of availability

Ironman: Open Source Containers and Virtualization in bare metal

Trabalho de projeto de mestrado, Engenharia Informática (Engenharia de Software) Universidade de Lisboa, Faculdade de Ciências, 2021Computer virtualization has become prevalent throughout the years for both business and personal use. It allows for hosting new machines, on computational resources that are left unused, running as independent computers. Apart from the traditional virtual machines, a more recent form of virtualization was introduced and will be explored in this project, containers, more specifically Linux Containers. While multiple virtualization tools are available, some of them require a premium payment, while others do not support container virtualization. For this project, LXD, an open source virtual instance manager, will be used to manage both virtual machines and containers. For added service availability, clustering support will also be developed. Clustering will enable multiple physical computers to host virtual instances as if they were a single machine. Coupled with the Ceph storage back end it allows for data to be replicated across all computers in the same cluster, enabling instance recovery when a computer from the cluster is faulty. The infrastructure deployment tool Puppet will be used to automate the installation and configuration of an LXD virtualization system for both a clustered and non clustered environment. This allows for simple and automatic physical host configuration limiting the required user input and thus decreasing the possibilities of system misconfiguration. LXD was tested for both environments and ultimately considered an effective virtualization tool, which when configured accordingly can be productized for a production environment

Improving access to ultrasound imaging in northern, remote communities

Access to healthcare services—including access to medical imaging—is an important determinant of health outcomes. This thesis aims to improve understanding of and address gaps in access to ultrasound imaging for patients in northern, remote communities, and advance a novel ultrasound technology with the ultimate goal of improving patient care and health outcomes. This thesis first brings greater understanding of patients’ perceptions of access and factors which shape access to ultrasound imaging in northern, remote communities in Saskatchewan, Canada. A qualitative study was performed using interpretive description as a methodological approach and a multi-dimensional conceptualization of access to care as a theoretical framework. The study identified barriers which patients in northern, remote communities face in accessing ultrasound imaging, and demonstrated that geographic remoteness from imaging facilities was a central barrier. To determine whether disparities in access to ultrasound imaging resulted in disparities in utilization of ultrasound services, two population-based studies assessed the association between sociodemographic and geographic factors and obstetrical and non-obstetrical ultrasound utilization in Saskatchewan. In the first study investigating obstetrical ultrasound utilization, multivariate logistic regression analysis demonstrated that women living in rural areas, remote areas, and low income neighbourhoods, as well as status First Nations women, were less likely to have a second trimester ultrasound, an important aspect of prenatal care. In a second study investigating non-obstetrical ultrasound utilization across the entire provincial population, multivariate Poisson regression analysis similarly demonstrated lower rates of non-obstetrical ultrasound utilization among individuals living in rural and remote areas, individuals residing in low income neighbourhoods, and status First Nations persons. To address the barriers which patients in northern, remote communities face in accessing ultrasound imaging and to minimize disparities in ultrasound imaging utilization as identified in previous studies in this thesis, telerobotic ultrasound technology was investigated as a solution to improve access to ultrasound imaging. Using this technology, radiologists and sonographers could remotely manipulate an ultrasound probe via a robotic arm, thereby remotely performing an ultrasound exam while patients remained in their home community. A clinical trial comparing conventional and telerobotic ultrasound approaches was undertaken, validating this technology for obstetrical ultrasound imaging. To determine the feasibility of using telerobotic technology to establish an ultrasound service delivery model to remotely provide diagnostic ultrasound exams in underserved communities, pilot telerobotic ultrasound clinics were developed in three northern, remote communities. Telerobotic ultrasound exams were sufficient for diagnosis in the majority of cases, minimizing travel or reducing wait times for these patients. This technology was subsequently evaluated during a COVID-19 outbreak in northern Saskatchewan, demonstrating the potential of this technology to provide critical ultrasound services to an underserved northern population and minimize health inequities during the COVID-19 pandemic. An economic evaluation was performed to compare a service delivery model using telerobotic ultrasound technology to alternative service delivery models. Telerobotic ultrasound combined with an itinerant sonographer service was found to be the lowest cost option from both a publicly funded healthcare payer perspective and a societal perspective for many northern, remote communities. This thesis provides key insights for health system leaders seeking improved understanding and novel solutions to improve access to ultrasound imaging in northern, remote communities. Findings suggest that telerobotic ultrasound is a viable solution to improve access to ultrasound imaging and reduce costs associated with ultrasound service delivery. Evidence in this thesis may be used to help improve ultrasound services and health equity for patients in underserved northern, remote communities. Continued respectful collaboration with northern, remote, Indigenous peoples and communities will be a critical aspect to ensure that ultrasound services meet community needs

A Telehealth Explosion: Using Lessons from the Pandemic to Shape the Future of Telehealth Regulation

From board rooms, to classrooms, to Saturday Night Live skits, the video conferencing app Zoom became a seemingly overnight sensation as a way to connect while businesses were shuttered and individuals were forced to stay at home when the coronavirus pandemic erupted in the United States in March 2020. From 10 million daily users in December 2019 to over 200 million daily users by March 2020, the company founded in 2011 became a market leader as the country tried to figure out how to continue business as usual—to the extent possible—during the global pandemic. While hospitals prepared for the onslaught of patients suffering from COVID–19, many physicians and physician offices around the country not tasked with treating patients suffering from COVID–19 shuttered their doors along with other businesses and contemplated ways in which they could still render necessary care to their patients.4 How could physicians advise or diagnose patients who exhibited coronavirus symptoms without exposing other patients to coronavirus? How could physicians who were themselves immunocompromised or at special risk if they were to contract COVID–19 continue to treat their patients? How could patients feel comfortable seeking care for non-COVID–19 related conditions without feeling like seeking such care could expose them to the virus? In short, providers were facing many of the same dilemmas that other businesses have been facing during the global pandemic, and just like the 200 million fellow Americans who turned to Zoom, the health care industry likewise looked to technology

Gollach : configuration of a cluster based linux virtual server

Includes bibliographical references.This thesis describes the Gollach cluster. The Gollach is an eight machine computing cluster that is aimed at being a general purpose computing resource for research purposes. This includes image processing and simulations. The main quest in this project is to create a cluster server that gives increased computational power and a unified system image (at several levels) without requiring the users to learn specialised tricks. At the same time the cluster must not be tasking to administer