Alpha Entanglement Codes: Practical Erasure Codes to Archive Data in Unreliable Environments

Data centres that use consumer-grade disks drives and distributed peer-to-peer systems are unreliable environments to archive data without enough redundancy. Most redundancy schemes are not completely effective for providing high availability, durability and integrity in the long-term. We propose alpha entanglement codes, a mechanism that creates a virtual layer of highly interconnected storage devices to propagate redundant information across a large scale storage system. Our motivation is to design flexible and practical erasure codes with high fault-tolerance to improve data durability and availability even in catastrophic scenarios. By flexible and practical, we mean code settings that can be adapted to future requirements and practical implementations with reasonable trade-offs between security, resource usage and performance. The codes have three parameters. Alpha increases storage overhead linearly but increases the possible paths to recover data exponentially. Two other parameters increase fault-tolerance even further without the need of additional storage. As a result, an entangled storage system can provide high availability, durability and offer additional integrity: it is more difficult to modify data undetectably. We evaluate how several redundancy schemes perform in unreliable environments and show that alpha entanglement codes are flexible and practical codes. Remarkably, they excel at code locality, hence, they reduce repair costs and become less dependent on storage locations with poor availability. Our solution outperforms Reed-Solomon codes in many disaster recovery scenarios.Comment: The publication has 12 pages and 13 figures. This work was partially supported by Swiss National Science Foundation SNSF Doc.Mobility 162014, 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN

Scaling and Resilience in Numerical Algorithms for Exascale Computing

The first Petascale supercomputer, the IBM Roadrunner, went online in 2008. Ten years later, the community is now looking ahead to a new generation of Exascale machines. During the decade that has passed, several hundred Petascale capable machines have been installed worldwide, yet despite the abundance of machines, applications that scale to their full size remain rare. Large clusters now routinely have 50.000+ cores, some have several million. This extreme level of parallelism, that has allowed a theoretical compute capacity in excess of a million billion operations per second, turns out to be difficult to use in many applications of practical interest. Processors often end up spending more time waiting for synchronization, communication, and other coordinating operations to complete, rather than actually computing. Component reliability is another challenge facing HPC developers. If even a single processor fail, among many thousands, the user is forced to restart traditional applications, wasting valuable compute time. These issues collectively manifest themselves as low parallel efficiency, resulting in waste of energy and computational resources. Future performance improvements are expected to continue to come in large part due to increased parallelism. One may therefore speculate that the difficulties currently faced, when scaling applications to Petascale machines, will progressively worsen, making it difficult for scientists to harness the full potential of Exascale computing. The thesis comprises two parts. Each part consists of several chapters discussing modifications of numerical algorithms to make them better suited for future Exascale machines. In the first part, the use of Parareal for Parallel-in-Time integration techniques for scalable numerical solution of partial differential equations is considered. We propose a new adaptive scheduler that optimize the parallel efficiency by minimizing the time-subdomain length without making communication of time-subdomains too costly. In conjunction with an appropriate preconditioner, we demonstrate that it is possible to obtain time-parallel speedup on the nonlinear shallow water equation, beyond what is possible using conventional spatial domain-decomposition techniques alone. The part is concluded with the proposal of a new method for constructing Parallel-in-Time integration schemes better suited for convection dominated problems. In the second part, new ways of mitigating the impact of hardware failures are developed and presented. The topic is introduced with the creation of a new fault-tolerant variant of Parareal. In the chapter that follows, a C++ Library for multi-level checkpointing is presented. The library uses lightweight in-memory checkpoints, protected trough the use of erasure codes, to mitigate the impact of failures by decreasing the overhead of checkpointing and minimizing the compute work lost. Erasure codes have the unfortunate property that if more data blocks are lost than parity codes created, the data is effectively considered unrecoverable. The final chapter contains a preliminary study on partial information recovery for incomplete checksums. Under the assumption that some meta knowledge exists on the structure of the data encoded, we show that the data lost may be recovered, at least partially. This result is of interest not only in HPC but also in data centers where erasure codes are widely used to protect data efficiently

Computer science: the hardware software and heart of IT

1st edition, 201

Understanding Quantum Technologies 2022

Understanding Quantum Technologies 2022 is a creative-commons ebook that provides a unique 360 degrees overview of quantum technologies from science and technology to geopolitical and societal issues. It covers quantum physics history, quantum physics 101, gate-based quantum computing, quantum computing engineering (including quantum error corrections and quantum computing energetics), quantum computing hardware (all qubit types, including quantum annealing and quantum simulation paradigms, history, science, research, implementation and vendors), quantum enabling technologies (cryogenics, control electronics, photonics, components fabs, raw materials), quantum computing algorithms, software development tools and use cases, unconventional computing (potential alternatives to quantum and classical computing), quantum telecommunications and cryptography, quantum sensing, quantum technologies around the world, quantum technologies societal impact and even quantum fake sciences. The main audience are computer science engineers, developers and IT specialists as well as quantum scientists and students who want to acquire a global view of how quantum technologies work, and particularly quantum computing. This version is an extensive update to the 2021 edition published in October 2021.Comment: 1132 pages, 920 figures, Letter forma

The Physics of the B Factories

This work is on the Physics of the B Factories. Part A of this book contains a brief description of the SLAC and KEK B Factories as well as their detectors, BaBar and Belle, and data taking related issues. Part B discusses tools and methods used by the experiments in order to obtain results. The results themselves can be found in Part C

The physics of the B Factories

“The Physics of the B Factories” describes a decade long effort of physicists in the quest for the precise determination of asymmetry — broken symmetry — between particles and anti-particles. We now recognize that the matter we see around us is the residue — one part in a billion — of the matter and antimatter that existed in the early universe, most of which annihilated into the cosmic background radiation that bathes us. But the question remains: how did the baryonic matter-antimatter asymmetry arise? This book describes the work done by some 1000 physicists and engineers from around the globe on two experimental facilities built to test our understanding of this phenomenon, one at the SLAC National Accelerator Laboratory in California, USA, and a second at the KEK Laboratory, Tsukuba, Japan, and what we have learned from them in broadening our understanding of nature. Why is our universe dominated by the matter of which we are made rather than equal parts of matter and antimatter? This question has puzzled physicists for decades. However, this was not the question we addressed when we wrote the paper on CP violation in 1972. Our question was whether we can explain the CP violation observed in the K meson decay within the framework of the renormalizable gauge theory. At that time, Sakharov’s seminal paper was already published, but it did not attract our attention. If we were aware of the paper, we would have been misled into seeking a model satisfying Sakharov’s conditions and our paper might not have appeared. In our paper, we discussed that we need new particles in order to accommodate CP violation into the renormalizable electroweak theory, and proposed the six-quark scheme as one of the possible ways introducing new particles. We thought that the six-quark scheme is very interesting, but it was just a possibility. The situation changed when the tau-lepton was found and it was followed by the discovery of the Upsilon particle. The existence of the third generation became reality. However, it was still uncertain whether the mixing of the six quarks is a real origin of the observed CP violation. Theoretical calculation of CP asymmetries in the neutral K meson system contains uncertainty from strong interaction effects. What settled this problem were the B Factories built at SLAC and KEK. These B Factories are extraordinary in many ways. In order to fulfill the requirements of special experiments, the beam energies of the colliding electron and positron are asymmetric, and the luminosity is unprecedentedly high. It is also remarkable that severe competition between the two laboratories boosted their performance. One of us (M. Kobayashi) has been watching the development at KEK very closely as the director of the Institute of Particle and Nuclear Studies of KEK for a period of time. As witnesses, we appreciate the amazing achievement of those who participated in these projects at both laboratories. The B Factories have contributed a great deal to our understanding of particle physics, as documented in this book. In particular, thanks to the high luminosity far exceeding the design value, experimental groups measured mixing angles precisely and verified that the dominant source of CP violation observed in the laboratory experiments is flavor mixing among the three generations of quarks. Obviously we owe our Nobel Prize to this result. Now we are awaiting the operation of the next generation Super B Factories. In spite of its great success, the Standard Model is not an ultimate theory. For example, it is not thought to be possible for the matter dominance of the universe to be explained by the Standard Model. This means that there will still be unknown particles and unknown interactions. We have a lot of theoretical speculations but experimental means are rather limited. There are great expectations for the Super B Factories to reveal a clue to the world beyond the Standard Model