Cold Storage Data Archives: More Than Just a Bunch of Tapes

The abundance of available sensor and derived data from large scientific experiments, such as earth observation programs, radio astronomy sky surveys, and high-energy physics already exceeds the storage hardware globally fabricated per year. To that end, cold storage data archives are the---often overlooked---spearheads of modern big data analytics in scientific, data-intensive application domains. While high-performance data analytics has received much attention from the research community, the growing number of problems in designing and deploying cold storage archives has only received very little attention. In this paper, we take the first step towards bridging this gap in knowledge by presenting an analysis of four real-world cold storage archives from three different application domains. In doing so, we highlight (i) workload characteristics that differentiate these archives from traditional, performance-sensitive data analytics, (ii) design trade-offs involved in building cold storage systems for these archives, and (iii) deployment trade-offs with respect to migration to the public cloud. Based on our analysis, we discuss several other important research challenges that need to be addressed by the data management community

An optical solution for the set splitting problem

We describe here an optical device, based on time-delays, for solving the set splitting problem which is well-known NP-complete problem. The device has a graph-like structure and the light is traversing it from a start node to a destination node. All possible (potential) paths in the graph are generated and at the destination we will check which one satisfies completely the problem's constrains.Comment: 10 pages, 2 figure

A road towards the photonic hardware implementation of artificial cognitive circuits

Many technologies we use are inspired by nature. This happens in different domains, ranging from mechanics to optics to computer sciences. Nature has incredible potentialities that man still does not know or that he striving to learn through experience. These potentialities concern the ability to solve complex problems through approaches of various types of distributed intelligence. In fact, there are forms of intelligence in nature that differ from that of man, but are nevertheless exceedingly efficient. Man has often used as a model those forms of distributed intelligence that allow colonies of animals to develop places of housing or collective behaviors of extreme complexity. Recently, M. Alonzo et alii (Sci.Rep. 8, 5716 (2018)) published a hardware implementation to solve complex routing problems in modern information networks by exploiting the immense possibilities offered by light. This article presents an addressable photonic circuit based on the decision-making processes of ant colonies looking for food. When ants search for food, they modify their surroundings by leaving traces of pheromone, which may be reinforced and function as a type of path marker for when food has been found. This process is based on stigmergy, or the modification of the environment to implement distributed decision-making processes. The photonic hardware implementation that this work proposes is a photonic X-junction that simulates this stigmergic procedure. The experimental implementation is based on the use of non-linear substrates, i.e. materials that can be modified by light, simulating the modification induced by the ants on the surrounding environment when they leave the pheromone traces. Here, two laser beams generate two crossing channels in which the index of refraction is increased with respect to the whole substrate. These channels act as integrated waveguides (almost self-written optical fibers) within which optical information can be propagated (as happens for the ants that follow traces of pheromone already “written”). The proposed device is a X-junction with two crossing waveguides, whose refractive index contrast is defined by the intensities of the writing light beams. The higher the writing intensity, the greater the induced index variation, as if it were an increasingly intense pheromone trace. The information will follow the most contrasted harm of the junction, which is driven and eventually switched by the writing light intensity. Any optical information that will be sent to the device will follow the most intense trace, i.e. the most contrasted waveguide. The paper demonstrates a device that can be wholly operated using the light and that can be the basis of complex hardware configurations that might reproduce the stigmergic distributed intelligence. This is a highly significant innovation in the field of electronic and photonic technologies, within which artificial cognition and decision processes are implemented into a hardware circuit and not in a software code

Exploiting Multiple Levels of Parallelism in Sparse Matrix-Matrix Multiplication

Sparse matrix-matrix multiplication (or SpGEMM) is a key primitive for many high-performance graph algorithms as well as for some linear solvers, such as algebraic multigrid. The scaling of existing parallel implementations of SpGEMM is heavily bound by communication. Even though 3D (or 2.5D) algorithms have been proposed and theoretically analyzed in the flat MPI model on Erdos-Renyi matrices, those algorithms had not been implemented in practice and their complexities had not been analyzed for the general case. In this work, we present the first ever implementation of the 3D SpGEMM formulation that also exploits multiple (intra-node and inter-node) levels of parallelism, achieving significant speedups over the state-of-the-art publicly available codes at all levels of concurrencies. We extensively evaluate our implementation and identify bottlenecks that should be subject to further research

An optical fiber network oracle for NP-complete problems

The modern information society is enabled by photonic fiber networks characterized by huge coverage and great complexity and ranging in size from transcontinental submarine telecommunication cables to fiber to the home and local segments. This world-wide network has yet to match the complexity of the human brain, which contains a hundred billion neurons, each with thousands of synaptic connections on average. However, it already exceeds the complexity of brains from primitive organisms, i.e., the honey bee, which has a brain containing approximately one million neurons. In this study, we present a discussion of the computing potential of optical networks as information carriers. Using a simple fiber network, we provide a proof-of-principle demonstration that this network can be treated as an optical oracle for the Hamiltonian path problem, the famous mathematical complexity problem of finding whether a set of towns can be travelled via a path in which each town is visited only once. Pronouncement of a Hamiltonian path is achieved by monitoring the delay of an optical pulse that interrogates the network, and this delay will be equal to the sum of the travel times needed to visit all of the nodes (towns). We argue that the optical oracle could solve this NP-complete problem hundreds of times faster than brute-force computing. Additionally, we discuss secure communication applications for the optical oracle and propose possible implementation in silicon photonics and plasmonic networks.Peer Reviewe

Survey of storage systems for high-performance computing

In current supercomputers, storage is typically provided by parallel distributed file systems for hot data and tape archives for cold data. These file systems are often compatible with local file systems due to their use of the POSIX interface and semantics, which eases development and debugging because applications can easily run both on workstations and supercomputers. There is a wide variety of file systems to choose from, each tuned for different use cases and implementing different optimizations. However, the overall application performance is often held back by I/O bottlenecks due to insufficient performance of file systems or I/O libraries for highly parallel workloads. Performance problems are dealt with using novel storage hardware technologies as well as alternative I/O semantics and interfaces. These approaches have to be integrated into the storage stack seamlessly to make them convenient to use. Upcoming storage systems abandon the traditional POSIX interface and semantics in favor of alternative concepts such as object and key-value storage; moreover, they heavily rely on technologies such as NVM and burst buffers to improve performance. Additional tiers of storage hardware will increase the importance of hierarchical storage management. Many of these changes will be disruptive and require application developers to rethink their approaches to data management and I/O. A thorough understanding of today's storage infrastructures, including their strengths and weaknesses, is crucially important for designing and implementing scalable storage systems suitable for demands of exascale computing

Computational Methods in Science and Engineering : Proceedings of the Workshop SimLabs@KIT, November 29 - 30, 2010, Karlsruhe, Germany

In this proceedings volume we provide a compilation of article contributions equally covering applications from different research fields and ranging from capacity up to capability computing. Besides classical computing aspects such as parallelization, the focus of these proceedings is on multi-scale approaches and methods for tackling algorithm and data complexity. Also practical aspects regarding the usage of the HPC infrastructure and available tools and software at the SCC are presented