CRAID: Online RAID upgrades using dynamic hot data reorganization

Current algorithms used to upgrade RAID arrays typically require large amounts of data to be migrated, even those that move only the minimum amount of data required to keep a balanced data load. This paper presents CRAID, a self-optimizing RAID array that performs an online block reorganization of frequently used, long-term accessed data in order to reduce this migration even further. To achieve this objective, CRAID tracks frequently used, long-term data blocks and copies them to a dedicated partition spread across all the disks in the array. When new disks are added, CRAID only needs to extend this process to the new devices to redistribute this partition, thus greatly reducing the overhead of the upgrade process. In addition, the reorganized access patterns within this partition improve the array’s performance, amortizing the copy overhead and allowing CRAID to offer a performance competitive with traditional RAIDs. We describe CRAID’s motivation and design and we evaluate it by replaying seven real-world workloads including a file server, a web server and a user share. Our experiments show that CRAID can successfully detect hot data variations and begin using new disks as soon as they are added to the array. Also, the usage of a dedicated partition improves the sequentiality of relevant data access, which amortizes the cost of reorganizations. Finally, we prove that a full-HDD CRAID array with a small distributed partition (<1.28% per disk) can compete in performance with an ideally restriped RAID-5 and a hybrid RAID-5 with a small SSD cache.Peer ReviewedPostprint (published version

Calibrating a high-resolution wavefront corrector with a static focal-plane camera

We present a method to calibrate a high-resolution wavefront-correcting device with a single, static camera, located in the focal plane; no moving of any component is needed. The method is based on a localized diversity and differential optical transfer functions (dOTF) to compute both the phase and amplitude in the pupil plane located upstream of the last imaging optics. An experiment with a spatial light modulator shows that the calibration is sufficient to robustly operate a focal-plane wavefront sensing algorithm controlling a wavefront corrector with ~40 000 degrees of freedom. We estimate that the locations of identical wavefront corrector elements are determined with a spatial resolution of 0.3% compared to the pupil diameter.Comment: 12 pages, 12 figures, accepted for publication in Applied Optic

The unlikely rise of masking interferometry: leading the way with 19th century technology

The exquisite precision delivered by interferometric techniques is rapidly being applied to more and more branches of optical astronomy. One particularly successful strategy to obtain structures at the scale of the diffraction limit is Aperture Masking Interferometry, which is presently experience a golden age with implementations at a host of large telescopes around the world. This startlingly durable technique, which turns 144 years old this year, presently sets the standard for the recovery of faint companions within a few resolution elements from the core of a stellar point spread function. This invited review will give a historical introduction and overview of the modern status of the technique, the science being delivered, and prospects for new advances and applications.Comment: This is an invited review for SPIE Amsterdam in 2012. It presents a brief history of masking interferometry, and some thoughts on future progress. 11 pages, 4 figs, lots of ref

A Multi-Code Analysis Toolkit for Astrophysical Simulation Data

The analysis of complex multiphysics astrophysical simulations presents a unique and rapidly growing set of challenges: reproducibility, parallelization, and vast increases in data size and complexity chief among them. In order to meet these challenges, and in order to open up new avenues for collaboration between users of multiple simulation platforms, we present yt (available at http://yt.enzotools.org/), an open source, community-developed astrophysical analysis and visualization toolkit. Analysis and visualization with yt are oriented around physically relevant quantities rather than quantities native to astrophysical simulation codes. While originally designed for handling Enzo's structure adaptive mesh refinement (AMR) data, yt has been extended to work with several different simulation methods and simulation codes including Orion, RAMSES, and FLASH. We report on its methods for reading, handling, and visualizing data, including projections, multivariate volume rendering, multi-dimensional histograms, halo finding, light cone generation and topologically-connected isocontour identification. Furthermore, we discuss the underlying algorithms yt uses for processing and visualizing data, and its mechanisms for parallelization of analysis tasks.Comment: 18 pages, 6 figures, emulateapj format. Resubmitted to Astrophysical Journal Supplement Series with revisions from referee. yt can be found at http://yt.enzotools.org