RAID-2: Design and implementation of a large scale disk array controller

We describe the implementation of a large scale disk array controller and subsystem incorporating over 100 high performance 3.5 inch disk drives. It is designed to provide 40 MB/s sustained performance and 40 GB capacity in three 19 inch racks. The array controller forms an integral part of a file server that attaches to a Gb/s local area network. The controller implements a high bandwidth interconnect between an interleaved memory, an XOR calculation engine, the network interface (HIPPI), and the disk interfaces (SCSI). The system is now functionally operational, and we are tuning its performance. We review the design decisions, history, and lessons learned from this three year university implementation effort to construct a truly large scale system assembly

The case for a Hardware Filesystem

As secondary storage devices get faster with flash based solid state drives (SSDs) and emerging technologies like phase change memories (PCM), overheads in system software like operating system (OS) and filesystem become prominent and may limit the potential performance improvements. Moreover, with rapidly increasing on-chip core count, monolithic operating systems will face scalability issues on these many-core chips. Future operating systems are likely to have a distributed nature, with a separation of operating system services amongst cores. Also, general purpose processors are known to be both performance and power inefficient while executing operating system code. In the domain of High Performance Computing with FPGAs too, relying on the OS for file I/O transactions using slow embedded processors, hinders performance. Migrating the filesystem into a dedicated hardware core, has the potential of improving the performance of data-intensive applications by bypassing the OS stack to provide higher bandwdith and reduced latency while accessing disks. To test the feasibility of this idea, an FPGA-based Hardware Filesystem (HWFS) was designed with five basic operations (open, read, write, delete and seek). Furthermore, multi-disk and RAID-0 (striping) support has been implemented as an option in the filesystem. In order to reduce design complexity and facilitate easier testing of the HWFS, a RAM disk was used initially. The filesystem core has been integrated and tested with a hardware application core (BLAST) as well as a multi-node FPGA network to provide remote-disk access. Finally, a SATA IP core was developed and directly integrated with HWFS to test with SSDs. For evaluation, HWFS's performance was compared to an Ext2 filesystem, both on an FPGA-based soft processor as well as a modern AMD Opteron Linux server with sequential and random workloads. Results prove that the Hardware Filesystem and supporting infrastructure provide substantial performance improvement over software only systems. The system is also resource efficient consuming less than 3% of logic and 5% of the Block RAMs of a Xilinx Virtex-6 chip

NSMS probe recorder design and development

The real-time Non-Contact Stress Measurement System (NSMS) currently used at AEDC calculates the vibration of rotating blades by capturing the time of arrival for each blade. The time of arrival is determined by a triggering circuit that is activated when the signal from the engine probe crosses a predetermined threshold. In its current configuration, the NSMS system only saves post-processed data. A system that records the raw signals from the probes was developed to allow reprocessing the data whenever necessary. The probe recorder system consists of analog-to-digital conversion hardware to capture the signals, data storage for the files, and digital-to-analog hardware to replay the signals. The system accommodates a maximum of 32 channels, a maximum sampling rate of 20 MHz, and a total bandwidth of up to 160 megabytes per second. Sixteen-bit resolution is used in digitizing the analog waveforms to minimize quantization errors.The incoming data is transferred using FPDP, capable of 160 MB/sec, and PCI-X, capable of 528 MB/sec. Large amounts of high speed (3200 MB/sec) random access memory coupled with two dual-core processors were included for data transfer buffering and program execution. As the final destination, a RAID array connected to a PCI Express interface was implemented for 240 MB/sec data storage. Laboratory tests were conducted on the system to verify performance. The RAID array exceeded expectations for disk writing but reduced bandwidth was observed for read operations. The relationship between the input analog signals and the reproduced waveforms was checked and, except for one case, performed identically to the simulated system transfer function. Long duration tests were performed to verify data transfers at the maximum settings and proved that the system could operate continuously without data loss.Due to the large amounts of data, a brief study of offline compression techniques was conducted. Lossy compression was investigated but was not implemented at this time due to unwanted distortion and loss of critical data. Lossless compression using WinZip was implemented as a compromise between ideal compression ratios and data retention expectations

Super-scalar RAM-CPU cache compression

High-performance data-intensive query processing tasks like OLAP, data mining or scientific data analysis can be severely I/O bound, even when high-e

QLOOP: Linux driver to mount QCOW2 virtual disks

Development of a Linux module that converts a QCOW2 file (a disk image format employed by QEMU and KVM projects) into a block device

QLOOP: Linux driver to mount QCOW2 virtual disks

Development of a Linux module that converts a QCOW2 file (a disk image format employed by QEMU and KVM projects) into a block device

Network storage system simulation and performance optimization

Master'sMASTER OF ENGINEERIN

A shared-disk parallel cluster file system

Dissertação apresentada para obtenção do Grau de Doutor em Informática Pela Universidade Nova de Lisboa, Faculdade de Ciências e TecnologiaToday, clusters are the de facto cost effective platform both for high performance computing (HPC) as well as IT environments. HPC and IT are quite different environments and differences include, among others, their choices on file systems and storage: HPC favours parallel file systems geared towards maximum I/O bandwidth, but which are not fully POSIX-compliant and were devised to run on top of (fault prone) partitioned storage; conversely, IT data centres favour both external disk arrays (to provide highly available storage) and POSIX compliant file systems, (either general purpose or shared-disk cluster file systems, CFSs). These specialised file systems do perform very well in their target environments provided that applications do not require some lateral features, e.g., no file locking on parallel file systems, and no high performance writes over cluster-wide shared files on CFSs. In brief, we can say that none of the above approaches solves the problem of providing high levels of reliability and performance to both worlds. Our pCFS proposal makes a contribution to change this situation: the rationale is to take advantage on the best of both – the reliability of cluster file systems and the high performance of parallel file systems. We don’t claim to provide the absolute best of each, but we aim at full POSIX compliance, a rich feature set, and levels of reliability and performance good enough for broad usage – e.g., traditional as well as HPC applications, support of clustered DBMS engines that may run over regular files, and video streaming. pCFS’ main ideas include: · Cooperative caching, a technique that has been used in file systems for distributed disks but, as far as we know, was never used either in SAN based cluster file systems or in parallel file systems. As a result, pCFS may use all infrastructures (LAN and SAN) to move data. · Fine-grain locking, whereby processes running across distinct nodes may define nonoverlapping byte-range regions in a file (instead of the whole file) and access them in parallel, reading and writing over those regions at the infrastructure’s full speed (provided that no major metadata changes are required). A prototype was built on top of GFS (a Red Hat shared disk CFS): GFS’ kernel code was slightly modified, and two kernel modules and a user-level daemon were added. In the prototype, fine grain locking is fully implemented and a cluster-wide coherent cache is maintained through data (page fragments) movement over the LAN. Our benchmarks for non-overlapping writers over a single file shared among processes running on different nodes show that pCFS’ bandwidth is 2 times greater than NFS’ while being comparable to that of the Parallel Virtual File System (PVFS), both requiring about 10 times more CPU. And pCFS’ bandwidth also surpasses GFS’ (600 times for small record sizes, e.g., 4 KB, decreasing down to 2 times for large record sizes, e.g., 4 MB), at about the same CPU usage.Lusitania, Companhia de Seguros S.A, Programa IBM Shared University Research (SUR