A Backend Framework for the Efficient Management of Power System Measurements

Increased adoption and deployment of phasor measurement units (PMU) has provided valuable fine-grained data over the grid. Analysis over these data can provide insight into the health of the grid, thereby improving control over operations. Realizing this data-driven control, however, requires validating, processing and storing massive amounts of PMU data. This paper describes a PMU data management system that supports input from multiple PMU data streams, features an event-detection algorithm, and provides an efficient method for retrieving archival data. The event-detection algorithm rapidly correlates multiple PMU data streams, providing details on events occurring within the power system. The event-detection algorithm feeds into a visualization component, allowing operators to recognize events as they occur. The indexing and data retrieval mechanism facilitates fast access to archived PMU data. Using this method, we achieved over 30x speedup for queries with high selectivity. With the development of these two components, we have developed a system that allows efficient analysis of multiple time-aligned PMU data streams.Comment: Published in Electric Power Systems Research (2016), not available ye

Efficient analysis and storage of large-scale genomic data

The impending advent of population-scaled sequencing cohorts involving tens of millions of individuals with matched phenotypic measurements will produce unprecedented volumes of genetic data. Storing and analysing such gargantuan datasets places computational performance at a pivotal position in medical genomics. In this thesis, I explore the potential for accelerating and parallelizing standard genetics workflows, file formats, and algorithms using both hardware-accelerated vectorization, parallel and distributed algorithms, and heterogeneous computing. First, I describe a novel bit-counting operation termed the positional population-count, which can be used together with succinct representations and standard efficient operations to accelerate many genetic calculations. In order to enable the use of this new operator and the canonical population count on any target machine I developed a unified low-level library using CPU dispatching to select the optimal method contingent on the available instruction set architecture and the given input size at run-time. As a proof-of-principle application, I apply the positional population-count operator to computing quality control-related summary statistics for terabyte-scaled sequencing readsets with >3,800-fold speed improvements. As another application, I describe a framework for efficiently computing the cardinality of set intersection using these operators and applied this framework to efficiently compute genome-wide linkage-disequilibrium in datasets with up to 67 million samples resulting in up to >60-fold improvements in speed for dense genotypic vectors and up to >250,000-fold savings in memory and >100,000-fold improvement in speed for sparse genotypic vectors. I next describe a framework for handling the terabytes of compressed output data and describe graphical routines for visualizing long-range linkage-disequilibrium blocks as seen over many human centromeres. Finally, I describe efficient algorithms for storing and querying very large genetic datasets and specialized algorithms for the genotype component of such datasets with >10,000-fold savings in memory compared to the current interchange format.Wellcome Trus

Computación paralela heterogénea en registro de imágenes y aplicaciones de álgebra lineal

This doctoral thesis focuses on GPU acceleration of medical image registration and sparse general matrix-matrix multiplication (SpGEMM). The comprehensive work presented here aims to enable new possibilities in Image Guided Surgery (IGS). IGS provides the surgeon with advanced navigation tools during surgery. Image registration, which is a part of IGS, is computationally demanding, therefore GPU acceleration is greatly desirable. spGEMM, which is an essential part in many scientific and data analytics applications, e.g., graph applications, is also a useful tool in biomechanical modeling and sparse vessel network registration. We present this work in two parts. The first part of this thesis describes the optimization of the most demanding part of non-rigid Free Form Deformation registration, i.e., B-spline interpolation. Our novel optimization technique minimizes the data movement between processing cores and memory and maximizes the utilization of the very fast register file. In addition, our approach re-formulates B-spline interpolation to fully utilize Fused Multiply Accumulation instructions for additional benefits in performance and accuracy. Our optimized B-spline interpolation provides significant speedup to image registration. The second part describes the optimization of spGEMM. Hardware manufacturers, with the aim of increasing the performance of deep-learning, created specialized dense matrix multiplication units, called Tensor Core Units (TCUs). However, until now, no work takes advantage of TCUs for sparse matrix multiplication. With this work we provide the first TCU implementation of spGEMM and prove its benefits over conventional GPU spGEMM.Esta tesis doctoral se centra en la aceleración por GPU del registro de imágenes médicas y la multiplicación de matrices dispersas (SpGEMM). El exhaustivo trabajo presentado aquí tiene como objetivo permitir nuevas posibilidades en la cirugía guiada por imagen (IGS). IGS proporciona al cirujano herramientas de navegación avanzadas durante la cirugía. El registro de imágenes, parte de IGS computacionalmente exigente, por lo tanto, la aceleración en GPU es muy deseable. spGEMM, la cual es una parte esencial en muchas aplicaciones científicas y de análisis de datos, por ejemplo, aplicaciones de gráficos, también es una herramienta útil en el modelado biomecánico y el registro de redes de vasos dispersos. Presentamos este trabajo en dos partes. La primera parte de esta tesis describe la optimización de la parte más exigente del registro de deformación de forma libre no rígida, es decir, la interpolación B-spline. Nuestra novedosa técnica de optimización minimiza el movimiento de datos entre los núcleos de procesamiento y la memoria y maximiza la utilización del archivo de registro rápido. Además, nuestro enfoque reformula la interpolación B-spline para utilizar completamente las instrucciones de multiplicación-acumulación fusionada (FMAC) para obtener beneficios adicionales en rendimiento y precisión. Nuestra interpolación B-spline optimizada proporciona una aceleración significativa en el registro de imágenes. La segunda parte describe la optimización de spGEMM. Los fabricantes de hardware, con el objetivo de aumentar el rendimiento del aprendizaje profundo, crearon unidades especializadas de multiplicación de matrices densas, llamadas Tensor Core Units (TCU). Sin embargo, hasta ahora, no se ha encontrado ningún trabajo aprovecha las TCU para la multiplicación de matrices dispersas. Con este trabajo, proporcionamos la primera implementación TCU de spGEMM y demostramos sus beneficios sobre la spGEMM convencional operada sobre dispositivos GPU

Indicators of replicative damage in equine tendon fibroblast monolayers

<p>Background: Superficial digital flexor tendon (SDFT) injuries of horses usually follow cumulative matrix microdamage; it is not known why the reparative abilities of tendon fibroblasts are overwhelmed or subverted. Relevant in vitro studies of this process require fibroblasts not already responding to stresses caused by the cell culture protocols. We investigated indicators of replicative damage in SDFT fibroblast monolayers, effects of this on their reparative ability, and measures that can be taken to reduce it.</p> <p>Results: We found significant evidence of replicative stress, initially observing consistently large numbers of binucleate (BN) cells. A more variable but prominent feature was the presence of numerous gammaH2AX (γH2AX) puncta in nuclei, this being a histone protein that is phosphorylated in response to DNA double-stranded breaks (DSBs). Enrichment for injury detection and cell cycle arrest factors (p53 (ser15) and p21) occurred most frequently in BN cells; however, their numbers did not correlate with DNA damage levels and it is likely that the two processes have different causative mechanisms. Such remarkable levels of injury and binucleation are usually associated with irradiation, or treatment with cytoskeletal-disrupting agents.</p> <p>Both DSBs and BN cells were greatest in subconfluent (replicating) monolayers. The DNA-damaged cells co-expressed the replication markers TPX2/repp86 and centromere protein F. Once damaged in the early stages of culture establishment, fibroblasts continued to express DNA breaks with each replicative cycle. However, significant levels of cell death were not measured, suggesting that DNA repair was occurring. Comet assays showed that DNA repair was delayed in proportion to levels of genotoxic stress.</p> <p>Conclusions: Researchers using tendon fibroblast monolayers should assess their “health” using γH2AX labelling. Continued use of early passage cultures expressing initially high levels of γH2AX puncta should be avoided for mechanistic studies and ex-vivo therapeutic applications, as this will not be resolved with further replicative cycling. Low density cell culture should be avoided as it enriches for both DNA damage and mitotic defects (polyploidy). As monolayers differing only slightly in baseline DNA damage levels showed markedly variable responses to a further injury, studies of effects of various stressors on tendon cells must be very carefully controlled.</p&gt

GraphMineSuite: Enabling High-Performance and Programmable Graph Mining Algorithms with Set Algebra

We propose GraphMineSuite (GMS): the first benchmarking suite for graph mining that facilitates evaluating and constructing high-performance graph mining algorithms. First, GMS comes with a benchmark specification based on extensive literature review, prescribing representative problems, algorithms, and datasets. Second, GMS offers a carefully designed software platform for seamless testing of different fine-grained elements of graph mining algorithms, such as graph representations or algorithm subroutines. The platform includes parallel implementations of more than 40 considered baselines, and it facilitates developing complex and fast mining algorithms. High modularity is possible by harnessing set algebra operations such as set intersection and difference, which enables breaking complex graph mining algorithms into simple building blocks that can be separately experimented with. GMS is supported with a broad concurrency analysis for portability in performance insights, and a novel performance metric to assess the throughput of graph mining algorithms, enabling more insightful evaluation. As use cases, we harness GMS to rapidly redesign and accelerate state-of-the-art baselines of core graph mining problems: degeneracy reordering (by up to >2x), maximal clique listing (by up to >9x), k-clique listing (by 1.1x), and subgraph isomorphism (by up to 2.5x), also obtaining better theoretical performance bounds

Bitmap Indexing : Energy Applications and Improvements

Large databases and data warehouses are becoming prevalent for the storage and management of energy data. Accelerating the rates at which data can be retrieved is beneficial not only to allow for more efficient search of the data, but also to be integrated with other energy system tools. In this work, a fast indexing and data retrieval method, known commonly as a bitmap index, is created to facilitate intelligent querying and indexing of data generated by phasor measurement units (PMU) at a rate of 60 Hz. In addition, compression of these bitmap indexes is parallelized, and the performance quality of bitmaps created over this dynamic database are tested. We find that bitmaps are amenable to managing efficient access to large amounts of PMU data. Furthermore, the bitmap-management process will provide decreased access time for data retrieval as well as decreased memory usage. From these experiments, we are able to improve compression times while discovering new research opportunities, decrease access times to the PMU database by 30 times the conventional method currently utilizes, and introduce a feasible method for bitmap operations over a dynamic database

Frequent itemset mining on multiprocessor systems

Frequent itemset mining is an important building block in many data mining applications like market basket analysis, recommendation, web-mining, fraud detection, and gene expression analysis. In many of them, the datasets being mined can easily grow up to hundreds of gigabytes or even terabytes of data. Hence, efficient algorithms are required to process such large amounts of data. In recent years, there have been many frequent-itemset mining algorithms proposed, which however (1) often have high memory requirements and (2) do not exploit the large degrees of parallelism provided by modern multiprocessor systems. The high memory requirements arise mainly from inefficient data structures that have only been shown to be sufficient for small datasets. For large datasets, however, the use of these data structures force the algorithms to go out-of-core, i.e., they have to access secondary memory, which leads to serious performance degradations. Exploiting available parallelism is further required to mine large datasets because the serial performance of processors almost stopped increasing. Algorithms should therefore exploit the large number of available threads and also the other kinds of parallelism (e.g., vector instruction sets) besides thread-level parallelism. In this work, we tackle the high memory requirements of frequent itemset mining twofold: we (1) compress the datasets being mined because they must be kept in main memory during several mining invocations and (2) improve existing mining algorithms with memory-efficient data structures. For compressing the datasets, we employ efficient encodings that show a good compression performance on a wide variety of realistic datasets, i.e., the size of the datasets is reduced by up to 6.4x. The encodings can further be applied directly while loading the dataset from disk or network. Since encoding and decoding is repeatedly required for loading and mining the datasets, we reduce its costs by providing parallel encodings that achieve high throughputs for both tasks. For a memory-efficient representation of the mining algorithms’ intermediate data, we propose compact data structures and even employ explicit compression. Both methods together reduce the intermediate data’s size by up to 25x. The smaller memory requirements avoid or delay expensive out-of-core computation when large datasets are mined. For coping with the high parallelism provided by current multiprocessor systems, we identify the performance hot spots and scalability issues of existing frequent-itemset mining algorithms. The hot spots, which form basic building blocks of these algorithms, cover (1) counting the frequency of fixed-length strings, (2) building prefix trees, (3) compressing integer values, and (4) intersecting lists of sorted integer values or bitmaps. For all of them, we discuss how to exploit available parallelism and provide scalable solutions. Furthermore, almost all components of the mining algorithms must be parallelized to keep the sequential fraction of the algorithms as small as possible. We integrate the parallelized building blocks and components into three well-known mining algorithms and further analyze the impact of certain existing optimizations. Our algorithms are already single-threaded often up an order of magnitude faster than existing highly optimized algorithms and further scale almost linear on a large 32-core multiprocessor system. Although our optimizations are intended for frequent-itemset mining algorithms, they can be applied with only minor changes to algorithms that are used for mining of other types of itemsets

Towards the Next Generation of Clinical Decision Support: Overcoming the Integration Challenges of Genomic Data and Electronic Health Records

The wide adoption of electronic health records (EHRs), the unprecedented abundance of genomic data, and the rapid advancements in computational methods have paved the way for next generation clinical decision support (NGCDS) systems. NGCDS provides significant opportunities for the prevention, early detection, and the personalized treatment of complex diseases. The integration of genomic and EHR data into the NGCDS workflow is faced with significant challenges due to the high complexity and sheer magnitude of the associated data. This dissertation performs an in depth investigation to address the computational and algorithmic challenges of integrating genomic and EHR data within the NGCDS workflow. In particular, the dissertation (i) defines the major genomic challenges NGCDS faces and discusses possible resolution directions, (ii) proposes an accelerated method for processing raw genomic data, (iii) introduces a data representation and compression method to store the processed genomic outcomes in a database schema, and finally, (iv) investigates the feasibility of using EHR data to produce accurate disease risk assessments. We hope that the proposed solutions will expedite the adoption of NGCDS and help advance the state of healthcare

Towards the Next Generation of Clinical Decision Support: Overcoming the Integration Challenges of Genomic Data and Electronic Health Records

The wide adoption of electronic health records (EHRs), the unprecedented abundance of genomic data, and the rapid advancements in computational methods have paved the way for next generation clinical decision support (NGCDS) systems. NGCDS provides significant opportunities for the prevention, early detection, and the personalized treatment of complex diseases. The integration of genomic and EHR data into the NGCDS workflow is faced with significant challenges due to the high complexity and sheer magnitude of the associated data. This dissertation performs an in depth investigation to address the computational and algorithmic challenges of integrating genomic and EHR data within the NGCDS workflow. In particular, the dissertation (i) defines the major genomic challenges NGCDS faces and discusses possible resolution directions, (ii) proposes an accelerated method for processing raw genomic data, (iii) introduces a data representation and compression method to store the processed genomic outcomes in a database schema, and finally, (iv) investigates the feasibility of using EHR data to produce accurate disease risk assessments. We hope that the proposed solutions will expedite the adoption of NGCDS and help advance the state of healthcare