Parallel detrended fluctuation analysis for fast event detection on massive PMU data

("(c) 2015 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.")Phasor measurement units (PMUs) are being rapidly deployed in power grids due to their high sampling rates and synchronized measurements. The devices high data reporting rates present major computational challenges in the requirement to process potentially massive volumes of data, in addition to new issues surrounding data storage. Fast algorithms capable of processing massive volumes of data are now required in the field of power systems. This paper presents a novel parallel detrended fluctuation analysis (PDFA) approach for fast event detection on massive volumes of PMU data, taking advantage of a cluster computing platform. The PDFA algorithm is evaluated using data from installed PMUs on the transmission system of Great Britain from the aspects of speedup, scalability, and accuracy. The speedup of the PDFA in computation is initially analyzed through Amdahl's Law. A revision to the law is then proposed, suggesting enhancements to its capability to analyze the performance gain in computation when parallelizing data intensive applications in a cluster computing environment

Statistical Analysis to Extract Effective Parameters on Overall Energy Consumption of Wireless Sensor Network (WSN)

In this paper, we use statistical tools to analysis dependency between Wireless Sensor Network (WSN) parameters and overall Energy Consumption (EC). Our approach has two main phases: profiling, and effective parameter extraction. In former, a sensor network simulator is re-run 800 times with different values for eight WSN parameters to profile consumed energy in nodes; then in latter, three statistical analyses (p-value, linear and non-linear correlation) are applied to the outcome of profiling phase to extract the most effective parameters on WSN overall energy consumption.Comment: 5-pages. This paper has been accepted in PDCAT-2012 conference (http://www.pdcat2012.org/

Computational methods to engineer process-structure-property relationships in organic electronics: The case of organic photovoltaics

Ever since the Nobel prize winning work by Heeger and his colleagues, organic electronics enjoyed increasing attention from researchers all over the world. While there is a large potential for organic electronics in areas of transistors, solar cells, diodes, flexible displays, RFIDs, smart textiles, smart tattoos, artificial skin, bio-electronics, medical devices and many more, there have been very few applications that reached the market. Organic photovoltaics especially can utilize large market of untapped solar power -- portable and affordable solar conversion devices. While there are several reasons for their unavailability, a major one is the challenge of controlling device morphology at several scales, simultaneously. The morphology is intricately related to the processing of the device and strongly influences performance. Added to this is the unending development of new polymeric materials in search of high power conversion efficiencies. Fully understanding this intricate relationship between materials, processing conditions and power conversion is highly resource and time intensive. The goal of this work is to provide tightly coupled computational routes to these expensive experiments, and demonstrate process control using in-silico experiments. This goal is achieved in multiple stages and is commonly called the process-structure-property loop in material science community. We leverage recent advances in high performance computing (HPC) and high throughput computing (HTC) towards this end. Two open-source software packages were developed: GRATE and PARyOpt. GRATE provides a means to reliably and repeatably quantify TEM images for identifying transport characteristics. It solves the problem of manually quantifying large number of large images with fine details. PARyOpt is a Gaussian process based optimization library that is especially useful for optimizing expensive phenomena. Both these are highly modular and designed to be easily integrated with existing software. It is anticipated that the organic electronics community will use these tools to accelerate discovery and development of new-age devices

Timely Long Tail Identification through Agent Based Monitoring and Analytics

The increasing complexity and scale of distributed systems has resulted in the manifestation of emergent behavior which substantially affects overall system performance. A significant emergent property is that of the "Long Tail", whereby a small proportion of task stragglers significantly impact job execution completion times. To mitigate such behavior, straggling tasks occurring within the system need to be accurately identified in a timely manner. However, current approaches focus on mitigation rather than identification, which typically identify stragglers too late in the execution lifecycle. This paper presents a method and tool to identify Long Tail behavior within distributed systems in a timely manner, through a combination of online and offline analytics. This is achieved through historical analysis to profile and model task execution patterns, which then inform online analytic agents that monitor task execution at runtime. Furthermore, we provide an empirical analysis of two large-scale production Cloud data enters that demonstrate the challenge of data skew within modern distributed systems, this analysis shows that approximately 5% of task stragglers caused by data skew impact 50% of the total jobs for batch processes. Our results demonstrate that our approach is capable of identifying task stragglers less than 11% into their execution lifecycle with 98% accuracy, signifying significant improvement over current state-of-the-art practice and enables far more effective mitigation strategies in large-scale distributed systems worldwide

Matrix Factorization at Scale: a Comparison of Scientific Data Analytics in Spark and C+MPI Using Three Case Studies

We explore the trade-offs of performing linear algebra using Apache Spark, compared to traditional C and MPI implementations on HPC platforms. Spark is designed for data analytics on cluster computing platforms with access to local disks and is optimized for data-parallel tasks. We examine three widely-used and important matrix factorizations: NMF (for physical plausability), PCA (for its ubiquity) and CX (for data interpretability). We apply these methods to TB-sized problems in particle physics, climate modeling and bioimaging. The data matrices are tall-and-skinny which enable the algorithms to map conveniently into Spark's data-parallel model. We perform scaling experiments on up to 1600 Cray XC40 nodes, describe the sources of slowdowns, and provide tuning guidance to obtain high performance