Ground-truth prediction to accelerate soft-error impact analysis for iterative methods

Understanding the impact of soft errors on applications can be expensive. Often, it requires an extensive error injection campaign involving numerous runs of the full application in the presence of errors. In this paper, we present a novel approach to arriving at the ground truth-the true impact of an error on the final output-for iterative methods by observing a small number of iterations to learn deviations between normal and error-impacted execution. We develop a machine learning based predictor for three iterative methods to generate ground-truth results without running them to completion for every error injected. We demonstrate that this approach achieves greater accuracy than alternative prediction strategies, including three existing soft error detection strategies. We demonstrate the effectiveness of the ground truth prediction model in evaluating vulnerability and the effectiveness of soft error detection strategies in the context of iterative methods.This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research under Award Number 66905, program manager Lucy Nowell. Pacific Northwest National Laboratory is operated by Battelle for DOE under Contract DE-AC05-76RL01830.Peer ReviewedPostprint (author's final draft

An extensive study on iterative solver resilience : characterization, detection and prediction

Soft errors caused by transient bit flips have the potential to significantly impactan applicalion's behavior. This has motivated the design of an array of techniques to detect, isolate, and correct soft errors using microarchitectural, architectural, compilation­based, or application-level techniques to minimize their impact on the executing application. The first step toward the design of good error detection/correction techniques involves an understanding of an application's vulnerability to soft errors. This work focuses on silent data e orruption's effects on iterative solvers and efforts to mitigate those effects. In this thesis, we first present the first comprehensive characterizalion of !he impact of soft errors on !he convergen ce characteris tics of six iterative methods using application-level fault injection. We analyze the impact of soft errors In terms of the type of error (single-vs multi-bit), the distribution and location of bits affected, the data structure and statement impacted, and varialion with time. We create a public access database with more than 1.5 million fault injection results. We then analyze the performance of soft error detection mechanisms and present the comparalive results. Molivated by our observations, we evaluate a machine-learning based detector that takes as features that are the runtime features observed by the individual detectors to arrive al their conclusions. Our evalualion demonstrates improved results over individual detectors. We then propase amachine learning based method to predict a program's error behavior to make fault injection studies more efficient. We demonstrate this method on asse ssing the performance of soft error detectors. We show that our method maintains 84% accuracy on average with up to 53% less cost. We also show, once a model is trained further fault injection tests would cost 10% of the expected full fault injection runs.“Soft errors” causados por cambios de estado transitorios en bits, tienen el potencial de impactar significativamente el comportamiento de una aplicación. Esto, ha motivado el diseño de una variedad de técnicas para detectar, aislar y corregir soft errors aplicadas a micro-arquitecturas, arquitecturas, tiempo de compilación y a nivel de aplicación para minimizar su impacto en la ejecución de una aplicación. El primer paso para diseñar una buna técnica de detección/corrección de errores, implica el conocimiento de las vulnerabilidades de la aplicación ante posibles soft errors. Este trabajo se centra en los efectos de la corrupción silenciosa de datos en soluciones iterativas, así como en los esfuerzos para mitigar esos efectos. En esta tesis, primeramente, presentamos la primera caracterización extensiva del impacto de soft errors sobre las características convergentes de seis métodos iterativos usando inyección de fallos a nivel de aplicación. Analizamos el impacto de los soft errors en términos del tipo de error (único vs múltiples-bits), de la distribución y posición de los bits afectados, las estructuras de datos, instrucciones afectadas y de las variaciones en el tiempo. Creamos una base de datos pública con más de 1.5 millones de resultados de inyección de fallos. Después, analizamos el desempeño de mecanismos de detección de soft errors actuales y presentamos los resultados de su comparación. Motivados por las observaciones de los resultados presentados, evaluamos un detector de soft errors basado en técnicas de machine learning que toma como entrada las características observadas en el tiempo de ejecución individual de los detectores anteriores al llegar a su conclusión. La evaluación de los resultados obtenidos muestra una mejora por sobre los detectores individualmente. Basados en estos resultados propusimos un método basado en machine learning para predecir el comportamiento de los errores en un programa con el fin de hacer el estudio de inyección de errores mas eficiente. Presentamos este método para evaluar el rendimiento de los detectores de soft errors. Demostramos que nuestro método mantiene una precisión del 84% en promedio con hasta un 53% de mejora en el tiempo de ejecución. También mostramos que una vez que un modelo ha sido entrenado, las pruebas de inyección de errores siguientes costarían 10% del tiempo esperado de ejecución.Postprint (published version

Weak Disambiguation for Partial Structured Output Learning

Existing disambiguation strategies for partial structured output learning just cannot generalize well to solve the problem that there are some candidates which can be false positive or similar to the ground-truth label. In this paper, we propose a novel weak disambiguation for partial structured output learning (WD-PSL). First, a piecewise large margin formulation is generalized to partial structured output learning, which effectively avoids handling large number of candidate structured outputs for complex structures. Second, in the proposed weak disambiguation strategy, each candidate label is assigned with a confidence value indicating how likely it is the true label, which aims to reduce the negative effects of wrong ground-truth label assignment in the learning process. Then two large margins are formulated to combine two types of constraints which are the disambiguation between candidates and non-candidates, and the weak disambiguation for candidates. In the framework of alternating optimization, a new 2n-slack variables cutting plane algorithm is developed to accelerate each iteration of optimization. The experimental results on several sequence labeling tasks of Natural Language Processing show the effectiveness of the proposed model

Accelerating Radiation Dose Calculation with High Performance Computing and Machine Learning for Large-scale Radiotherapy Treatment Planning

Radiation therapy is powered by modern techniques in precise planning and executionof radiation delivery, which are being rapidly improved to maximize its benefit to cancerpatients. In the last decade, radiotherapy experienced the introduction of advanced methodsfor automatic beam orientation optimization, real-time tumor tracking, daily planadaptation, and many others, which improve the radiation delivery precision, planning easeand reproducibility, and treatment efficacy. However, such advanced paradigms necessitatethe calculation of orders of magnitude more causal dose deposition data, increasing the timerequirement of all pre-planning dose calculation. Principles of high-performance computingand machine learning were applied to address the insufficient speeds of widely-used dosecalculation algorithms to facilitate translation of these advanced treatment paradigms intoclinical practice.To accelerate CT-guided X-ray therapies, Collapsed-Cone Convolution-Superposition(CCCS), a state-of-the-art analytical dose calculation algorithm, was accelerated through itsnovel implementation on highly parallelized GPUs. This context-based GPU-CCCS approachtakes advantage of X-ray dose deposition compactness to parallelize calculation acrosshundreds of beamlets, reducing hardware-specific overheads, and enabling acceleration bytwo to three orders of magnitude compared to existing GPU-based beamlet-by-beamletapproaches. Near-linear increases in acceleration are achieved with a distributed, multi-GPUimplementation of context-based GPU-CCCS.Dose calculation for MR-guided treatment is complicated by electron return effects(EREs), exhibited by ionizing electrons in the strong magnetic field of the MRI scanner. EREsnecessitate the use of much slower Monte Carlo (MC) dose calculation, limiting the clinicalapplication of advanced treatment paradigms due to time restrictions. An automaticallydistributed framework for very-large-scale MC dose calculation was developed, grantinglinear scaling of dose calculation speed with the number of utilized computational cores. Itwas then harnessed to efficiently generate a large dataset of paired high- and low-noise MCdoses in a 1.5 tesla magnetic field, which were used to train a novel deep convolutionalneural network (CNN), DeepMC, to predict low-noise dose from faster high-noise MC-simulation. DeepMC enables 38-fold acceleration of MR-guided X-ray beamlet dosecalculation, while remaining synergistic with existing MC acceleration techniques to achievemultiplicative speed improvements.This work redefines the expectation of X-ray dose calculation speed, making it possibleto apply new highly-beneficial treatment paradigms to standard clinical practice for the firsttime

A proximal iteration for deconvolving Poisson noisy images using sparse representations

We propose an image deconvolution algorithm when the data is contaminated by Poisson noise. The image to restore is assumed to be sparsely represented in a dictionary of waveforms such as the wavelet or curvelet transforms. Our key contributions are: First, we handle the Poisson noise properly by using the Anscombe variance stabilizing transform leading to a {\it non-linear} degradation equation with additive Gaussian noise. Second, the deconvolution problem is formulated as the minimization of a convex functional with a data-fidelity term reflecting the noise properties, and a non-smooth sparsity-promoting penalties over the image representation coefficients (e.g. $\ell_1$-norm). Third, a fast iterative backward-forward splitting algorithm is proposed to solve the minimization problem. We derive existence and uniqueness conditions of the solution, and establish convergence of the iterative algorithm. Finally, a GCV-based model selection procedure is proposed to objectively select the regularization parameter. Experimental results are carried out to show the striking benefits gained from taking into account the Poisson statistics of the noise. These results also suggest that using sparse-domain regularization may be tractable in many deconvolution applications with Poisson noise such as astronomy and microscopy

Parallel Implementation of Efficient Search Schemes for the Inference of Cancer Progression Models

The emergence and development of cancer is a consequence of the accumulation over time of genomic mutations involving a specific set of genes, which provides the cancer clones with a functional selective advantage. In this work, we model the order of accumulation of such mutations during the progression, which eventually leads to the disease, by means of probabilistic graphic models, i.e., Bayesian Networks (BNs). We investigate how to perform the task of learning the structure of such BNs, according to experimental evidence, adopting a global optimization meta-heuristics. In particular, in this work we rely on Genetic Algorithms, and to strongly reduce the execution time of the inference -- which can also involve multiple repetitions to collect statistically significant assessments of the data -- we distribute the calculations using both multi-threading and a multi-node architecture. The results show that our approach is characterized by good accuracy and specificity; we also demonstrate its feasibility, thanks to a 84x reduction of the overall execution time with respect to a traditional sequential implementation

Machine Learning-Driven Surrogate Models for Electrolytes

We have developed a lattice Monte Carlo (MC) simulation based on the diffusion-limited aggregation model that accounts for the effect of the physical properties of ionic liquids (ILs) on lithium dendrite growth. Our simulations show that the size asymmetry between the cation and anion, the dielectric constant, and the volume fraction of ILs are critical factors to significantly suppress the dendrite growth, primarily due to substantial changes in electric-field screening. Specifically, the volume fraction of ILs has the optimal value for dendrite suppression. The present simulation method indicates potential challenges for the model extension to macroscopic systems. Therefore, we also develop ensemble neural networks (ENNs) in machine learning methods with training datasets derived from the MC simulations by considering the input descriptors with the dielectric constant, the model parameter for the fractal dimension of the dendrite, the volume fraction of ILs, and the applied voltage. Our ENNs can predict the highly nonmonotonic trend of the simulation results from only one-tenth of simulation runs, thus significantly reducing the required computation time. To further examine the efficacy of our new ENN methods in practical applications, we apply ENNs to the study of the dielectric constants of salt-free and salt-doped solvents. Seven common solvents and NaCl solutions with various salt concentrations are considered examples. Despite the significant 50-time reduction in the number of training data, the predictions of the ENNs with batch normalization or bootstrap aggregating are largely consistent with the ground truths, tracing the optimal values out of statistically noisy data. Furthermore, we investigate the phase behaviors of cellulose and ILs mixtures by combining ENNs with unsupervised learning. As a result, K-means clustering and hierarchical clustering can automatically classify solubility phases and determine the boundaries of phases. Our work proves that machine learning could be a promising tool for studying soft matter systems

CoreDiff: Contextual Error-Modulated Generalized Diffusion Model for Low-Dose CT Denoising and Generalization

Low-dose computed tomography (CT) images suffer from noise and artifacts due to photon starvation and electronic noise. Recently, some works have attempted to use diffusion models to address the over-smoothness and training instability encountered by previous deep-learning-based denoising models. However, diffusion models suffer from long inference times due to the large number of sampling steps involved. Very recently, cold diffusion model generalizes classical diffusion models and has greater flexibility. Inspired by the cold diffusion, this paper presents a novel COntextual eRror-modulated gEneralized Diffusion model for low-dose CT (LDCT) denoising, termed CoreDiff. First, CoreDiff utilizes LDCT images to displace the random Gaussian noise and employs a novel mean-preserving degradation operator to mimic the physical process of CT degradation, significantly reducing sampling steps thanks to the informative LDCT images as the starting point of the sampling process. Second, to alleviate the error accumulation problem caused by the imperfect restoration operator in the sampling process, we propose a novel ContextuaL Error-modulAted Restoration Network (CLEAR-Net), which can leverage contextual information to constrain the sampling process from structural distortion and modulate time step embedding features for better alignment with the input at the next time step. Third, to rapidly generalize to a new, unseen dose level with as few resources as possible, we devise a one-shot learning framework to make CoreDiff generalize faster and better using only a single LDCT image (un)paired with NDCT. Extensive experimental results on two datasets demonstrate that our CoreDiff outperforms competing methods in denoising and generalization performance, with a clinically acceptable inference time. Source code is made available at https://github.com/qgao21/CoreDiff.Comment: IEEE Transactions on Medical Imaging, 202