Throughput-Distortion Computation Of Generic Matrix Multiplication: Toward A Computation Channel For Digital Signal Processing Systems

The generic matrix multiply (GEMM) function is the core element of high-performance linear algebra libraries used in many computationally-demanding digital signal processing (DSP) systems. We propose an acceleration technique for GEMM based on dynamically adjusting the imprecision (distortion) of computation. Our technique employs adaptive scalar companding and rounding to input matrix blocks followed by two forms of packing in floating-point that allow for concurrent calculation of multiple results. Since the adaptive companding process controls the increase of concurrency (via packing), the increase in processing throughput (and the corresponding increase in distortion) depends on the input data statistics. To demonstrate this, we derive the optimal throughput-distortion control framework for GEMM for the broad class of zero-mean, independent identically distributed, input sources. Our approach converts matrix multiplication in programmable processors into a computation channel: when increasing the processing throughput, the output noise (error) increases due to (i) coarser quantization and (ii) computational errors caused by exceeding the machine-precision limitations. We show that, under certain distortion in the GEMM computation, the proposed framework can significantly surpass 100% of the peak performance of a given processor. The practical benefits of our proposal are shown in a face recognition system and a multi-layer perceptron system trained for metadata learning from a large music feature database.Comment: IEEE Transactions on Signal Processing (vol. 60, 2012

Conditional deep generative models as surrogates for spatial field solution reconstruction with quantified uncertainty in Structural Health Monitoring applications

In recent years, increasingly complex computational models are being built to describe physical systems which has led to increased use of surrogate models to reduce computational cost. In problems related to Structural Health Monitoring (SHM), models capable of both handling high-dimensional data and quantifying uncertainty are required. In this work, our goal is to propose a conditional deep generative model as a surrogate aimed at such applications and high-dimensional stochastic structural simulations in general. To that end, a conditional variational autoencoder (CVAE) utilizing convolutional neural networks (CNNs) is employed to obtain reconstructions of spatially ordered structural response quantities for structural elements that are subjected to stochastic loading. Two numerical examples, inspired by potential SHM applications, are utilized to demonstrate the performance of the surrogate. The model is able to achieve high reconstruction accuracy compared to the reference Finite Element (FE) solutions, while at the same time successfully encoding the load uncertainty.Comment: 28 pages, 14 figures. Submitted to Elsevier Journal for publicatio

Control theoretically explainable application of autoencoder methods to fault detection in nonlinear dynamic systems

This paper is dedicated to control theoretically explainable application of autoencoders to optimal fault detection in nonlinear dynamic systems. Autoencoder-based learning is a standard method of machine learning technique and widely applied for fault (anomaly) detection and classification. In the context of representation learning, the so-called latent (hidden) variable plays an important role towards an optimal fault detection. In ideal case, the latent variable should be a minimal sufficient statistic. The existing autoencoder-based fault detection schemes are mainly application-oriented, and few efforts have been devoted to optimal autoencoder-based fault detection and explainable applications. The main objective of our work is to establish a framework for learning autoencoder-based optimal fault detection in nonlinear dynamic systems. To this aim, a process model form for dynamic systems is firstly introduced with the aid of control and system theory, which also leads to a clear system interpretation of the latent variable. The major efforts are devoted to the development of a control theoretical solution to the optimal fault detection problem, in which an analog concept to minimal sufficient statistic, the so-called lossless information compression, is introduced for dynamic systems and fault detection specifications. In particular, the existence conditions for such a latent variable are derived, based on which a loss function and further a learning algorithm are developed. This learning algorithm enables optimally training of autoencoders to achieve an optimal fault detection in nonlinear dynamic systems. A case study on three-tank system is given at the end of this paper to illustrate the capability of the proposed autoencoder-based fault detection and to explain the essential role of the latent variable in the proposed fault detection system

Error tolerant multimedia stream processing: There's plenty of room at the top (of the system stack)

There is a growing realization that the expected fault rates and energy dissipation stemming from increases in CMOS integration will lead to the abandonment of traditional system reliability in favor of approaches that offer reliability to hardware-induced errors across the application, runtime support, architecture, device and integrated-circuit (IC) layers. Commercial stakeholders of multimedia stream processing (MSP) applications, such as information retrieval, stream mining systems, and high-throughput image and video processing systems already feel the strain of inadequate system-level scaling and robustness under the always-increasing user demand. While such applications can tolerate certain imprecision in their results, today's MSP systems do not support a systematic way to exploit this aspect for cross-layer system resilience. However, research is currently emerging that attempts to utilize the error-tolerant nature of MSP applications for this purpose. This is achieved by modifications to all layers of the system stack, from algorithms and software to the architecture and device layer, and even the IC digital logic synthesis itself. Unlike conventional processing that aims for worst-case performance and accuracy guarantees, error-tolerant MSP attempts to provide guarantees for the expected performance and accuracy. In this paper we review recent advances in this field from an MSP and a system (layer-by-layer) perspective, and attempt to foresee some of the components of future cross-layer error-tolerant system design that may influence the multimedia and the general computing landscape within the next ten years. © 1999-2012 IEEE

On-Line Learning and Wavelet-Based Feature Extraction Methodology for Process Monitoring using High-Dimensional Functional Data

The recent advances in information technology, such as the various automatic data acquisition systems and sensor systems, have created tremendous opportunities for collecting valuable process data. The timely processing of such data for meaningful information remains a challenge. In this research, several data mining methodology that will aid information streaming of high-dimensional functional data are developed. For on-line implementations, two weighting functions for updating support vector regression parameters were developed. The functions use parameters that can be easily set a priori with the slightest knowledge of the data involved and have provision for lower and upper bounds for the parameters. The functions are applicable to time series predictions, on-line predictions, and batch predictions. In order to apply these functions for on-line predictions, a new on-line support vector regression algorithm that uses adaptive weighting parameters was presented. The new algorithm uses varying rather than fixed regularization constant and accuracy parameter. The developed algorithm is more robust to the volume of data available for on-line training as well as to the relative position of the available data in the training sequence. The algorithm improves prediction accuracy by reducing uncertainty in using fixed values for the regression parameters. It also improves prediction accuracy by reducing uncertainty in using regression values based on some experts’ knowledge rather than on the characteristics of the incoming training data. The developed functions and algorithm were applied to feedwater flow rate data and two benchmark time series data. The results show that using adaptive regression parameters performs better than using fixed regression parameters. In order to reduce the dimension of data with several hundreds or thousands of predictors and enhance prediction accuracy, a wavelet-based feature extraction procedure called step-down thresholding procedure for identifying and extracting significant features for a single curve was developed. The procedure involves transforming the original spectral into wavelet coefficients. It is based on multiple hypothesis testing approach and it controls family-wise error rate in order to guide against selecting insignificant features without any concern about the amount of noise that may be present in the data. Therefore, the procedure is applicable for data-reduction and/or data-denoising. The procedure was compared to six other data-reduction and data-denoising methods in the literature. The developed procedure is found to consistently perform better than most of the popular methods and performs at the same level with the other methods. Many real-world data with high-dimensional explanatory variables also sometimes have multiple response variables; therefore, the selection of the fewest explanatory variables that show high sensitivity to predicting the response variable(s) and low sensitivity to the noise in the data is important for better performance and reduced computational burden. In order to select the fewest explanatory variables that can predict each of the response variables better, a two-stage wavelet-based feature extraction procedure is proposed. The first stage uses step-down procedure to extract significant features for each of the curves. Then, representative features are selected out of the extracted features for all curves using voting selection strategy. Other selection strategies such as union and intersection were also described and implemented. The essence of the first stage is to reduce the dimension of the data without any consideration for whether or not they can predict the response variables accurately. The second stage uses Bayesian decision theory approach to select some of the extracted wavelet coefficients that can predict each of the response variables accurately. The two stage procedure was implemented using near-infrared spectroscopy data and shaft misalignment data. The results show that the second stage further reduces the dimension and the prediction results are encouraging

Machine Learning Tips and Tricks for Power Line Communications

4openopenTonello A.M.; Letizia N.A.; Righini D.; Marcuzzi F.Tonello, A. M.; Letizia, N. A.; Righini, D.; Marcuzzi, F

Fault detection for switched positive systems under successive packet dropouts with application to the Leslie matrix model

In this paper, the problem of fault detection filter design is dealt with for a class of switched positive systems with packet dropouts on the channel between the sensors and the filters. The phenomena of packet dropouts are governed by a Bernoulli process, and a stochastic switched positive system is established based on the augmented states of the plants and filters. Two criteria are developed to evaluate the performance of the fault detection for the system under investigation. Sufficient conditions are established on the existence of the desired filters for the mean-square stability with an L1 disturbance attenuation level, and an index for the L fault sensitivity is also derived through constructing a switched Lyapunov function in term of linear programming. Two illustrative examples, one of which is concerned with the Leslie matrix model, are provided to show the effectiveness and applicability of the proposed results. Copyright © 2015 John Wiley & Sons, Ltd.National Natural Science Foundation of China under Grants 61104114, 61201035, 61374070, 61473055. Fundamental Research Funds for the Central Universities in China under Grants DUT14QY14, DUT14QY31, and the Natural Science Foundation of Liaoning under Grants L2014026, 2015020075

Generative adversarial networks review in earthquake-related engineering fields

Within seismology, geology, civil and structural engineering, deep learning (DL), especially via generative adversarial networks (GANs), represents an innovative, engaging, and advantageous way to generate reliable synthetic data that represent actual samples' characteristics, providing a handy data augmentation tool. Indeed, in many practical applications, obtaining a significant number of high-quality information is demanding. Data augmentation is generally based on artificial intelligence (AI) and machine learning data-driven models. The DL GAN-based data augmentation approach for generating synthetic seismic signals revolutionized the current data augmentation paradigm. This study delivers a critical state-of-art review, explaining recent research into AI-based GAN synthetic generation of ground motion signals or seismic events, and also with a comprehensive insight into seismic-related geophysical studies. This study may be relevant, especially for the earth and planetary science, geology and seismology, oil and gas exploration, and on the other hand for assessing the seismic response of buildings and infrastructures, seismic detection tasks, and general structural and civil engineering applications. Furthermore, highlighting the strengths and limitations of the current studies on adversarial learning applied to seismology may help to guide research efforts in the next future toward the most promising directions