High-Performance FPGA Implementation of Equivariant Adaptive Separation via Independence Algorithm for Independent Component Analysis

Independent Component Analysis (ICA) is a dimensionality reduction technique that can boost efficiency of machine learning models that deal with probability density functions, e.g. Bayesian neural networks. Algorithms that implement adaptive ICA converge slower than their nonadaptive counterparts, however, they are capable of tracking changes in underlying distributions of input features. This intrinsically slow convergence of adaptive methods combined with existing hardware implementations that operate at very low clock frequencies necessitate fundamental improvements in both algorithm and hardware design. This paper presents an algorithm that allows efficient hardware implementation of ICA. Compared to previous work, our FPGA implementation of adaptive ICA improves clock frequency by at least one order of magnitude and throughput by at least two orders of magnitude. Our proposed algorithm is not limited to ICA and can be used in various machine learning problems that use stochastic gradient descent optimization

A Hardware-Friendly Algorithm for Scalable Training and Deployment of Dimensionality Reduction Models on FPGA

With ever-increasing application of machine learning models in various domains such as image classification, speech recognition and synthesis, and health care, designing efficient hardware for these models has gained a lot of popularity. While the majority of researches in this area focus on efficient deployment of machine learning models (a.k.a inference), this work concentrates on challenges of training these models in hardware. In particular, this paper presents a high-performance, scalable, reconfigurable solution for both training and deployment of different dimensionality reduction models in hardware by introducing a hardware-friendly algorithm. Compared to state-of-the-art implementations, our proposed algorithm and its hardware realization decrease resource consumption by 50\% without any degradation in accuracy

Variable learning rate EASI-based adaptive blind source separation in situation of nonstationary source and linear time-varying systems

In the case of multiple nonstationary independent source signals and linear instantaneous time-varying mixing systems, it is difficult to adaptively separate the multiple source signals. Therefore, the adaptive blind source separation (BSS) problem is firstly formally expressed and compared with tradition BSS problem. Then, we propose an adaptive blind identification and separation method based on the variable learning rate equivariant adaptive source separation via independence (EASI) algorithm. Furthermore, we analyze the scope and conditions of variable-learning rate EASI algorithm. The adaptive BSS simulation results also show that the variable learning rate EASI algorithm provides better separation effect than the fixed learning rate EASI and recursive least-squares algorithms

FFT-Based Deep Learning Deployment in Embedded Systems

Deep learning has delivered its powerfulness in many application domains, especially in image and speech recognition. As the backbone of deep learning, deep neural networks (DNNs) consist of multiple layers of various types with hundreds to thousands of neurons. Embedded platforms are now becoming essential for deep learning deployment due to their portability, versatility, and energy efficiency. The large model size of DNNs, while providing excellent accuracy, also burdens the embedded platforms with intensive computation and storage. Researchers have investigated on reducing DNN model size with negligible accuracy loss. This work proposes a Fast Fourier Transform (FFT)-based DNN training and inference model suitable for embedded platforms with reduced asymptotic complexity of both computation and storage, making our approach distinguished from existing approaches. We develop the training and inference algorithms based on FFT as the computing kernel and deploy the FFT-based inference model on embedded platforms achieving extraordinary processing speed.Comment: Design, Automation, and Test in Europe (DATE) For source code, please contact Mahdi Nazemi at <[email protected]

Hybrid solutions to instantaneous MIMO blind separation and decoding: narrowband, QAM and square cases

Future wireless communication systems are desired to support high data rates and high quality transmission when considering the growing multimedia applications. Increasing the channel throughput leads to the multiple input and multiple output and blind equalization techniques in recent years. Thereby blind MIMO equalization has attracted a great interest.Both system performance and computational complexities play important roles in real time communications. Reducing the computational load and providing accurate performances are the main challenges in present systems. In this thesis, a hybrid method which can provide an affordable complexity with good performance for Blind Equalization in large constellation MIMO systems is proposed first. Saving computational cost happens both in the signal sep- aration part and in signal detection part. First, based on Quadrature amplitude modulation signal characteristics, an efficient and simple nonlinear function for the Independent Compo- nent Analysis is introduced. Second, using the idea of the sphere decoding, we choose the soft information of channels in a sphere, and overcome the so- called curse of dimensionality of the Expectation Maximization (EM) algorithm and enhance the final results simultaneously. Mathematically, we demonstrate in the digital communication cases, the EM algorithm shows Newton -like convergence.Despite the widespread use of forward -error coding (FEC), most multiple input multiple output (MIMO) blind channel estimation techniques ignore its presence, and instead make the sim- plifying assumption that the transmitted symbols are uncoded. However, FEC induces code structure in the transmitted sequence that can be exploited to improve blind MIMO channel estimates. In final part of this work, we exploit the iterative channel estimation and decoding performance for blind MIMO equalization. Experiments show the improvements achievable by exploiting the existence of coding structures and that it can access the performance of a BCJR equalizer with perfect channel information in a reasonable SNR range. All results are confirmed experimentally for the example of blind equalization in block fading MIMO systems

Development of Novel Independent Component Analysis Techniques and their Applications

Real world problems very often provide minimum information regarding their causes. This is mainly due to the system complexities and noninvasive techniques employed by scientists and engineers to study such systems. Signal and image processing techniques used for analyzing such systems essentially tend to be blind. Earlier, training signal based techniques were used extensively for such analyses. But many times either these training signals are not practicable to be availed by the analyzer or become burden on the system itself. Hence blind signal/image processing techniques are becoming predominant in modern real time systems. In fact, blind signal processing has become a very important topic of research and development in many areas, especially biomedical engineering, medical imaging, speech enhancement, remote sensing, communication systems, exploration seismology, geophysics, econometrics, data mining, sensor networks etc. Blind Signal Processing has three major areas: Blind Signal Separation and Extraction, Independent Component Analysis (ICA) and Multichannel Blind Deconvolution and Equalization. ICA technique has also been typically applied to the other two areas mentioned above. Hence ICA research with its wide range of applications is quite interesting and has been taken up as the central domain of the present work

FGPA Implementation of Low-Complexity ICA Based Blind Multiple-Input-Multiple-Output OFDM Receivers

In this thesis Independent Component Analysis (ICA) based methods are used for blind detection in MIMO systems. ICA relies on higher order statistics (HOS) to recover the transmitted streams from the received mixture. Blind separation of the mixture is achieved based on the assumption of mutual statistical independence of the source streams. The use of HOS makes ICA methods less sensitive to Gaussian noise. ICA increase the spectral efficiency compared to conventional systems, without any training/pilot data required. ICA is usually used for blind source separation (BSS) from their mixtures by measuring non-Gaussianity using Kurtosis. Many scientific problems require FP arithmetic with high precision in their calculations. Moreover a large dynamic range of numbers is necessary for signal processing. FP arithmetic has the ability to automatically scale numbers and allows numbers to be represented in a wider range than fixed-point arithmetic. Nevertheless, FP algorithm is difficult to implement on the FPGA, because the algorithm is so complex that the area (logic elements) of FPGA leads to excessive consumption when implemented. A simplified 32-bit FP implementation includes adder, Subtractor, multiplier, divider, and square rooter The FPGA design is based on a hierarchical concept, and the experimental results of the design are presented

Research on performance enhancement for electromagnetic analysis and power analysis in cryptographic LSI

制度:新 ; 報告番号:甲3785号 ; 学位の種類:博士(工学) ; 授与年月日:2012/11/19 ; 早大学位記番号:新6161Waseda Universit

Time-shared channel identification for adaptive noise cancellation in breath sound extraction

Abstract: Noise artifacts are one of the key obstacles in applying continuous monitoring and conrputer-assisted analysis of lung sounds. Traditional adaptive noise cancellation (ANC) methodologies work reasonably well when signal and noise are stationary and independent. Clinical lung sound auscultation encounters an acoustic environment in which breath sounds are not stationary and often correlate with noise. Consequently, capability of ANC becomes significantly compromised. This paper introduces a new methodology for extracting authentic lung sounds from noise-corrupted measurements. Unlike traditional noise cancellation methods that rely on either frequency band separation or sig3M/noise independence to achieve noise reduction, this methodology combines the traditional noise canceling n{ethods with the unique feature of time-split stages in breathing sounds. By employing a multi-sensor system, the method first employs a high-pass filter to elhninate the off-hand noise, and then performs time-shared blind identification and noise cancellation with recursion from breathing cycle to cycle. Since no frequency separation or signal/noise independence is required, this method potentially has a robust and reliable capability of noise reduction, complementing the traditional methods