A Robust Analog VLSI Reichardt Motion Sensor

Silicon imagers with integrated motion-detection circuitry have been developed and tested for the past 15 years. Many previous circuits estimate motion by identifying and tracking spatial or temporal features. These approaches are prone to failure at low SNR conditions, where feature detection becomes unreliable. An alternate approach to motion detection is an intensity-based spatiotemporal correlation algorithm, such as the one proposed by Hassenstein and Reichardt in 1956 to explain aspects of insect vision. We implemented a Reichardt motion sensor with integrated photodetectors in a standard CMOS process. Our circuit operates at sub-microwatt power levels, the lowest reported for any motion sensor. We measure the effects of device mismatch on these parallel, analog circuits to show they are suitable for constructing 2-D VLSI arrays. Traditional correlation-based sensors suffer from strong contrast dependence. We introduce a circuit architecture that lessens this dependence. We also demonstrate robust performance of our sensor to complex stimuli in the presence of spatial and temporal noise

On data-selective learning

Adaptive filters are applied in several electronic and communication devices like smartphones, advanced headphones, DSP chips, smart antenna, and teleconference systems. Also, they have application in many areas such as system identification, channel equalization, noise reduction, echo cancellation, interference cancellation, signal prediction, and stock market. Therefore, reducing the energy consumption of the adaptive filtering algorithms has great importance, particularly in green technologies and in devices using battery. In this thesis, data-selective adaptive filters, in particular the set-membership (SM) adaptive filters, are the tools to reach the goal. There are well known SM adaptive filters in literature. This work introduces new algorithms based on the classical ones in order to improve their performances and reduce the number of required arithmetic operations at the same time. Therefore, firstly, we analyze the robustness of the classical SM adaptive filtering algorithms. Secondly, we extend the SM technique to trinion and quaternion systems. Thirdly, by combining SM filtering and partialupdating, we introduce a new improved set-membership affine projection algorithm with constrained step size to improve its stability behavior. Fourthly, we propose some new least-mean-square (LMS) based and recursive least-squares based adaptive filtering algorithms with low computational complexity for sparse systems. Finally, we derive some feature LMS algorithms to exploit the hidden sparsity in the parameters.Filtros adaptativos são aplicados em diversos aparelhos eletrônicos e de comunicação, como smartphones, fone de ouvido avançados, DSP chips, antenas inteligentes e sistemas de teleconferência. Eles também têm aplicação em várias áreas como identificação de sistemas, equalização de canal, cancelamento de eco, cancelamento de interferência, previsão de sinal e mercado de ações. Desse modo, reduzir o consumo de energia de algoritmos adaptativos tem importância significativa, especialmente em tecnologias verdes e aparelhos que usam bateria. Nesta tese, filtros adaptativos com seleção de dados, em particular filtros adaptativos da família set-membership (SM), são apresentados para cumprir essa missão. No presente trabalho objetivamos apresentar novos algoritmos, baseados nos clássicos, a fim de aperfeiçoar seus desempenhos e, ao mesmo tempo, reduzir o número de operações aritméticas exigidas. Dessa forma, primeiro analisamos a robustez dos filtros adaptativos SM clássicos. Segundo, estendemos o SM aos números trinions e quaternions. Terceiro, foram utilizadas também duas famílias de algoritmos, SM filtering e partial-updating, de uma maneira elegante, visando reduzir energia ao máximo possível e obter um desempenho competitivo em termos de estabilidade. Quarto, a tese propõe novos filtros adaptativos baseado em algoritmos least-mean-square (LMS) e mínimos quadrados recursivos com complexidade computacional baixa para espaços esparsos. Finalmente, derivamos alguns algoritmos feature LMS para explorar a esparsidade escondida nos parâmetros

Uses of Complex Wavelets in Deep Convolutional Neural Networks

Image understanding has long been a goal for computer vision. It has proved to be an exceptionally difficult task due to the large amounts of variability that are inherent to objects in a scene. Recent advances in supervised learning methods, particularly convolutional neural networks (CNNs), have pushed forth the frontier of what we have been able to train computers to do. Despite their successes, the mechanics of how these networks are able to recognize objects are little understood, and the networks themselves are often very difficult and time-consuming to train. It is very important that we improve our current approaches in every way possible. A CNN is built from connecting many learned convolutional layers in series. These convolutional layers are fairly crude in terms of signal processing - they are arbitrary taps of a finite impulse response filter, learned through stochastic gradient descent from random initial conditions. We believe that if we reformulate the problem, we may achieve many insights and benefits in training CNNs. Noting that modern CNNs are mostly viewed from and analyzed in the spatial domain, this thesis aims to view the convolutional layers in the frequency domain (viewing things in the frequency domain has proved useful in the past for denoising, filter design, compression and many other tasks). In particular, we use complex wavelets (rather than the Fourier transform or the discrete wavelet transform) as basis functions to reformulate image understanding with deep networks. In this thesis, we explore the most popular and well-developed form of using complex wavelets in deep learning, the ScatterNet from Stephane Mallat. We explore its current limitations by building a DeScatterNet and found that while it has many nice properties, it may not be sensitive to the most appropriate shapes for understanding natural images. We then develop a locally invariant convolutional layer, a combination of a complex wavelet transform, a modulus operation, and a learned mixing. To do this, we derive backpropagation equations and allow gradients to flow back through the (previously fixed) ScatterNet front end. Connecting several such locally invariant layers allows us to build learnable ScatterNet, a more flexible and general form of the ScatterNet (while still maintaining its desired properties). We show that the learnable ScatterNet can provide significant improvements over the regular ScatterNet when being used as a front end for a learning system. Additionally, we show that the locally invariant convolutional layer can directly replace convolutional layers in a deep CNN (and not just at the front-end). The locally invariant convolutional layers naturally downsample the input (because of the complex modulus) while increasing the channel dimension (because of the multiple wavelet orientations used). This is an operation that often happens in a CNN by a combination of a pooling and convolutional layer. It was at these locations in a CNN where the learnable ScatterNet performed best, implying it may be useful as learnable pooling layer. Finally, we develop a system to learn complex weights that act directly on the wavelet coefficients of signals, in place of a convolutional layer. We call this layer the wavelet gain layer and show it can be used alongside convolutional layers. The network designer may then choose to learn in the pixel or wavelet domains. This layer shows a lot of promise and affords more control over what regions of the frequency space we want our layer to learn from. Our experiments show that it can improve on learning in the pixel domain for early layers of a CNN

Deep Graph-Convolutional Image Denoising

Non-local self-similarity is well-known to be an effective prior for the image denoising problem. However, little work has been done to incorporate it in convolutional neural networks, which surpass non-local model-based methods despite only exploiting local information. In this paper, we propose a novel end-to-end trainable neural network architecture employing layers based on graph convolution operations, thereby creating neurons with non-local receptive fields. The graph convolution operation generalizes the classic convolution to arbitrary graphs. In this work, the graph is dynamically computed from similarities among the hidden features of the network, so that the powerful representation learning capabilities of the network are exploited to uncover self-similar patterns. We introduce a lightweight Edge-Conditioned Convolution which addresses vanishing gradient and over-parameterization issues of this particular graph convolution. Extensive experiments show state-of-the-art performance with improved qualitative and quantitative results on both synthetic Gaussian noise and real noise

Designs of Digital Filters and Neural Networks using Firefly Algorithm

Firefly algorithm is an evolutionary algorithm that can be used to solve complex multi-parameter problems in less time. The algorithm was applied to design digital filters of different orders as well as to determine the parameters of complex neural network designs. Digital filters have several applications in the fields of control systems, aerospace, telecommunication, medical equipment and applications, digital appliances, audio recognition processes etc. An Artificial Neural Network (ANN) is an information processing paradigm that is inspired by the way biological nervous systems, such as the brain, processes information and can be simulated using a computer to perform certain specific tasks like clustering, classification, and pattern recognition etc. The results of the designs using Firefly algorithm was compared to the state of the art algorithms and found that the digital filter designs produce results close to the Parks McClellan method which shows the algorithm’s capability of handling complex problems. Also, for the neural network designs, Firefly algorithm was able to efficiently optimize a number of parameter values. The performance of the algorithm was tested by introducing various input noise levels to the training inputs of the neural network designs and it produced the desired output with negligible error in a time-efficient manner. Overall, Firefly algorithm was found to be competitive in solving the complex design optimization problems like other popular optimization algorithms such as Differential Evolution, Particle Swarm Optimization and Genetic Algorithm. It provides a number of adjustable parameters which can be tuned according to the specified problem so that it can be applied to a number of optimization problems and is capable of producing quality results in a reasonable amount of time

Deep Graph-Convolutional Image Denoising

3noNon-local self-similarity is well-known to be an effective prior for the image denoising problem. However, little work has been done to incorporate it in convolutional neural networks, which surpass non-local model-based methods despite only exploiting local information. In this paper, we propose a novel end-to-end trainable neural network architecture employing layers based on graph convolution operations, thereby creating neurons with non-local receptive fields. The graph convolution operation generalizes the classic convolution to arbitrary graphs. In this work, the graph is dynamically computed from similarities among the hidden features of the network, so that the powerful representation learning capabilities of the network are exploited to uncover self-similar patterns. We introduce a lightweight Edge-Conditioned Convolution which addresses vanishing gradient and over-parameterization issues of this particular graph convolution. Extensive experiments show state-of-the-art performance with improved qualitative and quantitative results on both synthetic Gaussian noise and real noise.partially_openopenValsesia D.; Fracastoro G.; Magli E.Valsesia, D.; Fracastoro, G.; Magli, E

Advanced array processing techniques and systems

Research and development on smart antennas, which are recognized as a promising technique to improve the performance of mobile communications, have been extensive in the recent years. Smart antennas combine multiple antenna elements with a signal processing capability in both space and time to optimize its radiation and reception pattern automatically in response to the signal environment. This paper concentrates on the signal processing aspects of smart antenna systems. Smart antennas are often classified as either switched-beam or adaptive-array systems, for which a variety of algorithms have been developed to enhance the signal of interest and reject the interference. The antenna systems need to differentiate the desired signal from the interference, and normally requires either a priori knowledge or the signal direction to achieve its goal. There exists a variety of methods for direction of arrival (DOA) estimation with conflicting demands of accuracy and computation. Similarly, there are many algorithms to compute array weights to direct the maximum radiation of the array pattern toward the signal and place nulls toward the interference, each with its convergence property and computational complexity. This paper discusses some of the typical algorithms for DOA estimation and beamforming. The concept and details of each algorithm are provided. Smart antennas can significantly help in improving the performance of communication systems by increasing channel capacity and spectrum efficiency, extending range coverage, multiplexing channels with spatial division multiple access (SDMA), and compensating electronically for aperture distortion. They also reduce delay spread, multipath fading, co-channel interference, system complexity, bit error rates, and outage probability. In addition, smart antennas can locate mobile units or assist the location determination through DOA and range estimation. This capability can support and benefit many location-based services including emergency assistance, tracking services, safety services, billing services, and information services such as navigation, weather, traffic, and directory assistance

On Deep Machine Learning Based Techniques for Electric Power Systems

This thesis provides deep machine learning-based solutions to real-time mitigation of power quality disturbances such as flicker, voltage dips, frequency deviations, harmonics, and interharmonics using active power filters (APF). In an APF the processing delays reduce the performance when the disturbance to be mitigated is tima varying. The the delays originate from software (response time delay) and hardware (reaction time delay). To reduce the response time delays of APFs, this thesis propose and investigate several different techniques. First a technique based on multiple synchronous reference frame (MSRF) and order-optimized exponential smoothing (ES) to decrease the settling time delay of lowpass filtering steps. To reduce the computational time, this method is implemented in a parallel processing using a graphics processing unit (GPU) to estimate the time-varying harmonics and interharmonics of currents. Furthermore, the MSRF and three machine learning-based solutions are developed to predict future values of voltage and current in electric power systems which can mitigate the effects of the response and reaction time delays of the APFs. In the first and second solutions, a Butterworth filter is used to lowpass filter the\ua0 dq\ua0 components, and linear prediction and long short-term memory (LSTM) are used to predict the filtered\ua0 dq\ua0 components. The third solution is an end-to-end ML-based method developed based on a combination of convolutional neural networks (CNN) and LSTM. The Simulink implementation of the proposed ML-based APF is carried out to compensate for the current waveform harmonics, voltage dips, and flicker in Simulink environment embedded AI computing system Jetson TX2.\ua0In another study, we propose Deep Deterministic Policy Gradient (DDPG), a reinforcement learning (RL) method to replace the controller loops and estimation blocks such as PID, MSRF, and lowpass filters in grid-forming inverters. In a conventional approach it is well recognized that the controller tuning in the differen loops are difficult as the tuning of one loop influence the performance in other parts due to interdependencies.In DDPG the control policy is derived by optimizing a reward function which measure the performance in a data-driven fashion based on extensive experiments of the inverter in a simulation environment.\ua0Compared to a PID-based control architecture, the DDPG derived control policy leads to a solution where the response and reaction time delays are decreased by a factor of five in the investigated example.\ua0Classification of voltage dips originating from cable faults is another topic addressed in this thesis work. The Root Mean Square (RMS) of the voltage dips is proposed as preprocessing step to ease the feature learning for the developed\ua0 LSTM based classifier. Once a cable faults occur, it need to be located and repaired/replaced in order to restore the grid operation. Due to the high importance of stability in the power generation of renewable energy sources, we aim to locate high impedance cable faults in DC microgrid clusters which is a challenging case among different types of faults. The developed Support Vector Machine (SVM) algorithm process the maximum amplitude and\ua0 di/dt\ua0 of the current waveform of the fault as features, and the localization task is carried out with\ua0 95 %\ua0 accuracy.\ua0Two ML-based solutions together with a two-step feature engineering method are proposed to classify Partial Discharges (PD) originating from pulse width modulation (PWM) excitation in high voltage power electronic devices. As a first step, maximum amplitude, time of occurrence, area under PD curve, and time distance of each PD are extracted as features of interest. The extracted features are concatenated to form patterns for the ML algorithms as a second step. The suggested feature classification using the proposed ML algorithms resulted in\ua0 95.5 %\ua0 and\ua0 98.3 %\ua0\ua0 accuracy on a test data set using ensemble bagged decision trees and LSTM networks