A review on analysis and synthesis of nonlinear stochastic systems with randomly occurring incomplete information

Copyright q 2012 Hongli Dong et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.In the context of systems and control, incomplete information refers to a dynamical system in which knowledge about the system states is limited due to the difficulties in modeling complexity in a quantitative way. The well-known types of incomplete information include parameter uncertainties and norm-bounded nonlinearities. Recently, in response to the development of network technologies, the phenomenon of randomly occurring incomplete information has become more and more prevalent. Such a phenomenon typically appears in a networked environment. Examples include, but are not limited to, randomly occurring uncertainties, randomly occurring nonlinearities, randomly occurring saturation, randomly missing measurements and randomly occurring quantization. Randomly occurring incomplete information, if not properly handled, would seriously deteriorate the performance of a control system. In this paper, we aim to survey some recent advances on the analysis and synthesis problems for nonlinear stochastic systems with randomly occurring incomplete information. The developments of the filtering, control and fault detection problems are systematically reviewed. Latest results on analysis and synthesis of nonlinear stochastic systems are discussed in great detail. In addition, various distributed filtering technologies over sensor networks are highlighted. Finally, some concluding remarks are given and some possible future research directions are pointed out. © 2012 Hongli Dong et al.This work was supported in part by the National Natural Science Foundation of China under Grants 61273156, 61134009, 61273201, 61021002, and 61004067, the Engineering and Physical Sciences Research Council (EPSRC) of the UK under Grant GR/S27658/01, the Royal Society of the UK, the National Science Foundation of the USA under Grant No. HRD-1137732, and the Alexander von Humboldt Foundation of German

Measurements, Models, Systems and Design

531 s.

Whole Word Phonetic Displays for Speech Articulation Training

The main objective of this dissertation is to investigate and develop speech recognition technologies for speech training for people with hearing impairments. During the course of this work, a computer aided speech training system for articulation speech training was also designed and implemented. The speech training system places emphasis on displays to improve children\u27s pronunciation of isolated Consonant-Vowel-Consonant (CVC) words, with displays at both the phonetic level and whole word level. This dissertation presents two hybrid methods for combining Hidden Markov Models (HMMs) and Neural Networks (NNs) for speech recognition. The first method uses NN outputs as posterior probability estimators for HMMs. The second method uses NNs to transform the original speech features to normalized features with reduced correlation. Based on experimental testing, both of the hybrid methods give higher accuracy than standard HMM methods. The second method, using the NN to create normalized features, outperforms the first method in terms of accuracy. Several graphical displays were developed to provide real time visual feedback to users, to help them to improve and correct their pronunciations

Resource Management in Computing Systems

Resource management is an essential building block of any modern computer and communication network. In this thesis, the results of our research in the following two tracks are summarized in four papers. The first track includes three papers and covers modeling, prediction and control for multi-tier computing systems. In the first paper, a NARX-based multi-step-ahead response time predictor for single server queuing systems is presented which can be applied to CPU-constrained computing systems. The second paper introduces a NARX-based multi-step-ahead query response time predictor for database servers. Both mentioned predictors can predict the dynamics of response times in the whole operation range particularly in high load scenarios without changes having to be applied to the current protocols and operating systems. In the third paper, queuing theory is used to model the dynamics of a database server. Several heuristics are presented to tune the parameters of the proposed model to the measured data from the database. Furthermore, an admission controller is presented, and its parameters are tuned to control the response time of queries which are sent to the database to stay below a predefined reference value.The second track includes one paper, covering a problem formulation and optimal solution for a content replication problem in Telecom operator's content delivery networks (Telco-CDNs). The problem is formulated in the form of an integer programming problem trying to minimize the communication delay and cost according to several constraints such as limited content replication budget, limited storage size and limited downlink bandwidth of each regional content server. The solution of this problem is a performance bound for any distributed content replication algorithm which addresses the same problem

Differentiable Artificial Reverberation

Artificial reverberation (AR) models play a central role in various audio applications. Therefore, estimating the AR model parameters (ARPs) of a target reverberation is a crucial task. Although a few recent deep-learning-based approaches have shown promising performance, their non-end-to-end training scheme prevents them from fully exploiting the potential of deep neural networks. This motivates to introduce differentiable artificial reverberation (DAR) models which allows loss gradients to be back-propagated end-to-end. However, implementing the AR models with their difference equations "as is" in the deep-learning framework severely bottlenecks the training speed when executed with a parallel processor like GPU due to their infinite impulse response (IIR) components. We tackle this problem by replacing the IIR filters with finite impulse response (FIR) approximations with the frequency-sampling method (FSM). Using the FSM, we implement three DAR models -- differentiable Filtered Velvet Noise (FVN), Advanced Filtered Velvet Noise (AFVN), and Feedback Delay Network (FDN). For each AR model, we train its ARP estimation networks for analysis-synthesis (RIR-to-ARP) and blind estimation (reverberant-speech-to-ARP) task in an end-to-end manner with its DAR model counterpart. Experiment results show that the proposed method achieves consistent performance improvement over the non-end-to-end approaches in both objective metrics and subjective listening test results.Comment: Manuscript submitted to TASL

A comparison of the Kalman filter and recurrent neural networks for state estimation of dynamical systems

The study of dynamical systems is of great interest in many fields, with a wide range of applications. In some cases, these dynamical systems may be affected by noise and the availability of measurements may be limited. State estimations methods which can account for these challenges are valuable tools in analyzing these systems. While for linear systems the standard method is by using an algorithm called the Kalman filter, data-driven methods employing the versatility of artificial neural networks have also been proposed. In this thesis, we first introduce state estimation using the Kalman filter. Next, we provide an overview of a type of artificial neural network called recurrent neural networks (RNNs), which are particularly suited for tasks on time series data. We finally present the results of implementing RNN-based estimators for a number of dynamical systems with comparisons to Kalman filtering

Training Spiking Neural Networks Using Lessons From Deep Learning

The brain is the perfect place to look for inspiration to develop more efficient neural networks. The inner workings of our synapses and neurons provide a glimpse at what the future of deep learning might look like. This paper serves as a tutorial and perspective showing how to apply the lessons learnt from several decades of research in deep learning, gradient descent, backpropagation and neuroscience to biologically plausible spiking neural neural networks. We also explore the delicate interplay between encoding data as spikes and the learning process; the challenges and solutions of applying gradient-based learning to spiking neural networks; the subtle link between temporal backpropagation and spike timing dependent plasticity, and how deep learning might move towards biologically plausible online learning. Some ideas are well accepted and commonly used amongst the neuromorphic engineering community, while others are presented or justified for the first time here. A series of companion interactive tutorials complementary to this paper using our Python package, snnTorch, are also made available: https://snntorch.readthedocs.io/en/latest/tutorials/index.htm