1,235 research outputs found
Turbo Bayesian Compressed Sensing
Compressed sensing (CS) theory specifies a new signal acquisition approach, potentially allowing the acquisition of signals at a much lower data rate than the Nyquist sampling rate. In CS, the signal is not directly acquired but reconstructed from a few measurements. One of the key problems in CS is how to recover the original signal from measurements in the presence of noise. This dissertation addresses signal reconstruction problems in CS. First, a feedback structure and signal recovery algorithm, orthogonal pruning pursuit (OPP), is proposed to exploit the prior knowledge to reconstruct the signal in the noise-free situation. To handle the noise, a noise-aware signal reconstruction algorithm based on Bayesian Compressed Sensing (BCS) is developed. Moreover, a novel Turbo Bayesian Compressed Sensing (TBCS) algorithm is developed for joint signal reconstruction by exploiting both spatial and temporal redundancy. Then, the TBCS algorithm is applied to a UWB positioning system for achieving mm-accuracy with low sampling rate ADCs. Finally, hardware implementation of BCS signal reconstruction on FPGAs and GPUs is investigated. Implementation on GPUs and FPGAs of parallel Cholesky decomposition, which is a key component of BCS, is explored. Simulation results on software and hardware have demonstrated that OPP and TBCS outperform previous approaches, with UWB positioning accuracy improved by 12.8x. The accelerated computation helps enable real-time application of this work
Training Echo State Networks with Regularization through Dimensionality Reduction
In this paper we introduce a new framework to train an Echo State Network to
predict real valued time-series. The method consists in projecting the output
of the internal layer of the network on a space with lower dimensionality,
before training the output layer to learn the target task. Notably, we enforce
a regularization constraint that leads to better generalization capabilities.
We evaluate the performances of our approach on several benchmark tests, using
different techniques to train the readout of the network, achieving superior
predictive performance when using the proposed framework. Finally, we provide
an insight on the effectiveness of the implemented mechanics through a
visualization of the trajectory in the phase space and relying on the
methodologies of nonlinear time-series analysis. By applying our method on well
known chaotic systems, we provide evidence that the lower dimensional embedding
retains the dynamical properties of the underlying system better than the
full-dimensional internal states of the network
Data-driven discovery of coordinates and governing equations
The discovery of governing equations from scientific data has the potential
to transform data-rich fields that lack well-characterized quantitative
descriptions. Advances in sparse regression are currently enabling the
tractable identification of both the structure and parameters of a nonlinear
dynamical system from data. The resulting models have the fewest terms
necessary to describe the dynamics, balancing model complexity with descriptive
ability, and thus promoting interpretability and generalizability. This
provides an algorithmic approach to Occam's razor for model discovery. However,
this approach fundamentally relies on an effective coordinate system in which
the dynamics have a simple representation. In this work, we design a custom
autoencoder to discover a coordinate transformation into a reduced space where
the dynamics may be sparsely represented. Thus, we simultaneously learn the
governing equations and the associated coordinate system. We demonstrate this
approach on several example high-dimensional dynamical systems with
low-dimensional behavior. The resulting modeling framework combines the
strengths of deep neural networks for flexible representation and sparse
identification of nonlinear dynamics (SINDy) for parsimonious models. It is the
first method of its kind to place the discovery of coordinates and models on an
equal footing.Comment: 25 pages, 6 figures; added acknowledgment
Spatio-temporal learning with the online finite and infinite echo-state Gaussian processes
Successful biological systems adapt to change. In this paper, we are principally concerned with adaptive systems that operate in environments where data arrives sequentially and is multivariate in nature, for example, sensory streams in robotic systems. We contribute two reservoir inspired methods: 1) the online echostate Gaussian process (OESGP) and 2) its infinite variant, the online infinite echostate Gaussian process (OIESGP) Both algorithms are iterative fixed-budget methods that learn from noisy time series. In particular, the OESGP combines the echo-state network with Bayesian online learning for Gaussian processes. Extending this to infinite reservoirs yields the OIESGP, which uses a novel recursive kernel with automatic relevance determination that enables spatial and temporal feature weighting. When fused with stochastic natural gradient descent, the kernel hyperparameters are iteratively adapted to better model the target system. Furthermore, insights into the underlying system can be gleamed from inspection of the resulting hyperparameters. Experiments on noisy benchmark problems (one-step prediction and system identification) demonstrate that our methods yield high accuracies relative to state-of-the-art methods, and standard kernels with sliding windows, particularly on problems with irrelevant dimensions. In addition, we describe two case studies in robotic learning-by-demonstration involving the Nao humanoid robot and the Assistive Robot Transport for Youngsters (ARTY) smart wheelchair
A machine learning route between band mapping and band structure
The electronic band structure (BS) of solid state materials imprints the
multidimensional and multi-valued functional relations between energy and
momenta of periodically confined electrons. Photoemission spectroscopy is a
powerful tool for its comprehensive characterization. A common task in
photoemission band mapping is to recover the underlying quasiparticle
dispersion, which we call band structure reconstruction. Traditional methods
often focus on specific regions of interests yet require extensive human
oversight. To cope with the growing size and scale of photoemission data, we
develop a generic machine-learning approach leveraging the information within
electronic structure calculations for this task. We demonstrate its capability
by reconstructing all fourteen valence bands of tungsten diselenide and
validate the accuracy on various synthetic data. The reconstruction uncovers
previously inaccessible momentum-space structural information on both global
and local scales in conjunction with theory, while realizing a path towards
integrating band mapping data into materials science databases
Three more Decades in Array Signal Processing Research: An Optimization and Structure Exploitation Perspective
The signal processing community currently witnesses the emergence of sensor
array processing and Direction-of-Arrival (DoA) estimation in various modern
applications, such as automotive radar, mobile user and millimeter wave indoor
localization, drone surveillance, as well as in new paradigms, such as joint
sensing and communication in future wireless systems. This trend is further
enhanced by technology leaps and availability of powerful and affordable
multi-antenna hardware platforms. The history of advances in super resolution
DoA estimation techniques is long, starting from the early parametric
multi-source methods such as the computationally expensive maximum likelihood
(ML) techniques to the early subspace-based techniques such as Pisarenko and
MUSIC. Inspired by the seminal review paper Two Decades of Array Signal
Processing Research: The Parametric Approach by Krim and Viberg published in
the IEEE Signal Processing Magazine, we are looking back at another three
decades in Array Signal Processing Research under the classical narrowband
array processing model based on second order statistics. We revisit major
trends in the field and retell the story of array signal processing from a
modern optimization and structure exploitation perspective. In our overview,
through prominent examples, we illustrate how different DoA estimation methods
can be cast as optimization problems with side constraints originating from
prior knowledge regarding the structure of the measurement system. Due to space
limitations, our review of the DoA estimation research in the past three
decades is by no means complete. For didactic reasons, we mainly focus on
developments in the field that easily relate the traditional multi-source
estimation criteria and choose simple illustrative examples.Comment: 16 pages, 8 figures. This work has been submitted to the IEEE for
possible publication. Copyright may be transferred without notice, after
which this version may no longer be accessibl
Empirical Bayesian significance measure of neuronal spike response
Background: Functional connectivity analyses of multiple neurons provide a powerful bottom-up approach to reveal functions of local neuronal circuits by using simultaneous recording of neuronal activity. A statistical methodology, generalized linear modeling (GLM) of the spike response function, is one of the most promising methodologies to reduce false link discoveries arising from pseudo-correlation based on common inputs. Although recent advancement of fluorescent imaging techniques has increased the number of simultaneously recoded neurons up to the hundreds or thousands, the amount of information per pair of neurons has not correspondingly increased, partly because of the instruments' limitations, and partly because the number of neuron pairs increase in a quadratic manner. Consequently, the estimation of GLM suffers from large statistical uncertainty caused by the shortage in effective information. Results: In this study, we propose a new combination of GLM and empirical Bayesian testing for the estimation of spike response functions that enables both conservative false discovery control and powerful functional connectivity detection. We compared our proposed method's performance with those of sparse estimation of GLM and classical Granger causality testing. Our method achieved high detection performance of functional connectivity with conservative estimation of false discovery rate and q values in case of information shortage due to short observation time. We also showed that empirical Bayesian testing on arbitrary statistics in place of likelihood-ratio statistics reduce the computational cost without decreasing the detection performance. When our proposed method was applied to a functional multi-neuron calcium imaging dataset from the rat hippocampal region, we found significant functional connections that are possibly mediated by AMPA and NMDA receptors. Conclusions: The proposed empirical Bayesian testing framework with GLM is promising especially when the amount of information per a neuron pair is small because of growing size of observed network
Model-based Filtering of Interfering Signals in Ultrasonic Time Delay Estimations
This work presents model-based algorithmic approaches for interference-invariant time delay estimation, which are specifically suited for the estimation of small time delay differences with a necessary resolution well below the sampling time. Therefore, the methods can be applied particularly well for transit-time ultrasonic flow measurements, since the problem of interfering signals is especially prominent in this application
- …