855 research outputs found
Unitary Approximate Message Passing for Sparse Bayesian Learning and Bilinear Recovery
Over the past several years, the approximate message passing (AMP) algorithm has been applied to a broad range of problems, including compressed sensing (CS), robust regression, Bayesian estimation, etc. AMP was originally developed for compressed sensing based on the loopy belief propagation (BP). Compared to convex optimization based algorithms, AMP has low complexity and its performance can be rigorously characterized by a scalar state evolution (SE) in the case of a large independent and identically distributed (i.i.d.) (sub-) Gaussian matrix. AMP was then extended to solve general estimation problems with a generalized linear observation model. However, AMP performs poorly on a generic matrix such as non-zero mean, rank-deficient, correlated, or ill-conditioned matrix, resulting in divergence and degraded performance. It was discovered later that applying AMP to a unitary transform of the original model can remarkably enhance the robustness to difficult matrices. This variant is named unitary AMP (UAMP), or formally called UTAMP. In this thesis, leveraging UAMP, we propose UAMP-SBL for sparse signal recovery and Bi-UAMP for bilinear recovery, both of which inherit the low complexity and robustness of UAMP.
Sparse Bayesian learning (SBL) is a powerful tool for recovering a sparse signal from noisy measurements, which finds numerous applications in various areas. As a traditional implementation of SBL, e.g., Tipping’s method, involves matrix inversion in each iteration, the computational complexity can be prohibitive for large scale problems. To circumvent this, AMP and its variants have been used as low-complexity solutions. Unfortunately, they will diverge for ‘difficult’ measurement matrices as previously mentioned. In this thesis, leveraging UAMP, a novel SBL algorithm called UAMP-SBL is proposed where UAMP is incorporated into the structured variational message passing (SVMP) to handle the most computationally intensive part of message computations. It is shown that, compared to state-of-the-art AMP based SBL algorithms, the proposed UAMP-SBL is more robust and efficient, leading to remarkably better performance.
The bilinear recovery problem has many applications such as dictionary learning, selfcalibration, compressed sensing with matrix uncertainty, etc. Compared to existing nonmessage passing alternates, several AMP based algorithms have been developed to solve bilinear problems. By using UAMP, a more robust and faster approximate inference algorithm for bilinear recovery is proposed in this thesis, which is called Bi-UAMP. With the lifting approach, the original bilinear problem is reformulated as a linear one. Then, variational inference (VI), expectation propagation (EP) and BP are combined with UAMP to implement the approximate inference algorithm Bi-UAMP, where UAMP is adopted for the most computationally intensive part. It is shown that, compared to state-of-the-art bilinear recovery algorithms, the proposed Bi-UAMP is much more robust and faster, and delivers significantly better performance.
Recently, UAMP has also been employed for many other applications such as inverse synthetic aperture radar (ISAR) imaging, low-complexity direction of arrival (DOA) estimation, iterative detection for orthogonal time frequency space modulation (OTFS), channel estimation for RIS-Aided MIMO communications, etc. Promising performance was achieved in all of the applications, and more applications of UAMP are expected in the future
Time-varying Learning and Content Analytics via Sparse Factor Analysis
We propose SPARFA-Trace, a new machine learning-based framework for
time-varying learning and content analytics for education applications. We
develop a novel message passing-based, blind, approximate Kalman filter for
sparse factor analysis (SPARFA), that jointly (i) traces learner concept
knowledge over time, (ii) analyzes learner concept knowledge state transitions
(induced by interacting with learning resources, such as textbook sections,
lecture videos, etc, or the forgetting effect), and (iii) estimates the content
organization and intrinsic difficulty of the assessment questions. These
quantities are estimated solely from binary-valued (correct/incorrect) graded
learner response data and a summary of the specific actions each learner
performs (e.g., answering a question or studying a learning resource) at each
time instance. Experimental results on two online course datasets demonstrate
that SPARFA-Trace is capable of tracing each learner's concept knowledge
evolution over time, as well as analyzing the quality and content organization
of learning resources, the question-concept associations, and the question
intrinsic difficulties. Moreover, we show that SPARFA-Trace achieves comparable
or better performance in predicting unobserved learner responses than existing
collaborative filtering and knowledge tracing approaches for personalized
education
Matrix Factorization Based Blind Bayesian Receiver for Grant-Free Random Access in mmWave MIMO mMTC
Grant-free random access is promising for massive connectivity with sporadic
transmissions in massive machine type communications (mMTC), where the
hand-shaking between the access point (AP) and users is skipped, leading to
high access efficiency. In grant-free random access, the AP needs to identify
the active users and perform channel estimation and signal detection.
Conventionally, pilot signals are required for the AP to achieve user activity
detection and channel estimation before active user signal detection, which may
still result in substantial overhead and latency. In this paper, to further
reduce the overhead and latency, we explore the problem of grant-free random
access without the use of pilot signals in a millimeter wave (mmWave) multiple
input and multiple output (MIMO) system, where the AP performs blind joint user
activity detection, channel estimation and signal detection (UACESD). We show
that the blind joint UACESD can be formulated as a constrained composite matrix
factorization problem, which can be solved by exploiting the structures of the
channel matrix and signal matrix. Leveraging our recently developed unitary
approximate message passing based matrix factorization (UAMP-MF) algorithm, we
design a message passing based Bayesian algorithm to solve the blind joint
UACESD problem. Extensive simulation results demonstrate the effectiveness of
the blind grant-free random access scheme
S-AMP for Non-linear Observation Models
Recently we extended Approximate message passing (AMP) algorithm to be able
to handle general invariant matrix ensembles. In this contribution we extend
our S-AMP approach to non-linear observation models. We obtain generalized AMP
(GAMP) algorithm as the special case when the measurement matrix has zero-mean
iid Gaussian entries. Our derivation is based upon 1) deriving expectation
propagation (EP) like algorithms from the stationary-points equations of the
Gibbs free energy under first- and second-moment constraints and 2) applying
additive free convolution in free probability theory to get low-complexity
updates for the second moment quantities.Comment: 6 page
Signal Detection in MIMO Systems with Hardware Imperfections: Message Passing on Neural Networks
In this paper, we investigate signal detection in
multiple-input-multiple-output (MIMO) communication systems with hardware
impairments, such as power amplifier nonlinearity and in-phase/quadrature
imbalance. To deal with the complex combined effects of hardware imperfections,
neural network (NN) techniques, in particular deep neural networks (DNNs), have
been studied to directly compensate for the impact of hardware impairments.
However, it is difficult to train a DNN with limited pilot signals, hindering
its practical applications. In this work, we investigate how to achieve
efficient Bayesian signal detection in MIMO systems with hardware
imperfections. Characterizing combined hardware imperfections often leads to
complicated signal models, making Bayesian signal detection challenging. To
address this issue, we first train an NN to "model" the MIMO system with
hardware imperfections and then perform Bayesian inference based on the trained
NN. Modelling the MIMO system with NN enables the design of NN architectures
based on the signal flow of the MIMO system, minimizing the number of NN layers
and parameters, which is crucial to achieving efficient training with limited
pilot signals. We then represent the trained NN with a factor graph, and design
an efficient message passing based Bayesian signal detector, leveraging the
unitary approximate message passing (UAMP) algorithm. The implementation of a
turbo receiver with the proposed Bayesian detector is also investigated.
Extensive simulation results demonstrate that the proposed technique delivers
remarkably better performance than state-of-the-art methods
- …