339 research outputs found
Learning Models For Corrupted Multi-Dimensional Data: Fundamental Limits And Algorithms
Developing machine learning models for unstructured multi-dimensional datasets such as datasets with unreliable labels and noisy multi-dimensional signals with or without missing information have becoming a central necessity. We are not always fortunate enough to get noise-free datasets for developing classification and representation models. Though there is a number of techniques available to deal with noisy datasets, these methods do not exploit the multi-dimensional structures of the signals, which could be used to improve the overall classification and representation performance of the model.
In this thesis, we develop a Kronecker-structure (K-S) subspace model that exploits the multi-dimensional structure of the signal. First, we study the classification performance of K-S subspace models in two asymptotic regimes when the signal dimensions go to infinity and when the noise power tends to zero. We characterize the misclassification probability in terms of diversity order and we drive an exact expression for the diversity order. We further derive a tighter bound on misclassification probability in terms of pairwise geometry of the subspaces. The proposed scheme is optimal in most of the signal dimension regimes except in one regime where the signal dimension is less than twice the subspace dimension, however, hitting such a signal dimension regime is very rare in practice. We empirically show that the classification performance of K-S subspace models agrees with the diversity order analysis. We also develop an algorithm, Kronecker- Structured Learning of Discriminative Dictionaries (K-SLD2), for fast and compact K-S subspace learning for better classification and representation of multidimensional signals. We show that the K-SLD2 algorithm balances compact signal representation and good classification performance on synthetic and real-world datasets. Next, we develop a scheme to detect whether a given multi-dimensional signal with missing information lies on a given K-S subspace. We find that under some mild incoherence conditions we must observe ��(��1 log ��1) number of rows and ��(��2 log ��2) number of columns in order to detect the K-S subspace.
In order to account for unreliable labels in datasets we present Nonlinear, Noise- aware, Quasiclustering (NNAQC), a method for learning deep convolutional networks from datasets corrupted by unknown label noise. We append a nonlinear noise model to a standard convolutional network, which is learned in tandem with the parameters of the network. Further, we train the network using a loss function that encourages the clustering of training images. We argue that the non-linear noise model, while not rigorous as a probabilistic model, results in a more effective denoising operator during backpropagation. We evaluate the performance of NNAQC on artificially injected label noise to MNIST, CIFAR-10, CIFAR-100, and ImageNet datasets and on a large-scale Clothing1M dataset with inherent label noise. We show that on all these datasets, NNAQC provides significantly improved classification performance over the state of the art and is robust to the amount of label noise and the training samples
Geometric Expression Invariant 3D Face Recognition using Statistical Discriminant Models
Currently there is no complete face recognition system that is invariant to all facial expressions.
Although humans find it easy to identify and recognise faces regardless of changes in illumination,
pose and expression, producing a computer system with a similar capability has proved to
be particularly di cult. Three dimensional face models are geometric in nature and therefore
have the advantage of being invariant to head pose and lighting. However they are still susceptible
to facial expressions. This can be seen in the decrease in the recognition results using
principal component analysis when expressions are added to a data set.
In order to achieve expression-invariant face recognition systems, we have employed a tensor
algebra framework to represent 3D face data with facial expressions in a parsimonious
space. Face variation factors are organised in particular subject and facial expression modes.
We manipulate this using single value decomposition on sub-tensors representing one variation
mode. This framework possesses the ability to deal with the shortcomings of PCA in less constrained
environments and still preserves the integrity of the 3D data. The results show improved
recognition rates for faces and facial expressions, even recognising high intensity expressions
that are not in the training datasets.
We have determined, experimentally, a set of anatomical landmarks that best describe facial
expression e ectively. We found that the best placement of landmarks to distinguish di erent
facial expressions are in areas around the prominent features, such as the cheeks and eyebrows.
Recognition results using landmark-based face recognition could be improved with better placement.
We looked into the possibility of achieving expression-invariant face recognition by reconstructing
and manipulating realistic facial expressions. We proposed a tensor-based statistical
discriminant analysis method to reconstruct facial expressions and in particular to neutralise
facial expressions. The results of the synthesised facial expressions are visually more realistic
than facial expressions generated using conventional active shape modelling (ASM). We
then used reconstructed neutral faces in the sub-tensor framework for recognition purposes.
The recognition results showed slight improvement. Besides biometric recognition, this novel
tensor-based synthesis approach could be used in computer games and real-time animation
applications
High Dimensional Covariance Estimation for Spatio-Temporal Processes
High dimensional time series and array-valued data are ubiquitous in signal processing, machine learning, and science. Due to the additional (temporal) direction, the total dimensionality of the data is often extremely high, requiring large numbers of training examples to learn the distribution using unstructured techniques. However, due to difficulties in sampling, small population sizes, and/or rapid system changes in time, it is often the case that very few relevant training samples are available, necessitating the imposition of structure on the data if learning is to be done. The mean and covariance are useful tools to describe high dimensional distributions because (via the Gaussian likelihood function) they are a data-efficient way to describe a general multivariate distribution, and allow for simple inference, prediction, and regression via classical techniques.
In this work, we develop various forms of multidimensional covariance structure that explicitly exploit the array structure of the data, in a way analogous to the widely used low rank modeling of the mean. This allows dramatic reductions in the number of training samples required, in some cases to a single training sample. Covariance models of this form have been increasing in interest recently, and statistical performance bounds for high dimensional estimation in sample-starved scenarios are of great relevance.
This thesis focuses on the high-dimensional covariance estimation problem, exploiting spatio-temporal structure to reduce sample complexity. Contributions are made in the following areas: (1) development of a variety of rich Kronecker product-based covariance models allowing the exploitation of spatio-temporal and other structure with applications to sample-starved real data problems, (2) strong performance bounds for high-dimensional estimation of covariances under each model, and (3) a strongly adaptive online method for estimating changing optimal low-dimensional metrics (inverse covariances) for high-dimensional data from a series of similarity labels.PHDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/137082/1/greenewk_1.pd
Linear Transmit-Receive Strategies for Multi-user MIMO Wireless Communications
Die Notwendigkeit zur Unterdrueckung von Interferenzen auf der einen Seite
und zur Ausnutzung der durch Mehrfachzugriffsverfahren erzielbaren Gewinne
auf der anderen Seite rueckte die raeumlichen Mehrfachzugriffsverfahren
(Space Division Multiple Access, SDMA) in den Fokus der Forschung. Ein
Vertreter der raeumlichen Mehrfachzugriffsverfahren, die lineare
Vorkodierung, fand aufgrund steigender Anzahl an Nutzern und Antennen in
heutigen und zukuenftigen Mobilkommunikationssystemen besondere Beachtung,
da diese Verfahren das Design von Algorithmen zur Vorcodierung
vereinfachen. Aus diesem Grund leistet diese Dissertation einen Beitrag zur
Entwicklung linearer Sende- und Empfangstechniken fuer MIMO-Technologie mit
mehreren Nutzern. Zunaechst stellen wir ein Framework zur Approximation des
Datendurchsatzes in Broadcast-MIMO-Kanaelen mit mehreren Nutzern vor. In
diesem Framework nehmen wir das lineare Vorkodierverfahren regularisierte
Blockdiagonalisierung (RBD) an. Durch den Vergleich von Dirty Paper Coding
(DPC) und linearen Vorkodieralgorithmen (z.B. Zero Forcing (ZF) und
Blockdiagonalisierung (BD)) ist es uns moeglich, untere und obere Schranken
fuer den Unterschied bezueglich Datenraten und bezueglich Leistung zwischen
beiden anzugeben. Im Weiteren entwickeln wir einen Algorithmus fuer
koordiniertes Beamforming (Coordinated Beamforming, CBF), dessen Loesung
sich in geschlossener Form angeben laesst. Dieser CBF-Algorithmus basiert
auf der SeDJoCo-Transformation und loest bisher vorhandene Probleme im
Bereich CBF. Im Anschluss schlagen wir einen iterativen CBF-Algorithmus
namens FlexCoBF (flexible coordinated beamforming) fuer
MIMO-Broadcast-Kanaele mit mehreren Nutzern vor. Im Vergleich mit bis dato
existierenden iterativen CBF-Algorithmen kann als vielversprechendster
Vorteil die freie Wahl der linearen Sende- und Empfangsstrategie
herausgestellt werden. Das heisst, jede existierende Methode der linearen
Vorkodierung kann als Sendestrategie genutzt werden, waehrend die Strategie
zum Empfangsbeamforming frei aus MRC oder MMSE gewaehlt werden darf. Im
Hinblick auf Szenarien, in denen Mobilfunkzellen in Clustern
zusammengefasst sind, erweitern wir FlexCoBF noch weiter. Hier wurde das
Konzept der koordinierten Mehrpunktverbindung (Coordinated Multipoint
(CoMP) transmission) integriert. Zuletzt stellen wir drei Moeglichkeiten
vor, Kanalzustandsinformationen (Channel State Information, CSI) unter
verschiedenen Kanalumstaenden zu erlangen. Die Qualitaet der
Kanalzustandsinformationen hat einen starken Einfluss auf die Guete des
Uebertragungssystems. Die durch unsere neuen Algorithmen erzielten
Verbesserungen haben wir mittels numerischer Simulationen von Summenraten
und Bitfehlerraten belegt.In order to combat interference and exploit large multiplexing gains of the
multi-antenna systems, a particular interest in spatial division multiple
access (SDMA) techniques has emerged. Linear precoding techniques, as one
of the SDMA strategies, have obtained more attention due to the fact that
an increasing number of users and antennas involved into the existing and
future mobile communication systems requires a simplification of the
precoding design. Therefore, this thesis contributes to the design of
linear transmit and receive strategies for multi-user MIMO broadcast
channels in a single cell and clustered multiple cells. First, we present a
throughput approximation framework for multi-user MIMO broadcast channels
employing regularized block diagonalization (RBD) linear precoding.
Comparing dirty paper coding (DPC) and linear precoding algorithms (e.g.,
zero forcing (ZF) and block diagonalization (BD)), we further quantify
lower and upper bounds of the rate and power offset between them as a
function of the system parameters such as the number of users and antennas.
Next, we develop a novel closed-form coordinated beamforming (CBF)
algorithm (i.e., SeDJoCo based closed-form CBF) to solve the existing open
problem of CBF. Our new algorithm can support a MIMO system with an
arbitrary number of users and transmit antennas. Moreover, the application
of our new algorithm is not only for CBF, but also for blind source
separation (BSS), since the same mathematical model has been used in BSS
application.Then, we further propose a new iterative CBF algorithm (i.e.,
flexible coordinated beamforming (FlexCoBF)) for multi-user MIMO broadcast
channels. Compared to the existing iterative CBF algorithms, the most
promising advantage of our new algorithm is that it provides freedom in the
choice of the linear transmit and receive beamforming strategies, i.e., any
existing linear precoding method can be chosen as the transmit strategy and
the receive beamforming strategy can be flexibly chosen from MRC or MMSE
receivers. Considering clustered multiple cell scenarios, we extend the
FlexCoBF algorithm further and introduce the concept of the coordinated
multipoint (CoMP) transmission. Finally, we present three strategies for
channel state information (CSI) acquisition regarding various channel
conditions and channel estimation strategies. The CSI knowledge is required
at the base station in order to implement SDMA techniques. The quality of
the obtained CSI heavily affects the system performance. The performance
enhancement achieved by our new strategies has been demonstrated by
numerical simulation results in terms of the system sum rate and the bit
error rate
- …