Sliced Wasserstein Distance for Learning Gaussian Mixture Models

Gaussian mixture models (GMM) are powerful parametric tools with many applications in machine learning and computer vision. Expectation maximization (EM) is the most popular algorithm for estimating the GMM parameters. However, EM guarantees only convergence to a stationary point of the log-likelihood function, which could be arbitrarily worse than the optimal solution. Inspired by the relationship between the negative log-likelihood function and the Kullback-Leibler (KL) divergence, we propose an alternative formulation for estimating the GMM parameters using the sliced Wasserstein distance, which gives rise to a new algorithm. Specifically, we propose minimizing the sliced-Wasserstein distance between the mixture model and the data distribution with respect to the GMM parameters. In contrast to the KL-divergence, the energy landscape for the sliced-Wasserstein distance is more well-behaved and therefore more suitable for a stochastic gradient descent scheme to obtain the optimal GMM parameters. We show that our formulation results in parameter estimates that are more robust to random initializations and demonstrate that it can estimate high-dimensional data distributions more faithfully than the EM algorithm

Acoustic tomography of the atmosphere using iterated Unscented Kalman Filter

2012 Fall.Includes bibliographical references.Tomography approaches are of great interests because of their non-intrusive nature and their ability to generate a significantly larger amount of data in comparison to the in-situ measurement method. Acoustic tomography is an approach which reconstructs the unknown parameters that affect the propagation of acoustic rays in a field of interest by studying the temporal characteristics of the propagation. Acoustic tomography has been used in several different disciplines such as biomedical imaging, oceanographic studies and atmospheric studies. The focus of this thesis is to study acoustic tomography of the atmosphere in order to reconstruct the temperature and wind velocity fields in the atmospheric surface layer using the travel-times collected from several pairs of transmitter and receiver sensors distributed in the field. Our work consists of three main parts. The first part of this thesis is dedicated to reviewing the existing methods for acoustic tomography of the atmosphere, namely statistical inversion (SI), time dependent statistical inversion (TDSI), simultaneous iterative reconstruction technique (SIRT), and sparse recovery framework. The properties of these methods are then explained extensively and their shortcomings are also mentioned. In the second part of this thesis, a new acoustic tomography method based on Unscented Kalman Filter (UKF) is introduced in order to address some of the shortcomings of the existing methods. Using the UKF, the problem is cast as a state estimation problem in which the temperature and wind velocity fields are the desired states to be reconstructed. The field is discretized into several grids in which the temperature and wind velocity fields are assumed to be constant. Different models, namely random walk, first order 3-D autoregressive (AR) model, and 1-D temporal AR model are used to capture the state evolution in time-space . Given the time of arrival (TOA) equation for acoustic propagation as the observation equation, the temperature and wind velocity fields are then reconstructed using a fixed point iterative UKF. The focus in the third part of this thesis is on generating a meaningful synthetic data for the temperature and wind velocity fields to test the proposed algorithms. A 2-D Fractal Brownian motion (fBm)-based method is used in order to generate realizations of the temperature and wind velocity fields. The synthetic data is generated for 500 subsequent snapshots of wind velocity and temperature field realizations with spatial resolution of one meter and temporal resolution of 12 seconds. Given the location of acoustic sensors the TOA&rsquos are calculated for all the acoustic paths. In addition, white Gaussian noise is added to the calculated TOAs in order to simulate the measurement error. The synthetic data is then used to test the proposed method and the results are compared to those of the TDSI method. This comparison attests to the superiority of the proposed method in terms of accuracy of reconstruction, real-time processing and the ability to track the temporal changes in the data

Adversarial Example Detection and Classification With Asymmetrical Adversarial Training

The vulnerabilities of deep neural networks against adversarial examples have become a significant concern for deploying these models in sensitive domains. Devising a definitive defense against such attacks is proven to be challenging, and the methods relying on detecting adversarial samples are only valid when the attacker is oblivious to the detection mechanism. In this paper we first present an adversarial example detection method that provides performance guarantee to norm constrained adversaries. The method is based on the idea of training adversarial robust subspace detectors using asymmetrical adversarial training (AAT). The novel AAT objective presents a minimax problem similar to that of GANs; it has the same convergence property, and consequently supports the learning of class conditional distributions. We first demonstrate that the minimax problem could be reasonably solved by PGD attack, and then use the learned class conditional generative models to define generative detection/classification models that are both robust and more interpretable. We provide comprehensive evaluations of the above methods, and demonstrate their competitive performances and compelling properties on adversarial detection and robust classification problems.Comment: ICLR 202

Neuromodulated attention and goal-driven perception in uncertain domains.

In uncertain domains, the goals are often unknown and need to be predicted by the organism or system. In this paper, contrastive Excitation Backprop (c-EB) was used in two goal-driven perception tasks - one with pairs of noisy MNIST digits and the other with a robot in an action-based attention scenario. The first task included attending to even, odd, low, and high digits, whereas the second task included action goals, such as "eat", "work-on-computer", "read", and "say-hi" that led to attention to objects associated with those actions. The system needed to increase attention to target items and decrease attention to distractor items and background noise. Because the valid goal was unknown, an online learning model based on the cholinergic and noradrenergic neuromodulatory systems was used to predict a noisy goal (expected uncertainty) and re-adapt when the goal changed (unexpected uncertainty). This neurobiologically plausible model demonstrates how neuromodulatory systems can predict goals in uncertain domains and how attentional mechanisms can enhance the perception for that goal

Image to Image Translation for Domain Adaptation

We propose a general framework for unsupervised domain adaptation, which allows deep neural networks trained on a source domain to be tested on a different target domain without requiring any training annotations in the target domain. This is achieved by adding extra networks and losses that help regularize the features extracted by the backbone encoder network. To this end we propose the novel use of the recently proposed unpaired image-toimage translation framework to constrain the features extracted by the encoder network. Specifically, we require that the features extracted are able to reconstruct the images in both domains. In addition we require that the distribution of features extracted from images in the two domains are indistinguishable. Many recent works can be seen as specific cases of our general framework. We apply our method for domain adaptation between MNIST, USPS, and SVHN datasets, and Amazon, Webcam and DSLR Office datasets in classification tasks, and also between GTA5 and Cityscapes datasets for a segmentation task. We demonstrate state of the art performance on each of these datasets