Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks

Effective training of deep neural networks suffers from two main issues. The first is that the parameter spaces of these models exhibit pathological curvature. Recent methods address this problem by using adaptive preconditioning for Stochastic Gradient Descent (SGD). These methods improve convergence by adapting to the local geometry of parameter space. A second issue is overfitting, which is typically addressed by early stopping. However, recent work has demonstrated that Bayesian model averaging mitigates this problem. The posterior can be sampled by using Stochastic Gradient Langevin Dynamics (SGLD). However, the rapidly changing curvature renders default SGLD methods inefficient. Here, we propose combining adaptive preconditioners with SGLD. In support of this idea, we give theoretical properties on asymptotic convergence and predictive risk. We also provide empirical results for Logistic Regression, Feedforward Neural Nets, and Convolutional Neural Nets, demonstrating that our preconditioned SGLD method gives state-of-the-art performance on these models.Comment: AAAI 201

Bio : A Mulrimodal biometric authentication system for person identification and verification

Not availabl

Bayesian Repulsive Mixture Modeling with Mat\'ern Point Processes

Mixture models are a standard tool in statistical analysis, widely used for density modeling and model-based clustering. Current approaches typically model the parameters of the mixture components as independent variables. This can result in overlapping or poorly separated clusters when either the number of clusters or the form of the mixture components is misspecified. Such model misspecification can undermine the interpretability and simplicity of these mixture models. To address this problem, we propose a Bayesian mixture model with repulsion between mixture components. The repulsion is induced by a generalized Mat\'ern type-III repulsive point process model, obtained through a dependent sequential thinning scheme on a primary Poisson point process. We derive a novel and efficient Gibbs sampler for posterior inference, and demonstrate the utility of the proposed method on a number of synthetic and real-world problems

Adversarial Learning of Mappings Onto Regularized Spaces for Biometric Authentication

We present AuthNet: a novel framework for generic biometric authentication which, by learning a regularized mapping instead of a classification boundary, leads to higher performance and improved robustness. The biometric traits are mapped onto a latent space in which authorized and unauthorized users follow simple and well-behaved distributions. In turn, this enables simple and tunable decision boundaries to be employed in order to make a decision. We show that, differently from the deep learning and traditional template-based authentication systems, regularizing the latent space to simple target distributions leads to improved performance as measured in terms of Equal Error Rate (EER), accuracy, False Acceptance Rate (FAR) and Genuine Acceptance Rate (GAR). Extensive experiments on publicly available datasets of faces and fingerprints confirm the superiority of AuthNet over existing methods

Mapping and Localization in Urban Environments Using Cameras

In this work we present a system to fully automatically create a highly accurate visual feature map from image data aquired from within a moving vehicle. Moreover, a system for high precision self localization is presented. Furthermore, we present a method to automatically learn a visual descriptor. The map relative self localization is centimeter accurate and allows autonomous driving

The fundamentals of unimodal palmprint authentication based on a biometric system: A review

Biometric system can be defined as the automated method of identifying or authenticating the identity of a living person based on physiological or behavioral traits. Palmprint biometric-based authentication has gained considerable attention in recent years. Globally, enterprises have been exploring biometric authorization for some time, for the purpose of security, payment processing, law enforcement CCTV systems, and even access to offices, buildings, and gyms via the entry doors. Palmprint biometric system can be divided into unimodal and multimodal. This paper will investigate the biometric system and provide a detailed overview of the palmprint technology with existing recognition approaches. Finally, we introduce a review of previous works based on a unimodal palmprint system using different databases

Advanced industrial OCR using Autoencoders

Il contenuto di questa tesi di laurea descrive il lavoro svolto durante un tirocinio di sei mesi presso Datalogic ADC. L'obiettivo del lavoro è stato quello di utilizzare uno specifico tipo di rete neurale, chiamata Autoencoder, per scopi legati al riconoscimento o alla convalida di caratteri in un sistema OCR industriale. In primo luogo è stato creato un classificatore di immagini di caratteri basato su Denoising Autoencoder; successivamente, è stato studiato un metodo per utilizzare l'Autoencoder come un classificatore di secondo livello, per meglio distinguere le false attivazioni da quelle corrette in condizioni di incertezza di un classificatore generico. Entrambe le architetture sono state valutate su dataset reali di clienti di Datalogic e i risultati sperimentali ottenuti sono presentati in questa tesi

Feature regularization and learning for human activity recognition.

Doctoral Degree. University of KwaZulu-Natal, Durban.Feature extraction is an essential component in the design of human activity recognition model. However, relying on extracted features alone for learning often makes the model a suboptimal model. Therefore, this research work seeks to address such potential problem by investigating feature regularization. Feature regularization is used for encapsulating discriminative patterns that are needed for better and efficient model learning. Firstly, a within-class subspace regularization approach is proposed for eigenfeatures extraction and regularization in human activity recognition. In this ap- proach, the within-class subspace is modelled using more eigenvalues from the reliable subspace to obtain a four-parameter modelling scheme. This model enables a better and true estimation of the eigenvalues that are distorted by the small sample size effect. This regularization is done in one piece, thereby avoiding undue complexity of modelling eigenspectrum differently. The whole eigenspace is used for performance evaluation because feature extraction and dimensionality reduction are done at a later stage of the evaluation process. Results show that the proposed approach has better discriminative capacity than several other subspace approaches for human activity recognition. Secondly, with the use of likelihood prior probability, a new regularization scheme that improves the loss function of deep convolutional neural network is proposed. The results obtained from this work demonstrate that a well regularized feature yields better class discrimination in human activity recognition. The major contribution of the thesis is the development of feature extraction strategies for determining discriminative patterns needed for efficient model learning