Training Faster by Separating Modes of Variation in Batch-normalized Models

Batch Normalization (BN) is essential to effectively train state-of-the-art deep Convolutional Neural Networks (CNN). It normalizes inputs to the layers during training using the statistics of each mini-batch. In this work, we study BN from the viewpoint of Fisher kernels. We show that assuming samples within a mini-batch are from the same probability density function, then BN is identical to the Fisher vector of a Gaussian distribution. That means BN can be explained in terms of kernels that naturally emerge from the probability density function of the underlying data distribution. However, given the rectifying non-linearities employed in CNN architectures, distribution of inputs to the layers show heavy tail and asymmetric characteristics. Therefore, we propose approximating underlying data distribution not with one, but a mixture of Gaussian densities. Deriving Fisher vector for a Gaussian Mixture Model (GMM), reveals that BN can be improved by independently normalizing with respect to the statistics of disentangled sub-populations. We refer to our proposed soft piecewise version of BN as Mixture Normalization (MN). Through extensive set of experiments on CIFAR-10 and CIFAR-100, we show that MN not only effectively accelerates training image classification and Generative Adversarial networks, but also reaches higher quality models

Quantum Pseudodots Under the External Vector and Scalar Fields

We study the spherical quantum pseudodots in the Schrodinger equation using the pseudo-harmonic plus harmonic oscillator potentials considering the effect of the external electric and magnetic fields. The finite energy levels and the wave functions are calculated. Furthermore, the behavior of the essential thermodynamic quantities such as, the free energy, the mean energy, the entropy, the specific heat, the magnetization, the magnetic susceptibility and the persistent currents are also studied using the characteristic function. Our analytical results are found to be in good agreement with the other works. The numerical results on the energy levels as well as the thermodynamic quantities have also been given.Comment: 18 Fig

Deep Sparse Representation-based Classification

We present a transductive deep learning-based formulation for the sparse representation-based classification (SRC) method. The proposed network consists of a convolutional autoencoder along with a fully-connected layer. The role of the autoencoder network is to learn robust deep features for classification. On the other hand, the fully-connected layer, which is placed in between the encoder and the decoder networks, is responsible for finding the sparse representation. The estimated sparse codes are then used for classification. Various experiments on three different datasets show that the proposed network leads to sparse representations that give better classification results than state-of-the-art SRC methods. The source code is available at: github.com/mahdiabavisani/DSRC

Deep Multimodal Subspace Clustering Networks

We present convolutional neural network (CNN) based approaches for unsupervised multimodal subspace clustering. The proposed framework consists of three main stages - multimodal encoder, self-expressive layer, and multimodal decoder. The encoder takes multimodal data as input and fuses them to a latent space representation. The self-expressive layer is responsible for enforcing the self-expressiveness property and acquiring an affinity matrix corresponding to the data points. The decoder reconstructs the original input data. The network uses the distance between the decoder's reconstruction and the original input in its training. We investigate early, late and intermediate fusion techniques and propose three different encoders corresponding to them for spatial fusion. The self-expressive layers and multimodal decoders are essentially the same for different spatial fusion-based approaches. In addition to various spatial fusion-based methods, an affinity fusion-based network is also proposed in which the self-expressive layer corresponding to different modalities is enforced to be the same. Extensive experiments on three datasets show that the proposed methods significantly outperform the state-of-the-art multimodal subspace clustering methods

Exact solutions of a spatially-dependent mass Dirac equation for Coulomb field plus tensor interaction via Laplace transformation method

The spatially-dependent mass Dirac equation is solved exactly for attractive scalar and repulsive vector Coulomb potentials including a tensor interaction potential under the spin and pseudospin (p-spin) symmetric limits by using the Laplace transformation method (LTM). Closed forms of the energy eigenvalue equation and wave functions are obtained for arbitrary spin-orbit quantum number Some numerical results are given too. The effect of the tensor interaction on the bound states is presented. It is shown that the tensor interaction removes the degeneracy between two states in the spin doublets. We also investigate the effects of the spatially-dependent mass on the bound states under the conditions of the spin symmetric limit and in the absence of tensor interactionComment: 17 pages, 4 figure

Ring correlations in random networks

We examine the correlations between rings in random network glasses in two dimensions as a function of their separation. Initially, we use the topological separation (measured by the number of intervening rings), but this leads to pseudo-long-range correlations due to a lack of topological charge neutrality in the shells surrounding a central ring. This effect is associated with the non-circular nature of the shells. It is, therefore, necessary to use the geometrical distance between ring centers. Hence we find a generalization of the Aboav-Weaire law out to larger distances, with the correlations between rings decaying away when two rings are more than about 3 rings apart.Comment: 7 pages, 8 Figures, v2 updated bibliograph

On stability analysis by using Nyquist and Nichols Charts

This paper reviews stability analysis techniques by using the Nyquist and Nichols charts. The relationship between the Nyquist and Nichols stability criteria is fully described by using the crossing concept. The results are demonstrated through several numerical examples. This tutorial provides useful insights into the loop-shaping based control systems design such as Quantitative Feedback Theory

Bayesian inference in non-Markovian state-space models with applications to fractional order systems

Battery impedance spectroscopy models are given by fractional order (FO) differential equations. In the discrete-time domain, they give rise to state-space models where the latent process is not Markovian. Parameter estimation for these models is therefore challenging, especially for non-commensurate FO models. In this paper, we propose a Bayesian approach to identify the parameters of generic FO systems. The computational challenge is tackled with particle Markov chain Monte Carlo methods, with an implementation specifically designed for the non-Markovian setting. The approach is then applied to estimate the parameters of a battery non-commensurate FO equivalent circuit model. Extensive simulations are provided to study the practical identifiability of model parameters and their sensitivity to the choice of prior distributions, the number of observations, the magnitude of the input signal and the measurement noise

ALCN: Meta-Learning for Contrast Normalization Applied to Robust 3D Pose Estimation

To be robust to illumination changes when detecting objects in images, the current trend is to train a Deep Network with training images captured under many different lighting conditions. Unfortunately, creating such a training set is very cumbersome, or sometimes even impossible, for some applications such as 3D pose estimation of specific objects, which is the application we focus on in this paper. We therefore propose a novel illumination normalization method that lets us learn to detect objects and estimate their 3D pose under challenging illumination conditions from very few training samples. Our key insight is that normalization parameters should adapt to the input image. In particular, we realized this via a Convolutional Neural Network trained to predict the parameters of a generalization of the Difference-of-Gaussians method. We show that our method significantly outperforms standard normalization methods and demonstrate it on two challenging 3D detection and pose estimation problems.Comment: BMVC' 1

Structural Identifiability Analysis of Fractional Order Models with Applications in Battery Systems

This paper presents a method for structural identifiability analysis of fractional order systems by using the coefficient mapping concept to determine whether the model parameters can uniquely be identified from input-output data. The proposed method is applicable to general non-commensurate fractional order models. Examples are chosen from battery fractional order equivalent circuit models (FO-ECMs). The battery FO-ECM consists of a series of parallel resistors and constant phase elements (CPEs) with fractional derivatives appearing in the CPEs. The FO-ECM is non-commensurate if more than one CPE is considered in the model. Currently, estimation of battery FO-ECMs is performed mainly by fitting in the frequency domain, requiring costly electrochemical impedance spectroscopy equipment. This paper aims to analyse the structural identifiability of battery FO-ECMs directly in the time domain. It is shown that FO-ECMs with finite numbers of CPEs are structurally identifiable. In particular, the FO-ECM with a single CPE is structurally globally identifiable