G\mathcal{G}-softmax: Improving Intra-class Compactness and Inter-class Separability of Features

Intra-class compactness and inter-class separability are crucial indicators to measure the effectiveness of a model to produce discriminative features, where intra-class compactness indicates how close the features with the same label are to each other and inter-class separability indicates how far away the features with different labels are. In this work, we investigate intra-class compactness and inter-class separability of features learned by convolutional networks and propose a Gaussian-based softmax ($\mathcal{G}$-softmax) function that can effectively improve intra-class compactness and inter-class separability. The proposed function is simple to implement and can easily replace the softmax function. We evaluate the proposed $\mathcal{G}$-softmax function on classification datasets (i.e., CIFAR-10, CIFAR-100, and Tiny ImageNet) and on multi-label classification datasets (i.e., MS COCO and NUS-WIDE). The experimental results show that the proposed $\mathcal{G}$-softmax function improves the state-of-the-art models across all evaluated datasets. In addition, analysis of the intra-class compactness and inter-class separability demonstrates the advantages of the proposed function over the softmax function, which is consistent with the performance improvement. More importantly, we observe that high intra-class compactness and inter-class separability are linearly correlated to average precision on MS COCO and NUS-WIDE. This implies that improvement of intra-class compactness and inter-class separability would lead to improvement of average precision.Comment: 15 pages, published in TNNL

Inflationary spectra and partially decohered distributions

It is generally expected that decoherence processes will erase the quantum properties of the inflationary primordial spectra. However, given the weakness of gravitational interactions, one might end up with a distribution which is only partially decohered. Below a certain critical change, we show that the inflationary distribution retains quantum properties. We identify four of these: a squeezed spread in some direction of phase space, non-vanishing off-diagonal matrix elements, and two properties used in quantum optics called non-$P$-representability and non-separability. The last two are necessary conditions to violate Bell's inequalities. The critical value above which all these properties are lost is associated to the `grain' of coherent states. The corresponding value of the entropy is equal to half the maximal (thermal) value. Moreover it coincides with the entropy of the effective distribution obtained by neglecting the decaying modes. By considering backreaction effects, we also provide an upper bound for this entropy at the onset of the adiabatic era.Comment: 42 pages, 9 figures; 1 ref. adde