Data Dropout in Arbitrary Basis for Deep Network Regularization

An important problem in training deep networks with high capacity is to ensure that the trained network works well when presented with new inputs outside the training dataset. Dropout is an effective regularization technique to boost the network generalization in which a random subset of the elements of the given data and the extracted features are set to zero during the training process. In this paper, a new randomized regularization technique in which we withhold a random part of the data without necessarily turning off the neurons/data-elements is proposed. In the proposed method, of which the conventional dropout is shown to be a special case, random data dropout is performed in an arbitrary basis, hence the designation Generalized Dropout. We also present a framework whereby the proposed technique can be applied efficiently to convolutional neural networks. The presented numerical experiments demonstrate that the proposed technique yields notable performance gain. Generalized Dropout provides new insight into the idea of dropout, shows that we can achieve different performance gains by using different bases matrices, and opens up a new research question as of how to choose optimal bases matrices that achieve maximal performance gain

Weighing matrices and spherical codes

Mutually unbiased weighing matrices (MUWM) are closely related to an antipodal spherical code with 4 angles. In the present paper, we clarify the relationship between MUWM and the spherical sets, and give the complete solution about the maximum size of a set of MUWM of weight 4 for any order. Moreover we describe some natural generalization of a set of MUWM from the viewpoint of spherical codes, and determine several maximum sizes of the generalized sets. They include an affirmative answer of the problem of Best, Kharaghani, and Ramp.Comment: Title is changed from "Association schemes related to weighing matrices

Maximal determinants and saturated D-optimal designs of orders 19 and 37

A saturated D-optimal design is a {+1,-1} square matrix of given order with maximal determinant. We search for saturated D-optimal designs of orders 19 and 37, and find that known matrices due to Smith, Cohn, Orrick and Solomon are optimal. For order 19 we find all inequivalent saturated D-optimal designs with maximal determinant, 2^30 x 7^2 x 17, and confirm that the three known designs comprise a complete set. For order 37 we prove that the maximal determinant is 2^39 x 3^36, and find a sample of inequivalent saturated D-optimal designs. Our method is an extension of that used by Orrick to resolve the previously smallest unknown order of 15; and by Chadjipantelis, Kounias and Moyssiadis to resolve orders 17 and 21. The method is a two-step computation which first searches for candidate Gram matrices and then attempts to decompose them. Using a similar method, we also find the complete spectrum of determinant values for {+1,-1} matrices of order 13.Comment: 28 pages, 4 figure

Tropical totally positive matrices

We investigate the tropical analogues of totally positive and totally nonnegative matrices. These arise when considering the images by the nonarchimedean valuation of the corresponding classes of matrices over a real nonarchimedean valued field, like the field of real Puiseux series. We show that the nonarchimedean valuation sends the totally positive matrices precisely to the Monge matrices. This leads to explicit polyhedral representations of the tropical analogues of totally positive and totally nonnegative matrices. We also show that tropical totally nonnegative matrices with a finite permanent can be factorized in terms of elementary matrices. We finally determine the eigenvalues of tropical totally nonnegative matrices, and relate them with the eigenvalues of totally nonnegative matrices over nonarchimedean fields.Comment: The first author has been partially supported by the PGMO Program of FMJH and EDF, and by the MALTHY Project of the ANR Program. The second author is sported by the French Chateaubriand grant and INRIA postdoctoral fellowshi