10 research outputs found
Learning Combinations of Activation Functions
In the last decade, an active area of research has been devoted to design
novel activation functions that are able to help deep neural networks to
converge, obtaining better performance. The training procedure of these
architectures usually involves optimization of the weights of their layers
only, while non-linearities are generally pre-specified and their (possible)
parameters are usually considered as hyper-parameters to be tuned manually. In
this paper, we introduce two approaches to automatically learn different
combinations of base activation functions (such as the identity function, ReLU,
and tanh) during the training phase. We present a thorough comparison of our
novel approaches with well-known architectures (such as LeNet-5, AlexNet, and
ResNet-56) on three standard datasets (Fashion-MNIST, CIFAR-10, and
ILSVRC-2012), showing substantial improvements in the overall performance, such
as an increase in the top-1 accuracy for AlexNet on ILSVRC-2012 of 3.01
percentage points.Comment: 6 pages, 3 figures. Published as a conference paper at ICPR 2018.
Code:
https://bitbucket.org/francux/learning_combinations_of_activation_function
The Feynman problem and Fermionic entanglement: Fermionic theory versus qubit theory
The present paper is both a review on the Feynman problem, and an original
research presentation on the relations between Fermionic theories and qubits
theories, both regarded in the novel framework of operational probabilistic
theories. The most relevant results about the Feynman problem of simulating
Fermions with qubits are reviewed, and in the light of the new original results
the problem is solved. The answer is twofold. On the computational side the two
theories are equivalent, as shown by Bravyi and Kitaev (Ann. Phys. 298.1
(2002): 210-226). On the operational side the quantum theory of qubits and the
quantum theory of Fermions are different, mostly in the notion of locality,
with striking consequences on entanglement. Thus the emulation does not respect
locality, as it was suspected by Feynman (Int. J. Theor. Phys. 21.6 (1982):
467-488).Comment: 46 pages, review about the "Feynman problem". Fixed many typo
Automated Pruning for Deep Neural Network Compression
In this work we present a method to improve the pruning step of the current
state-of-the-art methodology to compress neural networks. The novelty of the
proposed pruning technique is in its differentiability, which allows pruning to
be performed during the backpropagation phase of the network training. This
enables an end-to-end learning and strongly reduces the training time. The
technique is based on a family of differentiable pruning functions and a new
regularizer specifically designed to enforce pruning. The experimental results
show that the joint optimization of both the thresholds and the network weights
permits to reach a higher compression rate, reducing the number of weights of
the pruned network by a further 14% to 33% compared to the current
state-of-the-art. Furthermore, we believe that this is the first study where
the generalization capabilities in transfer learning tasks of the features
extracted by a pruned network are analyzed. To achieve this goal, we show that
the representations learned using the proposed pruning methodology maintain the
same effectiveness and generality of those learned by the corresponding
non-compressed network on a set of different recognition tasks.Comment: 8 pages, 5 figures. Published as a conference paper at ICPR 201
Spooky action at a distance in general probabilistic theories
We call a probabilistic theory "complete" if it cannot be further refined by
no-signaling hidden-variable models, and name a theory "spooky" if every
equivalent hidden-variable model violates Shimony's Outcome Independence. We
prove that a complete theory is spooky if and only if it admits a pure steering
state in the sense of Schr\"odinger. Finally we show that steering of
complementary states leads to a Schr\"odinger's cat-like paradox.Comment: 7 pages, 1 figure, elsart, significantly revised versio
Determinism without causality
Causality has often been confused with the notion of determinism. It is mandatory to separate the two notions in view of the debate about quantum foundations. Quantum theory provides an example of causal non-deterministic theory. Here we introduce a toy operational theory that is deterministic and non-causal, thus proving that the two notions of causality and determinism are totally independent