2,128 research outputs found
Constructive Preference Elicitation over Hybrid Combinatorial Spaces
Preference elicitation is the task of suggesting a highly preferred
configuration to a decision maker. The preferences are typically learned by
querying the user for choice feedback over pairs or sets of objects. In its
constructive variant, new objects are synthesized "from scratch" by maximizing
an estimate of the user utility over a combinatorial (possibly infinite) space
of candidates. In the constructive setting, most existing elicitation
techniques fail because they rely on exhaustive enumeration of the candidates.
A previous solution explicitly designed for constructive tasks comes with no
formal performance guarantees, and can be very expensive in (or unapplicable
to) problems with non-Boolean attributes. We propose the Choice Perceptron, a
Perceptron-like algorithm for learning user preferences from set-wise choice
feedback over constructive domains and hybrid Boolean-numeric feature spaces.
We provide a theoretical analysis on the attained regret that holds for a large
class of query selection strategies, and devise a heuristic strategy that aims
at optimizing the regret in practice. Finally, we demonstrate its effectiveness
by empirical evaluation against existing competitors on constructive scenarios
of increasing complexity.Comment: AAAI 2018, computing methodologies, machine learning, learning
paradigms, supervised learning, structured output
Learning by message-passing in networks of discrete synapses
We show that a message-passing process allows to store in binary "material"
synapses a number of random patterns which almost saturates the information
theoretic bounds. We apply the learning algorithm to networks characterized by
a wide range of different connection topologies and of size comparable with
that of biological systems (e.g. ). The algorithm can be
turned into an on-line --fault tolerant-- learning protocol of potential
interest in modeling aspects of synaptic plasticity and in building
neuromorphic devices.Comment: 4 pages, 3 figures; references updated and minor corrections;
accepted in PR
Replica Symmetry Breaking and the Kuhn-Tucker Cavity Method in simple and multilayer Perceptrons
Within a Kuhn-Tucker cavity method introduced in a former paper, we study
optimal stability learning for situations, where in the replica formalism the
replica symmetry may be broken, namely
(i) the case of a simple perceptron above the critical loading, and
(ii) the case of two-layer AND-perceptrons, if one learns with maximal
stability.
We find that the deviation of our cavity solution from the replica symmetric
one in these cases is a clear indication of the necessity of replica symmetry
breaking. In any case the cavity solution tends to underestimate the storage
capabilities of the networks.Comment: 32 pages, LaTex Source with 9 .eps-files enclosed, accepted by J.
Phys I (France
Generalization from correlated sets of patterns in the perceptron
Generalization is a central aspect of learning theory. Here, we propose a
framework that explores an auxiliary task-dependent notion of generalization,
and attempts to quantitatively answer the following question: given two sets of
patterns with a given degree of dissimilarity, how easily will a network be
able to "unify" their interpretation? This is quantified by the volume of the
configurations of synaptic weights that classify the two sets in a similar
manner. To show the applicability of our idea in a concrete setting, we compute
this quantity for the perceptron, a simple binary classifier, using the
classical statistical physics approach in the replica-symmetric ansatz. In this
case, we show how an analytical expression measures the "distance-based
capacity", the maximum load of patterns sustainable by the network, at fixed
dissimilarity between patterns and fixed allowed number of errors. This curve
indicates that generalization is possible at any distance, but with decreasing
capacity. We propose that a distance-based definition of generalization may be
useful in numerical experiments with real-world neural networks, and to explore
computationally sub-dominant sets of synaptic solutions
On The Robustness of a Neural Network
With the development of neural networks based machine learning and their
usage in mission critical applications, voices are rising against the
\textit{black box} aspect of neural networks as it becomes crucial to
understand their limits and capabilities. With the rise of neuromorphic
hardware, it is even more critical to understand how a neural network, as a
distributed system, tolerates the failures of its computing nodes, neurons, and
its communication channels, synapses. Experimentally assessing the robustness
of neural networks involves the quixotic venture of testing all the possible
failures, on all the possible inputs, which ultimately hits a combinatorial
explosion for the first, and the impossibility to gather all the possible
inputs for the second.
In this paper, we prove an upper bound on the expected error of the output
when a subset of neurons crashes. This bound involves dependencies on the
network parameters that can be seen as being too pessimistic in the average
case. It involves a polynomial dependency on the Lipschitz coefficient of the
neurons activation function, and an exponential dependency on the depth of the
layer where a failure occurs. We back up our theoretical results with
experiments illustrating the extent to which our prediction matches the
dependencies between the network parameters and robustness. Our results show
that the robustness of neural networks to the average crash can be estimated
without the need to neither test the network on all failure configurations, nor
access the training set used to train the network, both of which are
practically impossible requirements.Comment: 36th IEEE International Symposium on Reliable Distributed Systems 26
- 29 September 2017. Hong Kong, Chin
- …