168 research outputs found
Correlations between hidden units in multilayer neural networks and replica symmetry breaking
We consider feed-forward neural networks with one hidden layer, tree
architecture and a fixed hidden-to-output Boolean function. Focusing on the
saturation limit of the storage problem the influence of replica symmetry
breaking on the distribution of local fields at the hidden units is
investigated. These field distributions determine the probability for finding a
specific activation pattern of the hidden units as well as the corresponding
correlation coefficients and therefore quantify the division of labor among the
hidden units. We find that although modifying the storage capacity and the
distribution of local fields markedly replica symmetry breaking has only a
minor effect on the correlation coefficients. Detailed numerical results are
provided for the PARITY, COMMITTEE and AND machines with K=3 hidden units and
nonoverlapping receptive fields.Comment: 9 pages, 3 figures, RevTex, accepted for publication in Phys. Rev.
Correlation of internal representations in feed-forward neural networks
Feed-forward multilayer neural networks implementing random input-output
mappings develop characteristic correlations between the activity of their
hidden nodes which are important for the understanding of the storage and
generalization performance of the network. It is shown how these correlations
can be calculated from the joint probability distribution of the aligning
fields at the hidden units for arbitrary decoder function between hidden layer
and output. Explicit results are given for the parity-, and-, and
committee-machines with arbitrary number of hidden nodes near saturation.Comment: 6 pages, latex, 1 figur
Statistical Mechanics of Learning: A Variational Approach for Real Data
Using a variational technique, we generalize the statistical physics approach
of learning from random examples to make it applicable to real data. We
demonstrate the validity and relevance of our method by computing approximate
estimators for generalization errors that are based on training data alone.Comment: 4 pages, 2 figure
Storage capacity of correlated perceptrons
We consider an ensemble of single-layer perceptrons exposed to random
inputs and investigate the conditions under which the couplings of these
perceptrons can be chosen such that prescribed correlations between the outputs
occur. A general formalism is introduced using a multi-perceptron costfunction
that allows to determine the maximal number of random inputs as a function of
the desired values of the correlations. Replica-symmetric results for and
are compared with properties of two-layer networks of tree-structure and
fixed Boolean function between hidden units and output. The results show which
correlations in the hidden layer of multi-layer neural networks are crucial for
the value of the storage capacity.Comment: 16 pages, Latex2
Statistical Mechanical Development of a Sparse Bayesian Classifier
The demand for extracting rules from high dimensional real world data is
increasing in various fields. However, the possible redundancy of such data
sometimes makes it difficult to obtain a good generalization ability for novel
samples. To resolve this problem, we provide a scheme that reduces the
effective dimensions of data by pruning redundant components for bicategorical
classification based on the Bayesian framework. First, the potential of the
proposed method is confirmed in ideal situations using the replica method.
Unfortunately, performing the scheme exactly is computationally difficult. So,
we next develop a tractable approximation algorithm, which turns out to offer
nearly optimal performance in ideal cases when the system size is large.
Finally, the efficacy of the developed classifier is experimentally examined
for a real world problem of colon cancer classification, which shows that the
developed method can be practically useful.Comment: 13 pages, 6 figure
Training a perceptron in a discrete weight space
On-line and batch learning of a perceptron in a discrete weight space, where
each weight can take different values, are examined analytically and
numerically. The learning algorithm is based on the training of the continuous
perceptron and prediction following the clipped weights. The learning is
described by a new set of order parameters, composed of the overlaps between
the teacher and the continuous/clipped students. Different scenarios are
examined among them on-line learning with discrete/continuous transfer
functions and off-line Hebb learning. The generalization error of the clipped
weights decays asymptotically as / in the case of on-line learning with binary/continuous activation
functions, respectively, where is the number of examples divided by N,
the size of the input vector and is a positive constant that decays
linearly with 1/L. For finite and , a perfect agreement between the
discrete student and the teacher is obtained for . A crossover to the generalization error ,
characterized continuous weights with binary output, is obtained for synaptic
depth .Comment: 10 pages, 5 figs., submitted to PR
Replica theory for learning curves for Gaussian processes on random graphs
Statistical physics approaches can be used to derive accurate predictions for
the performance of inference methods learning from potentially noisy data, as
quantified by the learning curve defined as the average error versus number of
training examples. We analyse a challenging problem in the area of
non-parametric inference where an effectively infinite number of parameters has
to be learned, specifically Gaussian process regression. When the inputs are
vertices on a random graph and the outputs noisy function values, we show that
replica techniques can be used to obtain exact performance predictions in the
limit of large graphs. The covariance of the Gaussian process prior is defined
by a random walk kernel, the discrete analogue of squared exponential kernels
on continuous spaces. Conventionally this kernel is normalised only globally,
so that the prior variance can differ between vertices; as a more principled
alternative we consider local normalisation, where the prior variance is
uniform
Evidence for Alternative Hypotheses
Most researchers want evidence for the direction of an effect, not evidence against a point null hypothesis. Such evidence is ideally on a scale that is easily in- terpretable, with an accompanying standard error. Further, the evidence from iden- tical experiments should be repeatable, and evidence from independent experiments should be easily combined, such as required in meta-analysis. Such a measure of evidence exists and has been shown to be closely related to the Kullback-Leibler symmetrized distance between null and alternative hypotheses for exponential fam- ilies. Here we provide more examples of the latter phenomenon, for distributions ly- ing outside the class of exponential families, including the non-central chi-squared family with unknown non-centrality parameter
Thiolutin is a zinc chelator that inhibits the Rpn11 and other JAMM metalloproteases
Thiolutin is a disulfide-containing antibiotic and anti-angiogenic compound produced by Streptomyces. Its biological targets are not known. We show that reduced thiolutin is a zinc chelator that inhibits the JAB1/MPN/Mov34 (JAMM) domain–containing metalloprotease Rpn11, a deubiquitinating enzyme of the 19S proteasome. Thiolutin also inhibits the JAMM metalloproteases Csn5, the deneddylase of the COP9 signalosome; AMSH, which regulates ubiquitin-dependent sorting of cell-surface receptors; and BRCC36, a K63-specific deubiquitinase of the BRCC36-containing isopeptidase complex and the BRCA1–BRCA2-containing complex. We provide evidence that other dithiolopyrrolones also function as inhibitors of JAMM metalloproteases
Climate change effects on phytoplankton depend on cell size and food web structure
We investigated the effects of warming on a natural phytoplankton community from the Baltic Sea, based on six mesocosm experiments conducted 2005–2009. We focused on differences in the dynamics of three phytoplankton size groups which are grazed to a variable extent by different zooplankton groups. While small-sized algae were mostly grazer-controlled, light and nutrient availability largely determined the growth of medium- and large-sized algae. Thus, the latter groups dominated at increased light levels. Warming increased mesozooplankton grazing on medium-sized algae, reducing their biomass. The biomass of small-sized algae was not affected by temperature, probably due to an interplay between indirect effects spreading through the food web. Thus, under the higher temperature and lower light levels anticipated for the next decades in the southern Baltic Sea, a higher share of smaller phytoplankton is expected. We conclude that considering the size structure of the phytoplankton community strongly improves the reliability of projections of climate change effects
- …