30,512 research outputs found
Learning to Predict Combinatorial Structures
The major challenge in designing a discriminative learning algorithm for
predicting structured data is to address the computational issues arising from
the exponential size of the output space. Existing algorithms make different
assumptions to ensure efficient, polynomial time estimation of model
parameters. For several combinatorial structures, including cycles, partially
ordered sets, permutations and other graph classes, these assumptions do not
hold. In this thesis, we address the problem of designing learning algorithms
for predicting combinatorial structures by introducing two new assumptions: (i)
The first assumption is that a particular counting problem can be solved
efficiently. The consequence is a generalisation of the classical ridge
regression for structured prediction. (ii) The second assumption is that a
particular sampling problem can be solved efficiently. The consequence is a new
technique for designing and analysing probabilistic structured prediction
models. These results can be applied to solve several complex learning problems
including but not limited to multi-label classification, multi-category
hierarchical classification, and label ranking.Comment: PhD thesis, Department of Computer Science, University of Bonn
(submitted, December 2009
Training neural networks to encode symbols enables combinatorial generalization
Combinatorial generalization - the ability to understand and produce novel
combinations of already familiar elements - is considered to be a core capacity
of the human mind and a major challenge to neural network models. A significant
body of research suggests that conventional neural networks can't solve this
problem unless they are endowed with mechanisms specifically engineered for the
purpose of representing symbols. In this paper we introduce a novel way of
representing symbolic structures in connectionist terms - the vectors approach
to representing symbols (VARS), which allows training standard neural
architectures to encode symbolic knowledge explicitly at their output layers.
In two simulations, we show that neural networks not only can learn to produce
VARS representations, but in doing so they achieve combinatorial generalization
in their symbolic and non-symbolic output. This adds to other recent work that
has shown improved combinatorial generalization under specific training
conditions, and raises the question of whether specific mechanisms or training
routines are needed to support symbolic processing
Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals
Graph networks are a new machine learning (ML) paradigm that supports both
relational reasoning and combinatorial generalization. Here, we develop
universal MatErials Graph Network (MEGNet) models for accurate property
prediction in both molecules and crystals. We demonstrate that the MEGNet
models outperform prior ML models such as the SchNet in 11 out of 13 properties
of the QM9 molecule data set. Similarly, we show that MEGNet models trained on
crystals in the Materials Project substantially outperform prior
ML models in the prediction of the formation energies, band gaps and elastic
moduli of crystals, achieving better than DFT accuracy over a much larger data
set. We present two new strategies to address data limitations common in
materials science and chemistry. First, we demonstrate a physically-intuitive
approach to unify four separate molecular MEGNet models for the internal energy
at 0 K and room temperature, enthalpy and Gibbs free energy into a single free
energy MEGNet model by incorporating the temperature, pressure and entropy as
global state inputs. Second, we show that the learned element embeddings in
MEGNet models encode periodic chemical trends and can be transfer-learned from
a property model trained on a larger data set (formation energies) to improve
property models with smaller amounts of data (band gaps and elastic moduli)
- …