6,133 research outputs found
Hierarchical Quantized Representations for Script Generation
Scripts define knowledge about how everyday scenarios (such as going to a
restaurant) are expected to unfold. One of the challenges to learning scripts
is the hierarchical nature of the knowledge. For example, a suspect arrested
might plead innocent or guilty, and a very different track of events is then
expected to happen. To capture this type of information, we propose an
autoencoder model with a latent space defined by a hierarchy of categorical
variables. We utilize a recently proposed vector quantization based approach,
which allows continuous embeddings to be associated with each latent variable
value. This permits the decoder to softly decide what portions of the latent
hierarchy to condition on by attending over the value embeddings for a given
setting. Our model effectively encodes and generates scripts, outperforming a
recent language modeling-based method on several standard tasks, and allowing
the autoencoder model to achieve substantially lower perplexity scores compared
to the previous language modeling-based method.Comment: EMNLP 201
Graphs in machine learning: an introduction
Graphs are commonly used to characterise interactions between objects of
interest. Because they are based on a straightforward formalism, they are used
in many scientific fields from computer science to historical sciences. In this
paper, we give an introduction to some methods relying on graphs for learning.
This includes both unsupervised and supervised methods. Unsupervised learning
algorithms usually aim at visualising graphs in latent spaces and/or clustering
the nodes. Both focus on extracting knowledge from graph topologies. While most
existing techniques are only applicable to static graphs, where edges do not
evolve through time, recent developments have shown that they could be extended
to deal with evolving networks. In a supervised context, one generally aims at
inferring labels or numerical values attached to nodes using both the graph
and, when they are available, node characteristics. Balancing the two sources
of information can be challenging, especially as they can disagree locally or
globally. In both contexts, supervised and un-supervised, data can be
relational (augmented with one or several global graphs) as described above, or
graph valued. In this latter case, each object of interest is given as a full
graph (possibly completed by other characteristics). In this context, natural
tasks include graph clustering (as in producing clusters of graphs rather than
clusters of nodes in a single graph), graph classification, etc. 1 Real
networks One of the first practical studies on graphs can be dated back to the
original work of Moreno [51] in the 30s. Since then, there has been a growing
interest in graph analysis associated with strong developments in the modelling
and the processing of these data. Graphs are now used in many scientific
fields. In Biology [54, 2, 7], for instance, metabolic networks can describe
pathways of biochemical reactions [41], while in social sciences networks are
used to represent relation ties between actors [66, 56, 36, 34]. Other examples
include powergrids [71] and the web [75]. Recently, networks have also been
considered in other areas such as geography [22] and history [59, 39]. In
machine learning, networks are seen as powerful tools to model problems in
order to extract information from data and for prediction purposes. This is the
object of this paper. For more complete surveys, we refer to [28, 62, 49, 45].
In this section, we introduce notations and highlight properties shared by most
real networks. In Section 2, we then consider methods aiming at extracting
information from a unique network. We will particularly focus on clustering
methods where the goal is to find clusters of vertices. Finally, in Section 3,
techniques that take a series of networks into account, where each network i
Interpreting self-organizing maps through space--time data models
Self-organizing maps (SOMs) are a technique that has been used with
high-dimensional data vectors to develop an archetypal set of states (nodes)
that span, in some sense, the high-dimensional space. Noteworthy applications
include weather states as described by weather variables over a region and
speech patterns as characterized by frequencies in time. The SOM approach is
essentially a neural network model that implements a nonlinear projection from
a high-dimensional input space to a low-dimensional array of neurons. In the
process, it also becomes a clustering technique, assigning to any vector in the
high-dimensional data space the node (neuron) to which it is closest (using,
say, Euclidean distance) in the data space. The number of nodes is thus equal
to the number of clusters. However, the primary use for the SOM is as a
representation technique, that is, finding a set of nodes which
representatively span the high-dimensional space. These nodes are typically
displayed using maps to enable visualization of the continuum of the data
space. The technique does not appear to have been discussed in the statistics
literature so it is our intent here to bring it to the attention of the
community. The technique is implemented algorithmically through a training set
of vectors. However, through the introduction of stochasticity in the form of a
space--time process model, we seek to illuminate and interpret its performance
in the context of application to daily data collection. That is, the observed
daily state vectors are viewed as a time series of multivariate process
realizations which we try to understand under the dimension reduction achieved
by the SOM procedure.Comment: Published in at http://dx.doi.org/10.1214/08-AOAS174 the Annals of
Applied Statistics (http://www.imstat.org/aoas/) by the Institute of
Mathematical Statistics (http://www.imstat.org
The brain: What is critical about it?
We review the recent proposal that the most fascinating brain properties are
related to the fact that it always stays close to a second order phase
transition. In such conditions, the collective of neuronal groups can reliably
generate robust and flexible behavior, because it is known that at the critical
point there is the largest abundance of metastable states to choose from. Here
we review the motivation, arguments and recent results, as well as further
implications of this view of the functioning brain.Comment: Proceedings of BIOCOMP2007 - Collective Dynamics: Topics on
Competition and Cooperation in the Biosciences. Vietri sul Mare, Italy (2007
Cortical Models for Movement Control
Defense Advanced Research Projects Agency and Office of Naval Research (N0014-95-l-0409)
- …