Hierarchical Quantized Representations for Script Generation

Scripts define knowledge about how everyday scenarios (such as going to a restaurant) are expected to unfold. One of the challenges to learning scripts is the hierarchical nature of the knowledge. For example, a suspect arrested might plead innocent or guilty, and a very different track of events is then expected to happen. To capture this type of information, we propose an autoencoder model with a latent space defined by a hierarchy of categorical variables. We utilize a recently proposed vector quantization based approach, which allows continuous embeddings to be associated with each latent variable value. This permits the decoder to softly decide what portions of the latent hierarchy to condition on by attending over the value embeddings for a given setting. Our model effectively encodes and generates scripts, outperforming a recent language modeling-based method on several standard tasks, and allowing the autoencoder model to achieve substantially lower perplexity scores compared to the previous language modeling-based method.Comment: EMNLP 201

How to use the Kohonen algorithm to simultaneously analyse individuals in a survey

The Kohonen algorithm (SOM, Kohonen,1984, 1995) is a very powerful tool for data analysis. It was originally designed to model organized connections between some biological neural networks. It was also immediately considered as a very good algorithm to realize vectorial quantization, and at the same time pertinent classification, with nice properties for visualization. If the individuals are described by quantitative variables (ratios, frequencies, measurements, amounts, etc.), the straightforward application of the original algorithm leads to build code vectors and to associate to each of them the class of all the individuals which are more similar to this code-vector than to the others. But, in case of individuals described by categorical (qualitative) variables having a finite number of modalities (like in a survey), it is necessary to define a specific algorithm. In this paper, we present a new algorithm inspired by the SOM algorithm, which provides a simultaneous classification of the individuals and of their modalities.Comment: Special issue ESANN 0

Understanding the Socio-Economic Distribution and Consequences of Patterns of Multiple Deprivation: An Application of Self-Organising Maps. ESRI WP302. June 2009

In this paper we apply self organising maps (SOM) to a detailed set of material deprivation indicators from the Irish component of European Union Community Statistics on Income and Living Conditions (EU-SILC). The first stage of our analysis involves the identification and description of sixteen clusters of multiple deprivation that allow us to provide a detailed account of such deprivation in contemporary Ireland. In going beyond this mapping stage, we consider both patterns of socio-economic differentiation in relation to cluster membership and the extent to which such membership contributes to our understanding of the manner in which individuals experience their economic circumstances. Our analysis makes clear the continuing importance of traditional forms of stratification relating to factors such as income, social class and housing tenure in accounting for patterns of multiple deprivation. However, it also confirms the role of acute life events and life cycle and location influences. It suggests that debates relating to the extent to which poverty and social exclusion have become individualized should take particular care to distinguish between different kinds of outcomes. Further analysis demonstrates that the SOM approach is considerably more successful than a comparable latent class analysis in identifying those exposed to subjective economic stress. This finding, combined with those relating to the role of socio-economic factors in accounting for cluster membership, confirms that a focus on a set of eight SOM macro clusters seems most appropriate if our interest lies in broad patterns stratification. For other purposes differentiation within clusters, which clearly takes a systematic form, may prove to be crucial