Predefined Sparseness in Recurrent Sequence Models

Inducing sparseness while training neural networks has been shown to yield models with a lower memory footprint but similar effectiveness to dense models. However, sparseness is typically induced starting from a dense model, and thus this advantage does not hold during training. We propose techniques to enforce sparseness upfront in recurrent sequence models for NLP applications, to also benefit training. First, in language modeling, we show how to increase hidden state sizes in recurrent layers without increasing the number of parameters, leading to more expressive models. Second, for sequence labeling, we show that word embeddings with predefined sparseness lead to similar performance as dense embeddings, at a fraction of the number of trainable parameters.Comment: the SIGNLL Conference on Computational Natural Language Learning (CoNLL, 2018

L-Py, an open L-systems framework in Python

International audienceL-systems were conceived as a mathematical framework for modeling growth of plants. In this paper, we present L-Py, a simulation software that mixes L-systems construction with the Python high-level modeling language. In addition to this software module, an integrated visual development environment has been developed that facilitates the creation of plant models. In particular, easy to use optimization tools have been integrated. Thanks to Python and its modular approach, this framework makes it possible to integrate a variety of tools defined in different modeling context, in particular tools from the OpenAlea platform. Additionally, it can be integrated as a simple growth simulation module into more complex computational pipelines

Physics-based characterization of soft marine sediments using vector sensors

In a 2007 experiment conducted in the northern North Sea, observations of a low-frequency seismo-acoustic wave field with a linear horizontal array of vector sensors located on the seafloor revealed a strong, narrow peak around 38 Hz in the power spectra and a presence of multi-mode horizontally and vertically polarized interface waves with phase speeds between 45 and 350 m/s. Dispersion curves of the interface waves exhibit piece-wise linear dependences between the logarithm of phase speed and logarithm of frequency with distinct slopes at large and small phase speeds, which suggests a seabed with a power-law shear speed dependence in two distinct sediment layers. The power spectrum peak is interpreted as a manifestation of a seismo-acoustic resonance. A simple geoacoustic model with a few free parameters is derived that quantitatively reproduces the key features of the observations. This article's approach to the inverse problem is guided by a theoretical analysis of interface wave dispersion and resonance reflection of compressional waves in soft marine sediments containing two or more layers of different composition. Combining data from various channels of the vector sensors is critical for separating waves of different polarizations and helps to identify various arrivals, check consistency of inversions, and evaluate sediment density

Competition-based Model of Pruning: Applications to Apple Trees

International audienceno abstrac

The normalized freebase distance

In this paper, we propose the Normalized Freebase Distance (NFD), a new measure for determing semantic concept relatedness that is based on similar principles as the Normalized Web Distance (NWD). We illustrate that the NFD is more effective when comparing ambiguous concepts

L-Py: An L-System Simulation Framework for Modeling Plant Architecture Development Based on a Dynamic Language

The study of plant development requires increasingly powerful modeling tools to help understand and simulate the growth and functioning of plants. In the last decade, the formalism of L-systems has emerged as a major paradigm for modeling plant development. Previous implementations of this formalism were made based on static languages, i.e., languages that require explicit definition of variable types before using them. These languages are often efficient but involve quite a lot of syntactic overhead, thus restricting the flexibility of use for modelers. In this work, we present an adaptation of L-systems to the Python language, a popular and powerful open-license dynamic language. We show that the use of dynamic language properties makes it possible to enhance the development of plant growth models: (i) by keeping a simple syntax while allowing for high-level programming constructs, (ii) by making code execution easy and avoiding compilation overhead, (iii) by allowing a high-level of model reusability and the building of complex modular models, and (iv) by providing powerful solutions to integrate MTG data-structures (that are a common way to represent plants at several scales) into L-systems and thus enabling to use a wide spectrum of computer tools based on MTGs developed for plant architecture. We then illustrate the use of L-Py in real applications to build complex models or to teach plant modeling in the classroom

Reconstruction-free sensitive wavefront sensor based on continuous position sensitive detectors

International audienceWe propose a new device that is able to perform highly sensitive wavefront measurements based on the use of continuous position sensitive detectors and without resorting to any reconstruction process. We demonstrate experimentally its ability to measure small wavefront distortions through the characterization of pump-induced refractive index changes in laser material. In addition, it is shown using computer-generated holograms that this device can detect phase discontinuities as well as improve the quality of sharp phase variations measurements. Results are compared to reference Shack-Hartmann measurements, and dramatic enhancements are obtained

Explaining Character-Aware Neural Networks for Word-Level Prediction: Do They Discover Linguistic Rules?

Character-level features are currently used in different neural network-based natural language processing algorithms. However, little is known about the character-level patterns those models learn. Moreover, models are often compared only quantitatively while a qualitative analysis is missing. In this paper, we investigate which character-level patterns neural networks learn and if those patterns coincide with manually-defined word segmentations and annotations. To that end, we extend the contextual decomposition technique (Murdoch et al. 2018) to convolutional neural networks which allows us to compare convolutional neural networks and bidirectional long short-term memory networks. We evaluate and compare these models for the task of morphological tagging on three morphologically different languages and show that these models implicitly discover understandable linguistic rules. Our implementation can be found at https://github.com/FredericGodin/ContextualDecomposition-NLP .Comment: Accepted at EMNLP 201

A Simple Geometric Method for Cross-Lingual Linguistic Transformations with Pre-trained Autoencoders

Powerful sentence encoders trained for multiple languages are on the rise. These systems are capable of embedding a wide range of linguistic properties into vector representations. While explicit probing tasks can be used to verify the presence of specific linguistic properties, it is unclear whether the vector representations can be manipulated to indirectly steer such properties. We investigate the use of a geometric mapping in embedding space to transform linguistic properties, without any tuning of the pre-trained sentence encoder or decoder. We validate our approach on three linguistic properties using a pre-trained multilingual autoencoder and analyze the results in both monolingual and cross-lingual settings