3,736 research outputs found

    Learning with Latent Language

    Full text link
    The named concepts and compositional operators present in natural language provide a rich source of information about the kinds of abstractions humans use to navigate the world. Can this linguistic background knowledge improve the generality and efficiency of learned classifiers and control policies? This paper aims to show that using the space of natural language strings as a parameter space is an effective way to capture natural task structure. In a pretraining phase, we learn a language interpretation model that transforms inputs (e.g. images) into outputs (e.g. labels) given natural language descriptions. To learn a new concept (e.g. a classifier), we search directly in the space of descriptions to minimize the interpreter's loss on training examples. Crucially, our models do not require language data to learn these concepts: language is used only in pretraining to impose structure on subsequent learning. Results on image classification, text editing, and reinforcement learning show that, in all settings, models with a linguistic parameterization outperform those without

    Compact Personalized Models for Neural Machine Translation

    Full text link
    We propose and compare methods for gradient-based domain adaptation of self-attentive neural machine translation models. We demonstrate that a large proportion of model parameters can be frozen during adaptation with minimal or no reduction in translation quality by encouraging structured sparsity in the set of offset tensors during learning via group lasso regularization. We evaluate this technique for both batch and incremental adaptation across multiple data sets and language pairs. Our system architecture - combining a state-of-the-art self-attentive model with compact domain adaptation - provides high quality personalized machine translation that is both space and time efficient.Comment: Published at the 2018 Conference on Empirical Methods in Natural Language Processin

    Joint prediction of travel mode choice and purpose from travel surveys: A multitask deep learning approach

    Get PDF
    The prediction and behavioural analysis of travel mode choice and purpose are critical for transport planning and have attracted increasing interest in research. Traditionally, the prediction of travel mode choice and trip purpose has been tackled separately, which fail to fully leverage the shared information between travel mode and purpose. This study addresses this gap by proposing a multitask learning deep neural network framework (MTLDNN) to jointly predict mode choice and purpose. We empirically evaluate and validate this framework using the household travel survey data in Greater London, UK. The results show that this framework has significantly lower cross-entropy loss than multinomial logit models (MNL) and single-task-learning deep neural network models (STLDNN). On the other hand, the predictive accuracy of MTLDNN is similar to STLDNN and is significantly higher than MNL. Moreover, in terms of behaviour analysis, the substitution pattern and choice probability of MTLDNN regarding input variables largely agree with MNL and STLDNN. This work demonstrates that MTLDNN is efficient in utilising the information shared by travel mode choice and purpose, and is capable of producing behaviourally reasonable substitution patterns across travel modes. Future research would develop more advanced MTLDNN frameworks for travel behaviour analysis and generalise MTLDNN to other travel behaviour topics
    • …
    corecore