128,674 research outputs found
Generating Synthetic Data for Neural Keyword-to-Question Models
Search typically relies on keyword queries, but these are often semantically
ambiguous. We propose to overcome this by offering users natural language
questions, based on their keyword queries, to disambiguate their intent. This
keyword-to-question task may be addressed using neural machine translation
techniques. Neural translation models, however, require massive amounts of
training data (keyword-question pairs), which is unavailable for this task. The
main idea of this paper is to generate large amounts of synthetic training data
from a small seed set of hand-labeled keyword-question pairs. Since natural
language questions are available in large quantities, we develop models to
automatically generate the corresponding keyword queries. Further, we introduce
various filtering mechanisms to ensure that synthetic training data is of high
quality. We demonstrate the feasibility of our approach using both automatic
and manual evaluation. This is an extended version of the article published
with the same title in the Proceedings of ICTIR'18.Comment: Extended version of ICTIR'18 full paper, 11 page
EIGEN: Ecologically-Inspired GENetic Approach for Neural Network Structure Searching from Scratch
Designing the structure of neural networks is considered one of the most
challenging tasks in deep learning, especially when there is few prior
knowledge about the task domain. In this paper, we propose an
Ecologically-Inspired GENetic (EIGEN) approach that uses the concept of
succession, extinction, mimicry, and gene duplication to search neural network
structure from scratch with poorly initialized simple network and few
constraints forced during the evolution, as we assume no prior knowledge about
the task domain. Specifically, we first use primary succession to rapidly
evolve a population of poorly initialized neural network structures into a more
diverse population, followed by a secondary succession stage for fine-grained
searching based on the networks from the primary succession. Extinction is
applied in both stages to reduce computational cost. Mimicry is employed during
the entire evolution process to help the inferior networks imitate the behavior
of a superior network and gene duplication is utilized to duplicate the learned
blocks of novel structures, both of which help to find better network
structures. Experimental results show that our proposed approach can achieve
similar or better performance compared to the existing genetic approaches with
dramatically reduced computation cost. For example, the network discovered by
our approach on CIFAR-100 dataset achieves 78.1% test accuracy under 120 GPU
hours, compared to 77.0% test accuracy in more than 65, 536 GPU hours in [35].Comment: CVPR 201
GinJinn: An object‐detection pipeline for automated feature extraction from herbarium specimens
Premise
The generation of morphological data in evolutionary, taxonomic, and ecological studies of plants using herbarium material has traditionally been a labor‐intensive task. Recent progress in machine learning using deep artificial neural networks (deep learning) for image classification and object detection has facilitated the establishment of a pipeline for the automatic recognition and extraction of relevant structures in images of herbarium specimens.
Methods and Results
We implemented an extendable pipeline based on state‐of‐the‐art deep‐learning object‐detection methods to collect leaf images from herbarium specimens of two species of the genus Leucanthemum . Using 183 specimens as the training data set, our pipeline extracted one or more intact leaves in 95% of the 61 test images.
Conclusions
We establish GinJinn as a deep‐learning object‐detection tool for the automatic recognition and extraction of individual leaves or other structures from herbarium specimens. Our pipeline offers greater flexibility and a lower entrance barrier than previous image‐processing approaches based on hand‐crafted features
From Imitation to Prediction, Data Compression vs Recurrent Neural Networks for Natural Language Processing
In recent studies [1][13][12] Recurrent Neural Networks were used for
generative processes and their surprising performance can be explained by their
ability to create good predictions. In addition, data compression is also based
on predictions. What the problem comes down to is whether a data compressor
could be used to perform as well as recurrent neural networks in natural
language processing tasks. If this is possible,then the problem comes down to
determining if a compression algorithm is even more intelligent than a neural
network in specific tasks related to human language. In our journey we
discovered what we think is the fundamental difference between a Data
Compression Algorithm and a Recurrent Neural Network
TextGAIL: Generative Adversarial Imitation Learning for Text Generation
Generative Adversarial Networks (GANs) for text generation have recently
received many criticisms, as they perform worse than their MLE counterparts. We
suspect previous text GANs' inferior performance is due to the lack of a
reliable guiding signal in their discriminators. To address this problem, we
propose a generative adversarial imitation learning framework for text
generation that uses large pre-trained language models to provide more reliable
reward guidance. Our approach uses contrastive discriminator, and proximal
policy optimization (PPO) to stabilize and improve text generation performance.
For evaluation, we conduct experiments on a diverse set of unconditional and
conditional text generation tasks. Experimental results show that TextGAIL
achieves better performance in terms of both quality and diversity than the MLE
baseline. We also validate our intuition that TextGAIL's discriminator
demonstrates the capability of providing reasonable rewards with an additional
task.Comment: AAAI 202
- …