Crowd-sourcing NLG Data: Pictures Elicit Better Data

Recent advances in corpus-based Natural Language Generation (NLG) hold the promise of being easily portable across domains, but require costly training data, consisting of meaning representations (MRs) paired with Natural Language (NL) utterances. In this work, we propose a novel framework for crowdsourcing high quality NLG training data, using automatic quality control measures and evaluating different MRs with which to elicit data. We show that pictorial MRs result in better NL data being collected than logic-based MRs: utterances elicited by pictorial MRs are judged as significantly more natural, more informative, and better phrased, with a significant increase in average quality ratings (around 0.5 points on a 6-point scale), compared to using the logical MRs. As the MR becomes more complex, the benefits of pictorial stimuli increase. The collected data will be released as part of this submission.Comment: The 9th International Natural Language Generation conference INLG, 2016. 10 pages, 2 figures, 3 table

Verbal Learning and Memory After Cochlear Implantation in Postlingually Deaf Adults: Some New Findings with the CVLT-II

OBJECTIVES: Despite the importance of verbal learning and memory in speech and language processing, this domain of cognitive functioning has been virtually ignored in clinical studies of hearing loss and cochlear implants in both adults and children. In this article, we report the results of two studies that used a newly developed visually based version of the California Verbal Learning Test-Second Edition (CVLT-II), a well-known normed neuropsychological measure of verbal learning and memory. DESIGN: The first study established the validity and feasibility of a computer-controlled visual version of the CVLT-II, which eliminates the effects of audibility of spoken stimuli, in groups of young normal-hearing and older normal-hearing (ONH) adults. A second study was then carried out using the visual CVLT-II format with a group of older postlingually deaf experienced cochlear implant (ECI) users (N = 25) and a group of ONH controls (N = 25) who were matched to ECI users for age, socioeconomic status, and nonverbal IQ. In addition to the visual CVLT-II, subjects provided data on demographics, hearing history, nonverbal IQ, reading fluency, vocabulary, and short-term memory span for visually presented digits. ECI participants were also tested for speech recognition in quiet. RESULTS: The ECI and ONH groups did not differ on most measures of verbal learning and memory obtained with the visual CVLT-II, but deficits were identified in ECI participants that were related to recency recall, the buildup of proactive interference, and retrieval-induced forgetting. Within the ECI group, nonverbal fluid IQ, reading fluency, and resistance to the buildup of proactive interference from the CVLT-II consistently predicted better speech recognition outcomes. CONCLUSIONS: Results from this study suggest that several underlying foundational neurocognitive abilities are related to core speech perception outcomes after implantation in older adults. Implications of these findings for explaining individual differences and variability and predicting speech recognition outcomes after implantation are discussed

Improving Semantic Embedding Consistency by Metric Learning for Zero-Shot Classification

This paper addresses the task of zero-shot image classification. The key contribution of the proposed approach is to control the semantic embedding of images -- one of the main ingredients of zero-shot learning -- by formulating it as a metric learning problem. The optimized empirical criterion associates two types of sub-task constraints: metric discriminating capacity and accurate attribute prediction. This results in a novel expression of zero-shot learning not requiring the notion of class in the training phase: only pairs of image/attributes, augmented with a consistency indicator, are given as ground truth. At test time, the learned model can predict the consistency of a test image with a given set of attributes , allowing flexible ways to produce recognition inferences. Despite its simplicity, the proposed approach gives state-of-the-art results on four challenging datasets used for zero-shot recognition evaluation.Comment: in ECCV 2016, Oct 2016, amsterdam, Netherlands. 201

Multimodal person recognition for human-vehicle interaction

Next-generation vehicles will undoubtedly feature biometric person recognition as part of an effort to improve the driving experience. Today's technology prevents such systems from operating satisfactorily under adverse conditions. A proposed framework for achieving person recognition successfully combines different biometric modalities, borne out in two case studies