Recovering from failure by asking for help

Robots inevitably fail, often without the ability to recover autonomously. We demonstrate an approach for enabling a robot to recover from failures by communicating its need for specific help to a human partner using natural language. Our approach automatically detects failures, then generates targeted spoken-language requests for help such as “Please give me the white table leg that is on the black table.” Once the human partner has repaired the failure condition, the system resumes full autonomy. We present a novel inverse semantics algorithm for generating effective help requests. In contrast to forward semantic models that interpret natural language in terms of robot actions and perception, our inverse semantics algorithm generates requests by emulating the human’s ability to interpret a request using the Generalized Grounding Graph (G[superscript 3]) framework. To assess the effectiveness of our approach, we present a corpus-based online evaluation, as well as an end-to-end user study, demonstrating that our approach increases the effectiveness of human interventions compared to static requests for help.Boeing CompanyU.S. Army Research Laboratory (Robotics Collaborative Technology Alliance

Contrastive prosody and the subsequent mention of alternatives during discourse processing

Linguistic research has long viewed prosody as an important indicator of information structure in intonationally rich languages like English. Correspondingly, numerous psycholinguistic studies have shown significant effects of prosody, particularly with respect to the immediate processing of a prosodically prominent phrase. Although co-reference resolution is known to be influenced by information structure, it has been less clear whether prosodic prominence can affect decisions about next mention in a discourse, and if so, how. We present results from an open-ended story continuation task, conducted as part of a series of experiments that examine how prosody influences the anticipation and resolution of co-reference. Overall results from the project suggest that prosodic prominence can increase or decrease reference to a saliently pitch-accented phrase, depending on additional circumstances of the referential decision. We argue that an adequate account of prosody’s role in co-reference requires consideration of how the processing system interfaces with multiple levels of linguistic representation

Where Snow is a Landmark: Route Direction Elements in Alpine Contexts

Route directions research has mostly focused on urban space so far, highlighting human concepts of street networks based on a range of recurring elements such as route segments, decision points, landmarks and actions. We explored the way route directions reflect the features of space and activity in the context of mountaineering. Alpine route directions are only rarely segmented through decision points related to reorientation; instead, segmentation is based on changing topography. Segments are described with various degrees of detail, depending on difficulty. For landmark description, direction givers refer to properties such as type of surface, dimension, colour of landscape features; terrain properties (such as snow) can also serve as landmarks. Action descriptions reflect the geometrical conceptualization of landscape features and dimensionality of space. Further, they are very rich in the semantics of manner of motion

Language development after cochlear implantation: an epigenetic model

Growing evidence supports the notion that dynamic gene expression, subject to epigenetic control, organizes multiple influences to enable a child to learn to listen and to talk. Here, we review neurobiological and genetic influences on spoken language development in the context of results of a longitudinal trial of cochlear implantation of young children with severe to profound sensorineural hearing loss in the Childhood Development after Cochlear Implantation study. We specifically examine the results of cochlear implantation in participants who were congenitally deaf (N = 116). Prior to intervention, these participants were subject to naturally imposed constraints in sensory (acoustic–phonologic) inputs during critical phases of development when spoken language skills are typically achieved rapidly. Their candidacy for a cochlear implant was prompted by delays (n = 20) or an essential absence of spoken language acquisition (n = 96). Observations thus present an opportunity to evaluate the impact of factors that influence the emergence of spoken language, particularly in the context of hearing restoration in sensitive periods for language acquisition. Outcomes demonstrate considerable variation in spoken language learning, although significant advantages exist for the congenitally deaf children implanted prior to 18 months of age. While age at implantation carries high predictive value in forecasting performance on measures of spoken language, several factors show significant association, particularly those related to parent–child interactions. Importantly, the significance of environmental variables in their predictive value for language development varies with age at implantation. These observations are considered in the context of an epigenetic model in which dynamic genomic expression can modulate aspects of auditory learning, offering insights into factors that can influence a child’s acquisition of spoken language after cochlear implantation. Increased understanding of these interactions could lead to targeted interventions that interact with the epigenome to influence language outcomes with intervention, particularly in periods in which development is subject to time-sensitive experience