To dash or to dawdle: verb-associated speed of motion influences eye movements during spoken sentence comprehension

In describing motion events verbs of manner provide information about the speed of agents or objects in those events. We used eye tracking to investigate how inferences about this verb-associated speed of motion would influence the time course of attention to a visual scene that matched an event described in language. Eye movements were recorded as participants heard spoken sentences with verbs that implied a fast (“dash”) or slow (“dawdle”) movement of an agent towards a goal. These sentences were heard whilst participants concurrently looked at scenes depicting the agent and a path which led to the goal object. Our results indicate a mapping of events onto the visual scene consistent with participants mentally simulating the movement of the agent along the path towards the goal: when the verb implies a slow manner of motion, participants look more often and longer along the path to the goal; when the verb implies a fast manner of motion, participants tend to look earlier at the goal and less on the path. These results reveal that event comprehension in the presence of a visual world involves establishing and dynamically updating the locations of entities in response to linguistic descriptions of events

Natural Language Generation and Fuzzy Sets : An Exploratory Study on Geographical Referring Expression Generation

This work was supported by the Spanish Ministry for Economy and Competitiveness (grant TIN2014-56633-C3-1-R) and by the European Regional Development Fund (ERDF/FEDER) and the Galician Ministry of Education (grants GRC2014/030 and CN2012/151). Alejandro Ramos-Soto is supported by the Spanish Ministry for Economy and Competitiveness (FPI Fellowship Program) under grant BES-2012-051878.Postprin

Latent Variable Model for Multi-modal Translation

In this work, we propose to model the interaction between visual and textual features for multi-modal neural machine translation (MMT) through a latent variable model. This latent variable can be seen as a multi-modal stochastic embedding of an image and its description in a foreign language. It is used in a target-language decoder and also to predict image features. Importantly, our model formulation utilises visual and textual inputs during training but does not require that images be available at test time. We show that our latent variable MMT formulation improves considerably over strong baselines, including a multi-task learning approach (Elliott and K\'ad\'ar, 2017) and a conditional variational auto-encoder approach (Toyama et al., 2016). Finally, we show improvements due to (i) predicting image features in addition to only conditioning on them, (ii) imposing a constraint on the minimum amount of information encoded in the latent variable, and (iii) by training on additional target-language image descriptions (i.e. synthetic data).Comment: Paper accepted at ACL 2019. Contains 8 pages (11 including references, 13 including appendix), 6 figure

The Integration of Cadastral Base Mapping with Cadastral Parcel Attribution

A cadastre is a parcel-based, up-to-date land information system containing a record of interests in land. Creation and maintenance of a cadastre usually involves coordination between different public and private organizations that are responsible for the various data. The U.S. Bureau of Land Management (BLM) has built a Geographic Coordinate Data Base (GCDB) that currently provides cadastral base map data for more than 38,000 townships across the country, with many of the western states nearly complete. The GCDB strategy is that the coordinates can and do change as more recent and accurate information becomes available. The locational reliability of the GCDB as a digital representation of the Public Lands Survey System is widely recognized. This thesis examines issues in building upon this framework for the depiction of the local parcels as a core component of the national cadastre, maintainable at the local government level, such as a municipality or county. As new data in the federal base framework are provided, the local parcel fabric may need to be updated without creating gaps and overlaps. The measurement management methodology has been expanded to provide this maintenance capability. This ultimately leads to the desired political outcome of a consistent, reliable, spatial representation of legal land objects. The legal land descriptions encoded in the GCDB framework can be extracted and utilized to provide consistent parcel attribution of aliquot part parcels. As most states have external databases that maintain an index of real property parcels. Experimentation identifies that integration with these external databases is an extremely accurate, expedient, and cost-effective method of cadastral parcel attribution at the state or local government level, depicted on a uniform parcel-based map. The methodology presented yields great success in automatic identification and interpretation of encoded legal land descriptions of aliquot part parcels. Building upon the FGDC Cadastral Data Content Standard, expansion of this can lead to automatic parcel identification and attribution as high as 96% in some areas

Communicative success in spatial dialogue: The impact of functional features and dialogue strategies

This paper addresses the impact of dialogue strategies and functional features of spatial arrangements on communicative success. To examine the sharing of cognition between two minds in order to achieve a joint goal, we collected a corpus of 24 extended German-language dialogues in a referential communication task that involved furnishing a dolls’ house. Results show how successful communication, as evidenced by correct placement of furniture items, is affected by a) functionality of the furniture arrangement, b) previous task experience, and c) dialogue features such as description length and orientation information. To enhance research in this area, our 'Dolldialogue' corpus is now available as a free resource on www.dolldialogue.spac

Mind the Gap: Another look at the problem of the semantic gap in image retrieval

This paper attempts to review and characterise the problem of the semantic gap in image retrieval and the attempts being made to bridge it. In particular, we draw from our own experience in user queries, automatic annotation and ontological techniques. The first section of the paper describes a characterisation of the semantic gap as a hierarchy between the raw media and full semantic understanding of the media's content. The second section discusses real users' queries with respect to the semantic gap. The final sections of the paper describe our own experience in attempting to bridge the semantic gap. In particular we discuss our work on auto-annotation and semantic-space models of image retrieval in order to bridge the gap from the bottom up, and the use of ontologies, which capture more semantics than keyword object labels alone, as a technique for bridging the gap from the top down

Identifying and inferring objects from textual descriptions of scenes from books

Fiction authors rarely provide detailed descriptions of scenes, preferring the reader to fill in the details using their imagination. Therefore, to perform detailed text-to-scene conversion from books, we need to not only identify explicit objects but also infer implicit objects. In this paper, we describe an approach to inferring objects using Wikipedia and WordNet. In our experiments, we are able to infer implicit objects such as monitor and computer by identifying explicit objects such as keyboard