Crosslinguistic Image Schema Differential Hypothesis Clarifies Non-Prototypical and Polysemous Spatial Preposition ‘on’ for L2 Learners

A key question for linguistics involves how to determine and account for expressions of non-prototypical spatial relationships between languages. To address this issue, Crosslinguistic Image Schema Differential (CISD) hypothesis is introduced to examine various uses of the English preposition on produced by L2 (second language) learners. Data collection consisted of a grammar test designed to elicit and measure participants’ knowledge of the English preposition on by completing cloze sentences in English, translating these sentences into the L1 (first language), and then drawing visual images of the sentences presented as redescriptions of perceptual events, i.e., image schemas. The most remarkable findings were that two space-relational types (‘encirclement with contact’ and ‘at an edge’) and one image schema (‘concave surface’) were almost completely lacking in the Japanese learners of English (JLEs) who participated in this study. This investigation indicates that simple explicit explanations are possible utilizing the CISD hypothesis.アクセプト後にアブストラクトの変更あり

A Conceptual Framework for Modelling Spatial Relations

Various approaches lie behind the modelling of spatial relations, which is a heterogeneous and interdisciplinary field. In this paper, we introduce a conceptual framework to describe the characteristics of various models and how they relate each other. A first categorization is made among three representation levels: geometric, computational, and user. At the geometric level, spatial objects can be seen as point-sets and relations can be formally defined at the mathematical level. At the computational level, objects are represented as data types and relations are computed via spatial operators. At the user level, objects and relations belong to a context-dependent user ontology. Another way of providing a categorization is following the underlying geometric space that describes the relations: we distinguish among topologic, projective, and metric relations. Then, we consider the cardinality of spatial relations, which is defined as the number of objects that participate in the relation. Another issue is the granularity at which the relation is described, ranging from general descriptions to very detailed ones. We also consider the dimension of the various geometric objects and the embedding space as a fundamental way of categorizing relations

Identifying and modelling polysemous senses of spatial prepositions in referring expressions

In this paper we analyse the issue of reference using spatial language and examine how the polysemy exhibited by spatial prepositions can be incorporated into semantic models for situated dialogue. After providing a brief overview of polysemy in spatial language and a review of related work, we describe an experimental study we used to collect data on a set of relevant spatial prepositions. We then establish a semantic model in which to integrate polysemy (the Baseline Prototype Model), which we test against a Simple Relation Model and a Perceptron Model. To incorporate polysemy into the baseline model we introduce two methods of identifying polysemes in grounded settings. The first is based on ‘ideal meanings’ and a modification of the ‘principled polysemy’ framework and the second is based on ‘object-specific features’. In order to compare polysemes and aid typicality judgements we then introduce a notion of ‘polyseme hierarchy’. Finally, we test the performance of the polysemy models against the Baseline Prototype Model and Perceptron Model and discuss the improvements shown by the polysemy models

Context-sensitive interpretation of natural language location descriptions : a thesis submitted in partial fulfilment of the requirements for the award of Doctor of Philosophy in Information Technology at Massey University, Auckland, New Zealand

People frequently describe the locations of objects using natural language. Location descriptions may be either structured, such as 26 Victoria Street, Auckland, or unstructured. Relative location descriptions (e.g., building near Sky Tower) are a common form of unstructured location description, and use qualitative terms to describe the location of one object relative to another (e.g., near, close to, in, next to). Understanding the meaning of these terms is easy for humans, but much more difficult for machines since the terms are inherently vague and context sensitive. In this thesis, we study the semantics (or meaning) of qualitative, geospatial relation terms, specifically geospatial prepositions. Prepositions are one of the most common forms of geospatial relation term, and they are commonly used to describe the location of objects in the geographic (geospatial) environment, such as rivers, mountains, buildings, and towns. A thorough understanding of the semantics of geospatial relation terms is important because it enables more accurate automated georeferencing of text location descriptions than use of place names only. Location descriptions that use geospatial prepositions are found in social media, web sites, blogs, and academic reports, and georeferencing can allow mapping of health, disaster and biological data that is currently inaccessible to the public. Such descriptions have unstructured format, so, their analysis is not straightforward. The specific research questions that we address are: RQ1. Which geospatial prepositions (or groups of prepositions) and senses are semantically similar? RQ2. Is the role of context important in the interpretation of location descriptions? RQ3. Is the object distance associated with geospatial prepositions across a range of geospatial scenes and scales accurately predictable using machine learning methods? RQ4. Is human annotation a reliable form of annotation for the analysis of location descriptions? To address RQ1, we determine the nature and degree of similarity among geospatial prepositions by analysing data collected with a human subjects experiment, using clustering, extensional mapping and t-stochastic neighbour embedding (t-SNE) plots to form a semantic similarity matrix. In addition to calculating similarity scores among prepositions, we identify the senses of three groups of geospatial prepositions using Venn diagrams, t-sne plots and density-based clustering, and define the relationships between the senses. Furthermore, we use two text mining approaches to identify the degree of similarity among geospatial prepositions: bag of words and GloVe embeddings. By using these methods and further analysis, we identify semantically similar groups of geospatial prepositions including: 1- beside, close to, near, next to, outside and adjacent to; 2- across, over and through and 3- beyond, past, by and off. The prepositions within these groups also share senses. Through is recognised as a specialisation of both across and over. Proximity and adjacency prepositions also have similar senses that express orientation and overlapping relations. Past, off and by share a proximal sense but beyond has a different sense from these, representing on the other side. Another finding is the more frequent use of the preposition close to for pairs of linear objects than near, which is used more frequently for non-linear ones. Also, next to is used to describe proximity more than touching (in contrast to other prepositions like adjacent to). Our application of text mining to identify semantically similar prepositions confirms that a geospatial corpus (NCGL) provides a better representation of the semantics of geospatial prepositions than a general corpus. Also, we found that GloVe embeddings provide adequate semantic similarity measures for more specialised geospatial prepositions, but less so for those that have more generalised applications and multiple senses. We explore the role of context (RQ2) by studying three sites that vary in size, nature, and context in London: Trafalgar Square, Buckingham Palace, and Hyde Park. We use the Google search engine to extract location descriptions that contain these three sites with 9 different geospatial prepositions (in, on, at, next to, close to, adjacent to, near, beside, outside) and calculate their acceptance profiles (the profile of the use of a preposition at different distances from the reference object) and acceptance thresholds (maximum distance from a reference object at which a preposition can acceptably be used). We use these to compare prepositions, and to explore the influence of different contexts. Our results show that near, in and outside are used for larger distances, while beside, adjacent to and at are used for smaller distances. Also, the acceptance threshold for close to is higher than for other proximity/adjacency prepositions such as next to, adjacent to and beside. The acceptance threshold of next to is larger than adjacent to, which confirms the findings in ‎Chapter 2 which identifies next to describing a proximity rather than touching spatial relation. We also found that relatum characteristics such as image schema affect the use of prepositions such as in, on and at. We address RQ3 by developing a machine learning regression model (using the SMOReg algorithm) to predict the distance associated with use of geospatial prepositions in specific expressions. We incorporate a wide range of input variables including the similarity matrix of geospatial prepositions (RQ1); preposition senses; semantic information in the form of embeddings; characteristics of the located and reference objects in the expression including their liquidity/solidity, scale and geometry type and contextual factors such as the density of features of different types in the surrounding area. We evaluate the model on two different datasets with 25% improvement against the best baseline respectively. Finally, we consider the importance of annotation of geospatial location descriptions (RQ4). As annotated data is essential for the successful study of automated interpretation of natural language descriptions, we study the impact and accuracy of human annotation on different geospatial elements. Agreement scores show that human annotators can annotate geospatial relation terms (e.g., geospatial prepositions) with higher agreement than other geospatial elements. This thesis advances understanding of the semantics of geospatial prepositions, particularly considering their semantic similarity and the impact of context on their interpretation. We quantify the semantic similarity of a set of 24 geospatial prepositions; identify senses and the relationships among them for 13 geospatial prepositions; compare the acceptance thresholds of 9 geospatial prepositions and describe the influence of context on them; and demonstrate that richer semantic and contextual information can be incorporated in predictive models to interpret relative geospatial location descriptions more accurately

The Influence of Scale, Context and Spatial Preposition in Linguistic Topology

* (Author for correspondence) Abstract. Following a similar method to that of Mark and Egenhofer (1994), a questionnaire-based experiment tested for possible effects of scale, context and spatial relation type on the acceptability of spatial prepositions. The results suggest that the previous assumption of scale invariance in spatial language is incorrect. The physical world as experienced by humans, and described by human language, is not a fractal: scale appears to change its very physical nature, and hence the meaning of its spatial relations. The experiment demonstrated how scale influences preposition use, and how different prepositions appeared to evoke different levels of acceptability in themselves. Context, in terms of object type (solid or liquid), interacted with these factors to demonstrate specific constraints upon spatial language use. The results are discussed in terms of figure-ground relations, as well as the role of human experience and the classification of the world into 'objects ' in different ways at different scales. Since this was a preliminary and artificially-constrained experiment, the need for further research is emphasized.

Jeux pédagogiques collaboratifs situés (conception et mise en oeuvre dirigées par les modèles)

Un jeu pédagogique constitue une déclinaison relative à l apprentissage du concept de jeu sérieux (serious game). Ce type d'outil permet la ludification (gamification) de l'activité afin d'utiliser des éléments de jeu dans un contexte non ludique et conduit à catalyser l attention, faire accroître l engagement et augmenter la motivation des joueurs-apprenants dans les situations d apprentissage. Les jeux pédagogiques reposent sur la mise en situation et l immersion des apprenants, utilisant les ressorts ludiques dans des simulations axées vers la résolution de problèmes. Parmi des recherches antérieures, certains retours d expériences font écho d une trop grande artificialité de l activité notamment par manque de contextualisation de l apprentissage dans l environnement d utilisation des connaissances apprises. Nous avons proposé la mise en place un environnement mixte (physique et numérique) et l utilisation de techniques collaboratives pour raffiner l approche pédagogique. Ces orientations nous ont menés à la mise en place de ce que nous appelons des Jeux Pédagogiques Collaboratifs Situés (JPCS). Les deux questions de recherche qui nous ont été posées dans le cadre du projet SEGAREM et qui sont devenues les nôtres sont : 1/ comment accompagner les jeux sérieux par l approche Réalité Augmentée (RA) et l'approche Interface Tangible (IT)? 2/ comment rendre la conception et la mise en œuvre des JPCS (Jeux Pédagogiques Collaboratifs Situés) plus explicite et plus systématique ? Les réponses que nous présentons dans cette thèse sont les suivantes : 1/ la conception et la mise en œuvre des pupitres interactifs supportant des objets réels augmentés, associés à un protocole de communication existant, proposant un support générique des techniques d interaction détectée et de prise en compte du contexte physique d utilisation ; 2/ une approche de production de JPCS se situant après l étape de scénarisation ludo-pédagogique qui constitue notre cahier des charges. Nous avons basé notre approche sur des modèles pour permettre un support d expression qui précise les caractéristiques des JPCS. Ces modèles sont soutenus par des éditeurs contextuels et produisent comme résultat des fichiers de descriptions en XML. La projection des descriptions obtenues sur une architecture générique d exécution du JPCS permet une spécialisation pour obtenir une version exécutable. Dans les six modèles, certains sont adaptés des travaux antérieurs de l équipe, d'autres issues de la littérature et les derniers sont directement proposés ici. Ces six modèles décrivent l activité (un modèle d orchestration de l activité et un modèle de tâches), la structure de différents environnements, l état initial de l environnement et les conditions nécessaires d un état final et les interactions possibles entre les joueurs et l environnement. Nos travaux tant sur les pupitres que sur les modèles et le support d exécution ont été concrétisés dans la mise en place de Lea(r)nIt. Ce JPCS avait pour but de consolider des acquis méthodologiques en Lean Manufacturing par l utilisation et l optimisation d une chaîne de production simulée sur pupitres (supportant interactions tactiles, interactions tangibles et pouvant être assemblés) et sur téléphones mobiles (permettant la mobilité des joueurs-apprenants).A Learning game is a declension of the serious game concept dedicated to the learning activity. A Learning game is based on a scenario and immersion of the learners with use of game mechanics on problem based simulation. The gamification concept is the use of game elements in a non-playful activity with as impact attention, motivation and engagement. However, some research feedback explains that too much artificiality on learning activity caused by a lack of contextualization of the activity on the professional environment. We propose to use Mixed Reality and Collaborative Supported Computer Work as technological solution to support situated and collaborative situation in aim to enhance pedagogical strategy and allow a better learning. We call it Situated Collaborative Learning Game (SCLG) as a concept of pedagogical tools to enhance learning of content with use of collaborative learning (when learners interactions is useful to learn), situated learning (when the environment context is meaningful) and human-physical objet interaction (with use of mixed reality, with kinesthetic and tangible interaction in augmented reality) and game based learning (when learner's motivation is improved by the learning activity). In these contexts, our two research questions are: 1 / How to create a serious games support by use of Augmented Reality (AR) approach and Tangible Interface (IT) approach? 2 / How to make design and development of SCLG (situated collaborative learning game) more explicit and systematic? We propose two solutions: 1/ the design and the production of four interactive desks with support of tangible interaction on and above the table. These devices are linked to a communication protocol which allows a generic support of technical interaction. 2/ A generic way to design the CSLG system, with integration of advanced human computer interaction support (as augmented reality and tangible interaction) and ubiquitous computing in Learning Games. For that, we propose, with a user centered oriented and model oriented design, a way to make a CSLG factory. For that, we propose use of six models to determinate the behavior of the CSLG. These six models describe learners activity (with use of three different models to follow the activity theory s), the mixed game environment, deployment of entities on the environment, and human computer interactions. All of these models are linked by an orchestration model and can be project on a multi-agent multi-layers architecture by use of XML description file. We propose tools to help each step of our design and production process. Our work on interactive desks, on the six models and on the runtime support has been realized in the production of Lea(r)nIT. This SCLG consolidate methodological knowledge of Lean Manufacturing by use and optimization of a simulated chain production on four desks (which support touch and tangible interactions and can be assembled) and on mobile phones (to allow movement of learners).LYON-Ecole Centrale (690812301) / SudocSudocFranceF

Reference Object Choice in Spatial Language: Machine and Human Models

The thesis underpinning this study is as follows; it is possible to build machine models that are indistinguishable from the mental models used by humans to generate language to describe their environment. This is to say that the machine model should perform in such a way that a human listener could not discern whether a description of a scene was generated by a human or by the machine model. Many linguistic processes are used to generate even simple scene descriptions and developing machine models of all of them is beyond the scope of this study. The goal of this study is, therefore, to model a sufficient part of the scene description process, operating in a sufficiently realistic environment, so that the likelihood of being able to build machine models of the remaining processes, operating in the real world, can be established. The relatively under-researched process of reference object selection is chosen as the focus of this study. A reference object is, for instance, the `table' in the phrase ``The flowers are on the table''. This study demonstrates that the reference selection process is of similar complexity to others involved in generating scene descriptions which include: assigning prepositions, selecting reference frames and disambiguating objects (usually termed `generating referring expressions'). The secondary thesis of this study is therefore; it is possible to build a machine model that is indistinguishable from the mental models used by humans in selecting reference objects. Most of the practical work in the study is aimed at establishing this. An environment sufficiently near to the real-world for the machine models to operate on is developed as part of this study. It consists of a series of 3-dimensional scenes containing multiple objects that are recognisable to humans and `readable' by the machine models. The rationale for this approach is discussed. The performance of human subjects in describing this environment is evaluated, and measures by which the human performance can be compared to the performance of the machine models are discussed. The machine models used in the study are variants on Bayesian networks. A new approach to learning the structure of a subset of Bayesian networks is presented. Simple existing Bayesian classifiers such as naive or tree augmented naive networks did not perform sufficiently well. A significant result of this study is that useful machine models for reference object choice are of such complexity that a machine learning approach is required. Earlier proposals based on sum-of weighted-factors or similar constructions will not produce satisfactory models. Two differently derived sets of variables are used and compared in this study. Firstly variables derived from the basic geometry of the scene and the properties of objects are used. Models built from these variables match the choice of reference of a group of humans some 73\% of the time, as compared with 90\% for the median human subject. Secondly variables derived from `ray casting' the scene are used. Ray cast variables performed much worse than anticipated, suggesting that humans use object knowledge as well as immediate perception in the reference choice task. Models combining geometric and ray-cast variables match the choice of reference of the group of humans some 76\% of the time. Although niether of these machine models are likely to be indistinguishable from a human, the reference choices are rarely, if ever, entirely ridiculous. A secondary goal of the study is to contribute to the understanding of the process by which humans select reference objects. Several statistically significant results concerning the necessary complexity of the human models and the nature of the variables within them are established. Problems that remain with both the representation of the near-real-world environment and the Bayesian models and variables used within them are detailed. While these problems cast some doubt on the results it is argued that solving these problems is possible and would, on balance, lead to improved performance of the machine models. This further supports the assertion that machine models producing reference choices indistinguishable from those of humans are possible