Crossmodal content binding in information-processing architectures

Operating in a physical context, an intelligent robot faces two fundamental problems. First, it needs to combine information from its different sensors to form a representation of the environment that is more complete than any of its sensors on its own could provide. Second, it needs to combine high-level representations (such as those for planning and dialogue) with its sensory information, to ensure that the interpretations of these symbolic representations are grounded in the situated context. Previous approaches to this problem have used techniques such as (low-level) information fusion, ontological reasoning, and (high-level) concept learning. This paper presents a framework in which these, and other approaches, can be combined to form a shared representation of the current state of the robot in relation to its environment and other agents. Preliminary results from an implemented system are presented to illustrate how the framework supports behaviours commonly required of an intelligent robot

Towards an Indexical Model of Situated Language Comprehension for Cognitive Agents in Physical Worlds

We propose a computational model of situated language comprehension based on the Indexical Hypothesis that generates meaning representations by translating amodal linguistic symbols to modal representations of beliefs, knowledge, and experience external to the linguistic system. This Indexical Model incorporates multiple information sources, including perceptions, domain knowledge, and short-term and long-term experiences during comprehension. We show that exploiting diverse information sources can alleviate ambiguities that arise from contextual use of underspecific referring expressions and unexpressed argument alternations of verbs. The model is being used to support linguistic interactions in Rosie, an agent implemented in Soar that learns from instruction.Comment: Advances in Cognitive Systems 3 (2014

Bootstrapping Probabilistic Models of Qualitative Spatial Relations for Active Visual Object Search

In many real world applications, autonomous mobile robots are required to observe or retrieve objects in their environment, despite not having accurate estimates of the objects ’ locations. Finding objects in real-world settings is a non-trivial task, given the complexity and the dynamics of human environments. However, by understanding and exploiting the structure of such environments, e.g. where objects are commonly placed as part of everyday activities, robots can perform search tasks more efficiently and effectively than without such knowledge. In this paper we investigate how probabilistic models of qualitative spatial relations can improve the performance in object search tasks. Specifically, we learn Gaussian Mixture Models of spatial relations between object classes from descriptive statistics of real office environments. Experimental results with a range of sensor models suggest that our model improves overall performance in object search tasks.

Qualitative kinematics of planar robots: Intelligent connection

AbstractThis paper proposes a qualitative representation for robot kinematics in order to close the gap, raised by the perception–action problem, with a focus on intelligent connection of qualitative states to their corresponding numeric data in a robotic system. First, qualitative geometric primitives are introduced by combining a qualitative orientation component and qualitative translation component using normalisation techniques. A position in Cartesian space can be mathematically described by the scalable primitives. Secondly, qualitative robot kinematics of an n-link planar robot is derived in terms of the qualitative geometry primitives. Finally, it shows how to connect quantitativeness and qualitativeness of a robotic system. On the one hand, the integration of normalisation and domain knowledge generates normalised labels to introduce the meaningful parameters into the proposed representation. On the other hand, the normalised labels of this representation can be converted to a quantitative description using aggregation operators, whose numeric outputs can be used to generate desired trajectories based on mature interpolation techniques

Care in digital farming - from acting on to living with

Development of digital technology to handle complex situations in agriculture hasfor long time mainly been technology driven, resulting in limited adoption. Thisthesis aims to: 1) Introduce methods and theories from the research field of humancomputerinteraction in the agricultural domain to improve design and developmentprocesses of digital technology. 2) Introduce the concept of care to increaseknowledge about farmers' technology use in their socio-technical system (practice),as well as to introduce a relational perspective in agriculture. The two systemicallydescribed complex decision situations are fertilization with a decision supportsystem, that uses satellite images and automated milking systems. 3) Evaluate twodifferent theoretical lenses to study the concept of care in practice, DistributedCognition and Activity Theory. The studies of farmers' socio-technical systemsshow that farmers develop an enhanced professional vision to interpret data from thetechnology and learn more about the field/crop or the cow. New technology changesthe relationship between the farmer and the field/crop or cow, but the experiencedfarmer supplements what they see through the technology with direct contact with,for example, the cow. The need for a stockperson’s eye is thus at least as great afterthe introduction of robots in milk production. A relational perspective involves anunderstanding of our mutual dependence with the crop or the cow in these examples,as well as nature and its ecosystem services. Introduction of the concept of care anda relational approach, meaning that farming is to live with, not just act on, cansupport the transformation of agriculture that we know is necessary. In thistransformational process, technology has an important role to play. However, it mustbe developed in cooperation and dialog with end-users to fit in their socio-technicalecologicalsystem and thus support their care

Self-beliefs in the introductory programming lab and game-based fantasy role-play

This thesis was submitted for the degree of Doctor of philosophy and awarded by Brunel University LondonIt is important for students to engage in adequate deliberate practice in order to develop programming expertise. However, students often encounter anxiety when they begin to learn. This can present a challenge to educators because such anxiety can influence practice behaviour. This thesis situates this challenge within the Control- Value Theory of Achievement Emotions, emphasising a need for domain-specific research and presenting new research tools which can be used to investigate the area. Analysis of data collected from three cohorts of introductory programming students on web programming (2011-12) and robot programming (2012-13 and 2013-14) courses show that programming self-concept and programming aptitude mindset can predict programming anxiety and that programming anxiety is negatively correlated with programming practice. However, levels of anxiety remained consistently high across this period. A method to enrich these psychological constructs through a multimedia-rich learning environment is proposed. Drawing upon the interplay between narrative reinforcement and procedural rhetoric that can be achieved in a fantasy role-play, students' self-concept can be enhanced. A double-blind randomised controlled trial demonstrates promising results, however small effect sizes suggest further research is needed

What is not where: the challenge of integrating spatial representations into deep learning architectures

This paper examines to what degree current deep learning architectures for image caption generation capture spatial language. On the basis of the evaluation of examples of generated captions from the literature we argue that systems capture what objects are in the image data but not where these objects are located: the captions generated by these systems are the output of a language model conditioned on the output of an object detector that cannot capture fine-grained location information. Although language models provide useful knowledge for image captions, we argue that deep learning image captioning architectures should also model geometric relations between objects.Comment: 15 pages, 10 figures, Appears in CLASP Papers in Computational Linguistics Vol 1: Proceedings of the Conference on Logic and Machine Learning in Natural Language (LaML 2017), pp. 41-5

What Is Not Where: the Challenge of Integrating Spatial Representations Into Deep Learning Architectures

This paper examines to what degree current deep learning architectures for image caption generation capture spatial lan- guage. On the basis of the evaluation of examples of generated captions from the literature we argue that systems capture what objects are in the image data but not where these objects are located: the cap- tions generated by these systems are the output of a language model conditioned on the output of an object detector that cannot capture fine-grained location information. Although language models provide useful knowledge for image captions, we argue that deep learning image captioning architectures should also model geometric rela- tions between objects