Learning the Semantics of Manipulation Action

In this paper we present a formal computational framework for modeling manipulation actions. The introduced formalism leads to semantics of manipulation action and has applications to both observing and understanding human manipulation actions as well as executing them with a robotic mechanism (e.g. a humanoid robot). It is based on a Combinatory Categorial Grammar. The goal of the introduced framework is to: (1) represent manipulation actions with both syntax and semantic parts, where the semantic part employs $\lambda$-calculus; (2) enable a probabilistic semantic parsing schema to learn the $\lambda$-calculus representation of manipulation action from an annotated action corpus of videos; (3) use (1) and (2) to develop a system that visually observes manipulation actions and understands their meaning while it can reason beyond observations using propositional logic and axiom schemata. The experiments conducted on a public available large manipulation action dataset validate the theoretical framework and our implementation

Natural language acquisition and rhetoric in artificial intelligence

During the 1980s, artificial intelligence research started to undergo a quiet, but important shift in focus from research in computer science to research in the human sciences and humanities. Though in the past, artificial intelligence has primarily been researched by computer scientists, the need for input from the human sciences has invited a great amount of cross-disciplinary work by members of many different callings. Rarely do people start out in the field of artificial intelligence; rather, the dream of building an intelligent machine infects them as they see the parallels between their work and the projects being undertaken in artificial intelligence. Because artificial intelligence is, in essence, studying the qualities of humanness, few disciplines can avoid somehow being tied in

Understanding analogical reasoning : viewpoints from psychology and related disciplines

Analogy and metaphor have a long history of study in linguistics, education, philosophy and psychology. Consensus over what analogy is or how analogy functions in language and thought, however, has been elusive. This paper, the first in a two part series, examines these various research traditions, attempting to bring out major lines of agreement over the role of analogy in individual human experience. As well as being a general literature review which may be helpful for newcomers to the study of analogy, this paper attempts to extract from these literatures existing theories, models and concepts which may be interesting or useful for computational studies of analogical reasoning

From Biological to Synthetic Neurorobotics Approaches to Understanding the Structure Essential to Consciousness (Part 3)

This third paper locates the synthetic neurorobotics research reviewed in the second paper in terms of themes introduced in the first paper. It begins with biological non-reductionism as understood by Searle. It emphasizes the role of synthetic neurorobotics studies in accessing the dynamic structure essential to consciousness with a focus on system criticality and self, develops a distinction between simulated and formal consciousness based on this emphasis, reviews Tani and colleagues' work in light of this distinction, and ends by forecasting the increasing importance of synthetic neurorobotics studies for cognitive science and philosophy of mind going forward, finally in regards to most- and myth-consciousness

Language of physics, language of math: Disciplinary culture and dynamic epistemology

Mathematics is a critical part of much scientific research. Physics in particular weaves math extensively into its instruction beginning in high school. Despite much research on the learning of both physics and math, the problem of how to effectively include math in physics in a way that reaches most students remains unsolved. In this paper, we suggest that a fundamental issue has received insufficient exploration: the fact that in science, we don't just use math, we make meaning with it in a different way than mathematicians do. In this reflective essay, we explore math as a language and consider the language of math in physics through the lens of cognitive linguistics. We begin by offering a number of examples that show how the use of math in physics differs from the use of math as typically found in math classes. We then explore basic concepts in cognitive semantics to show how humans make meaning with language in general. The critical elements are the roles of embodied cognition and interpretation in context. Then we show how a theoretical framework commonly used in physics education research, resources, is coherent with and extends the ideas of cognitive semantics by connecting embodiment to phenomenological primitives and contextual interpretation to the dynamics of meaning making with conceptual resources, epistemological resources, and affect. We present these ideas with illustrative case studies of students working on physics problems with math and demonstrate the dynamical nature of student reasoning with math in physics. We conclude with some thoughts about the implications for instruction.Comment: 27 pages, 9 figure

Solving Bongard Problems with a Visual Language and Pragmatic Reasoning

More than 50 years ago Bongard introduced 100 visual concept learning problems as a testbed for intelligent vision systems. These problems are now known as Bongard problems. Although they are well known in the cognitive science and AI communities only moderate progress has been made towards building systems that can solve a substantial subset of them. In the system presented here, visual features are extracted through image processing and then translated into a symbolic visual vocabulary. We introduce a formal language that allows representing complex visual concepts based on this vocabulary. Using this language and Bayesian inference, complex visual concepts can be induced from the examples that are provided in each Bongard problem. Contrary to other concept learning problems the examples from which concepts are induced are not random in Bongard problems, instead they are carefully chosen to communicate the concept, hence requiring pragmatic reasoning. Taking pragmatic reasoning into account we find good agreement between the concepts with high posterior probability and the solutions formulated by Bongard himself. While this approach is far from solving all Bongard problems, it solves the biggest fraction yet