175 research outputs found
Learning Semantic Components from Subsymbolic Multimodal Perception
International audiencePerceptual systems often include sensors from several modalities. However, existing robots do not yet sufficiently discover patterns that are spread over the flow of multimodal data they receive. In this paper we present a framework that learns a dictionary of words from full spoken utterances, together with a set of gestures from human demonstrations and the semantic connection between words and gestures. We explain how to use a nonnegative matrix factorization algorithm to learn a dictionary of components that represent meaningful elements present in the multimodal perception, without providing the system with a symbolic representation of the semantics. We illustrate this framework by showing how a learner discovers word-like components from observation of gestures made by a human together with spoken descriptions of the gestures, and how it captures the semantic association between the two
The Roles of Symbols in Neural-based AI: They are Not What You Think!
We propose that symbols are first and foremost external communication tools
used between intelligent agents that allow knowledge to be transferred in a
more efficient and effective manner than having to experience the world
directly. But, they are also used internally within an agent through a form of
self-communication to help formulate, describe and justify subsymbolic patterns
of neural activity that truly implement thinking. Symbols, and our languages
that make use of them, not only allow us to explain our thinking to others and
ourselves, but also provide beneficial constraints (inductive bias) on learning
about the world. In this paper we present relevant insights from neuroscience
and cognitive science, about how the human brain represents symbols and the
concepts they refer to, and how today's artificial neural networks can do the
same. We then present a novel neuro-symbolic hypothesis and a plausible
architecture for intelligent agents that combines subsymbolic representations
for symbols and concepts for learning and reasoning. Our hypothesis and
associated architecture imply that symbols will remain critical to the future
of intelligent systems NOT because they are the fundamental building blocks of
thought, but because they are characterizations of subsymbolic processes that
constitute thought.Comment: 28 page
Recommended from our members
Anchoring Knowledge in Interaction: Towards a Harmonic Subsymbolic/Symbolic Framework and Architecture of Computational Cognition
We outline a proposal for a research program leading to a new paradigm, architectural framework, and prototypical implementation, for the cognitively inspired anchoring of an agent’s learning, knowledge formation, and higher reasoning abilities in real-world interactions: Learning through interaction in real-time in a real environment triggers the incremental accumulation and repair of knowledge that leads to the formation of theories at a higher level of abstraction. The transformations at this higher level filter down and inform the learning process as part of a permanent cycle of learning through experience, higher-order deliberation, theory formation and revision.
The envisioned framework will provide a precise computational theory, algorithmic descriptions, and an implementation in cyber-physical systems, addressing the lifting of action patterns from the subsymbolic to the symbolic knowledge level, effective methods for theory formation, adaptation, and evolution, the anchoring of knowledge-level objects, real-world interactions and manipulations, and the realization and evaluation of such a system in different scenarios. The expected results can provide new foundations for future agent architectures, multi-agent systems, robotics, and cognitive systems, and can facilitate a deeper understanding of the development and interaction in human-technological settings
Recommended from our members
Towards integrated neural-symbolic systems for human-level AI: Two research programs helping to bridge the gaps
After a human-level AI-oriented overview of the status quo in neural-symbolic integration, two research programs aiming at overcoming long-standing challenges in the field are suggested to the community: The first program targets a better understanding of foundational differences and relationships on the level of computational complexity between symbolic and subsymbolic computation and representation, potentially providing explanations for the empirical differences between the paradigms in application scenarios and a foothold for subsequent attempts at overcoming these. The second program suggests a new approach and computational architecture for the cognitively-inspired anchoring of an agent's learning, knowledge formation, and higher reasoning abilities in real-world interactions through a closed neural-symbolic acting/sensing-processing-reasoning cycle, potentially providing new foundations for future agent architectures, multi-agent systems, robotics, and cognitive systems and facilitating a deeper understanding of the development and interaction in human-technological settings
The Tensor Brain: A Unified Theory of Perception, Memory and Semantic Decoding
We present a unified computational theory of an agent's perception and
memory. In our model, perception, episodic memory, and semantic memory are
realized by different operational modes of the oscillating interactions between
a symbolic index layer and a subsymbolic representation layer. The two layers
form a bilayer tensor network (BTN). Although memory appears to be about the
past, its main purpose is to support the agent in the present and the future.
Recent episodic memory provides the agent with a sense of the here and now.
Remote episodic memory retrieves relevant past experiences to provide
information about possible future scenarios. This aids the agent in
decision-making. "Future" episodic memory, based on expected future events,
guides planning and action. Semantic memory retrieves specific information,
which is not delivered by current perception, and defines priors for future
observations. We argue that it is important for the agent to encode individual
entities, not just classes and attributes. We demonstrate that a form of
self-supervised learning can acquire new concepts and refine existing ones. We
test our model on a standard benchmark data set, which we expanded to contain
richer representations for attributes, classes, and individuals. Our key
hypothesis is that obtaining a better understanding of perception and memory is
a crucial prerequisite to comprehending human-level intelligence.Comment: Accepted for publication at Neural Computatio
Symbol Emergence in Robotics: A Survey
Humans can learn the use of language through physical interaction with their
environment and semiotic communication with other people. It is very important
to obtain a computational understanding of how humans can form a symbol system
and obtain semiotic skills through their autonomous mental development.
Recently, many studies have been conducted on the construction of robotic
systems and machine-learning methods that can learn the use of language through
embodied multimodal interaction with their environment and other systems.
Understanding human social interactions and developing a robot that can
smoothly communicate with human users in the long term, requires an
understanding of the dynamics of symbol systems and is crucially important. The
embodied cognition and social interaction of participants gradually change a
symbol system in a constructive manner. In this paper, we introduce a field of
research called symbol emergence in robotics (SER). SER is a constructive
approach towards an emergent symbol system. The emergent symbol system is
socially self-organized through both semiotic communications and physical
interactions with autonomous cognitive developmental agents, i.e., humans and
developmental robots. Specifically, we describe some state-of-art research
topics concerning SER, e.g., multimodal categorization, word discovery, and a
double articulation analysis, that enable a robot to obtain words and their
embodied meanings from raw sensory--motor information, including visual
information, haptic information, auditory information, and acoustic speech
signals, in a totally unsupervised manner. Finally, we suggest future
directions of research in SER.Comment: submitted to Advanced Robotic
Neurocognitive Informatics Manifesto.
Informatics studies all aspects of the structure of natural and artificial information systems. Theoretical and abstract approaches to information have made great advances, but human information processing is still unmatched in many areas, including information management, representation and understanding. Neurocognitive informatics is a new, emerging field that should help to improve the matching of artificial and natural systems, and inspire better computational algorithms to solve problems that are still beyond the reach of machines. In this position paper examples of neurocognitive inspirations and promising directions in this area are given
SERKET: An Architecture for Connecting Stochastic Models to Realize a Large-Scale Cognitive Model
To realize human-like robot intelligence, a large-scale cognitive
architecture is required for robots to understand the environment through a
variety of sensors with which they are equipped. In this paper, we propose a
novel framework named Serket that enables the construction of a large-scale
generative model and its inference easily by connecting sub-modules to allow
the robots to acquire various capabilities through interaction with their
environments and others. We consider that large-scale cognitive models can be
constructed by connecting smaller fundamental models hierarchically while
maintaining their programmatic independence. Moreover, connected modules are
dependent on each other, and parameters are required to be optimized as a
whole. Conventionally, the equations for parameter estimation have to be
derived and implemented depending on the models. However, it becomes harder to
derive and implement those of a larger scale model. To solve these problems, in
this paper, we propose a method for parameter estimation by communicating the
minimal parameters between various modules while maintaining their programmatic
independence. Therefore, Serket makes it easy to construct large-scale models
and estimate their parameters via the connection of modules. Experimental
results demonstrated that the model can be constructed by connecting modules,
the parameters can be optimized as a whole, and they are comparable with the
original models that we have proposed
- …