Conceptual spatial representations for indoor mobile robots

We present an approach for creating conceptual representations of human-made indoor environments using mobile robots. The concepts refer to spatial and functional properties of typical indoor environments. Following findings in cognitive psychology, our model is composed of layers representing maps at different levels of abstraction. The complete system is integrated in a mobile robot endowed with laser and vision sensors for place and object recognition. The system also incorporates a linguistic framework that actively supports the map acquisition process, and which is used for situated dialogue. Finally, we discuss the capabilities of the integrated system

Foundations and Recent Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions

Multimodal machine learning is a vibrant multi-disciplinary research field that aims to design computer agents with intelligent capabilities such as understanding, reasoning, and learning through integrating multiple communicative modalities, including linguistic, acoustic, visual, tactile, and physiological messages. With the recent interest in video understanding, embodied autonomous agents, text-to-image generation, and multisensor fusion in application domains such as healthcare and robotics, multimodal machine learning has brought unique computational and theoretical challenges to the machine learning community given the heterogeneity of data sources and the interconnections often found between modalities. However, the breadth of progress in multimodal research has made it difficult to identify the common themes and open questions in the field. By synthesizing a broad range of application domains and theoretical frameworks from both historical and recent perspectives, this paper is designed to provide an overview of the computational and theoretical foundations of multimodal machine learning. We start by defining two key principles of modality heterogeneity and interconnections that have driven subsequent innovations, and propose a taxonomy of 6 core technical challenges: representation, alignment, reasoning, generation, transference, and quantification covering historical and recent trends. Recent technical achievements will be presented through the lens of this taxonomy, allowing researchers to understand the similarities and differences across new approaches. We end by motivating several open problems for future research as identified by our taxonomy

Narratives as a Fundamental Component of Consciousness

In this paper, we propose a conceptual architecture that models human (spatially-temporally-modally) cohesive narrative development using a computer representation of quale properties. Qualia are proposed to be the fundamental "cognitive" components humans use to generate cohesive narratives. The engineering approach is based on cognitively inspired technologies and incorporates the novel concept of quale representation for computation of primitive cognitive components of narrative. The ultimate objective of this research is to develop an architecture that emulates the human ability to generate cohesive narratives with incomplete or perturbated information

A Logical Framework for Behaviour Reasoning and Assistance in a Smart Home

Abstract- Smart Homes (SH) have emerged as a realistic intelligent assistive environment capable of providing assistive living for the elderly and the disabled. Nevertheless, it still remains a challenge to assist the inhabitants of a SH in performing the “right” action(s) at the “right ” time in the “right ” place. To address this challenge, this paper introduces a novel logical framework for cognitive behavioural modelling, reasoning and assistance based on a highly developed logical theory of actions- the Event Calculus. Cognitive models go beyond data-centric behavioural models in that they govern an inhabitant’s behaviour by reasoning about its knowledge, actions and environmental events. In our work we outline the theoretical foundation of such an approach and describe cognitive modelling of SH. We discuss the reasoning capabilities and algorithms of the cognitive SH model and present the details of the various tasks it can support. A system architecture is proposed to illustrate the use of the framework in facilitating assistive living. We demonstrate the perceived effectiveness of the approach through presentation of its operation in the context of a real world daily activity scenario. Index Terms – Event calculus, cognitive modelling

Semantic linking through spaces for cyber-physical-socio intelligence:a methodology

Humans consciously and subconsciously establish various links, emerge semantic images and reason in mind, learn linking effect and rules, select linked individuals to interact, and form closed loops through links while co-experiencing in multiple spaces in lifetime. Machines are limited in these abilities although various graph-based models have been used to link resources in the cyber space. The following are fundamental limitations of machine intelligence: (1) machines know few links and rules in the physical space, physiological space, psychological space, socio space and mental space, so it is not realistic to expect machines to discover laws and solve problems in these spaces; and, (2) machines can only process pre-designed algorithms and data structures in the cyber space. They are limited in ability to go beyond the cyber space, to learn linking rules, to know the effect of linking, and to explain computing results according to physical, physiological, psychological and socio laws. Linking various spaces will create a complex space — the Cyber-Physical-Physiological-Psychological-Socio-Mental Environment CP3SME. Diverse spaces will emerge, evolve, compete and cooperate with each other to extend machine intelligence and human intelligence. From multi-disciplinary perspective, this paper reviews previous ideas on various links, introduces the concept of cyber-physical society, proposes the ideal of the CP3SME including its definition, characteristics, and multi-disciplinary revolution, and explores the methodology of linking through spaces for cyber-physical-socio intelligence. The methodology includes new models, principles, mechanisms, scientific issues, and philosophical explanation. The CP3SME aims at an ideal environment for humans to live and work. Exploration will go beyond previous ideals on intelligence and computing

Engineering Knowledge for Assistive Living

This paper introduces a knowledge based approach to assistive living in smart homes. It proposes a system architecture that makes use of knowledge in the lifecycle of assistive living. The paper describes ontology based knowledge engineering practices and discusses mechanisms for exploiting knowledge for activity recognition and assistance. It presents system implementation and experiments, and discusses initial results

Vision Language Models in Autonomous Driving and Intelligent Transportation Systems

The applications of Vision-Language Models (VLMs) in the fields of Autonomous Driving (AD) and Intelligent Transportation Systems (ITS) have attracted widespread attention due to their outstanding performance and the ability to leverage Large Language Models (LLMs). By integrating language data, the vehicles, and transportation systems are able to deeply understand real-world environments, improving driving safety and efficiency. In this work, we present a comprehensive survey of the advances in language models in this domain, encompassing current models and datasets. Additionally, we explore the potential applications and emerging research directions. Finally, we thoroughly discuss the challenges and research gap. The paper aims to provide researchers with the current work and future trends of VLMs in AD and ITS