Symbol Emergence in Robotics: A Survey

Humans can learn the use of language through physical interaction with their environment and semiotic communication with other people. It is very important to obtain a computational understanding of how humans can form a symbol system and obtain semiotic skills through their autonomous mental development. Recently, many studies have been conducted on the construction of robotic systems and machine-learning methods that can learn the use of language through embodied multimodal interaction with their environment and other systems. Understanding human social interactions and developing a robot that can smoothly communicate with human users in the long term, requires an understanding of the dynamics of symbol systems and is crucially important. The embodied cognition and social interaction of participants gradually change a symbol system in a constructive manner. In this paper, we introduce a field of research called symbol emergence in robotics (SER). SER is a constructive approach towards an emergent symbol system. The emergent symbol system is socially self-organized through both semiotic communications and physical interactions with autonomous cognitive developmental agents, i.e., humans and developmental robots. Specifically, we describe some state-of-art research topics concerning SER, e.g., multimodal categorization, word discovery, and a double articulation analysis, that enable a robot to obtain words and their embodied meanings from raw sensory--motor information, including visual information, haptic information, auditory information, and acoustic speech signals, in a totally unsupervised manner. Finally, we suggest future directions of research in SER.Comment: submitted to Advanced Robotic

Learning and Leveraging Structured Knowledge from User-Generated Social Media Data

Knowledge has long been a crucial element in Artificial Intelligence (AI), which can be traced back to knowledge-based systems, or expert systems, in the 1960s. Knowledge provides contexts to facilitate machine understanding and improves the explainability and performance of many semantic-based applications. The acquisition of knowledge is, however, a complex step, normally requiring much effort and time from domain experts. In machine learning as one key domain of AI, the learning and leveraging of structured knowledge, such as ontologies and knowledge graphs, have become popular in recent years with the advent of massive user-generated social media data. The main hypothesis in this thesis is therefore that a substantial amount of useful knowledge can be derived from user-generated social media data. A popular, common type of social media data is social tagging data, accumulated from users' tagging in social media platforms. Social tagging data exhibit unstructured characteristics, including noisiness, flatness, sparsity, incompleteness, which prevent their efficient knowledge discovery and usage. The aim of this thesis is thus to learn useful structured knowledge from social media data regarding these unstructured characteristics. Several research questions have then been formulated related to the hypothesis and the research challenges. A knowledge-centred view has been considered throughout this thesis: knowledge bridges the gap between massive user-generated data to semantic-based applications. The study first reviews concepts related to structured knowledge, then focuses on two main parts, learning structured knowledge and leveraging structured knowledge from social tagging data. To learn structured knowledge, a machine learning system is proposed to predict subsumption relations from social tags. The main idea is to learn to predict accurate relations with features, generated with probabilistic topic modelling and founded on a formal set of assumptions on deriving subsumption relations. Tag concept hierarchies can then be organised to enrich existing Knowledge Bases (KBs), such as DBpedia and ACM Computing Classification Systems. The study presents relation-level evaluation, ontology-level evaluation, and the novel, Knowledge Base Enrichment based evaluation, and shows that the proposed approach can generate high quality and meaningful hierarchies to enrich existing KBs. To leverage structured knowledge of tags, the research focuses on the task of automated social annotation and propose a knowledge-enhanced deep learning model. Semantic-based loss regularisation has been proposed to enhance the deep learning model with the similarity and subsumption relations between tags. Besides, a novel, guided attention mechanism, has been proposed to mimic the users' behaviour of reading the title before digesting the content for annotation. The integrated model, Joint Multi-label Attention Network (JMAN), significantly outperformed the state-of-the-art, popular baseline methods, with consistent performance gain of the semantic-based loss regularisers on several deep learning models, on four real-world datasets. With the careful treatment of the unstructured characteristics and with the novel probabilistic and neural network based approaches, useful knowledge can be learned from user-generated social media data and leveraged to support semantic-based applications. This validates the hypothesis of the research and addresses the research questions. Future studies are considered to explore methods to efficiently learn and leverage other various types of structured knowledge and to extend current approaches to other user-generated data

A whole brain probabilistic generative model: Toward realizing cognitive architectures for developmental robots

Building a human-like integrative artificial cognitive system, that is, an artificial general intelligence (AGI), is the holy grail of the artificial intelligence (AI) field. Furthermore, a computational model that enables an artificial system to achieve cognitive development will be an excellent reference for brain and cognitive science. This paper describes an approach to develop a cognitive architecture by integrating elemental cognitive modules to enable the training of the modules as a whole. This approach is based on two ideas: (1) brain-inspired AI, learning human brain architecture to build human-level intelligence, and (2) a probabilistic generative model (PGM)-based cognitive architecture to develop a cognitive system for developmental robots by integrating PGMs. The proposed development framework is called a whole brain PGM (WB-PGM), which differs fundamentally from existing cognitive architectures in that it can learn continuously through a system based on sensory-motor information.In this paper, we describe the rationale for WB-PGM, the current status of PGM-based elemental cognitive modules, their relationship with the human brain, the approach to the integration of the cognitive modules, and future challenges. Our findings can serve as a reference for brain studies. As PGMs describe explicit informational relationships between variables, WB-PGM provides interpretable guidance from computational sciences to brain science. By providing such information, researchers in neuroscience can provide feedback to researchers in AI and robotics on what the current models lack with reference to the brain. Further, it can facilitate collaboration among researchers in neuro-cognitive sciences as well as AI and robotics

The Future of Humanoid Robots

This book provides state of the art scientific and engineering research findings and developments in the field of humanoid robotics and its applications. It is expected that humanoids will change the way we interact with machines, and will have the ability to blend perfectly into an environment already designed for humans. The book contains chapters that aim to discover the future abilities of humanoid robots by presenting a variety of integrated research in various scientific and engineering fields, such as locomotion, perception, adaptive behavior, human-robot interaction, neuroscience and machine learning. The book is designed to be accessible and practical, with an emphasis on useful information to those working in the fields of robotics, cognitive science, artificial intelligence, computational methods and other fields of science directly or indirectly related to the development and usage of future humanoid robots. The editor of the book has extensive R&D experience, patents, and publications in the area of humanoid robotics, and his experience is reflected in editing the content of the book

Cultural Heritage Storytelling, Engagement and Management in the Era of Big Data and the Semantic Web

The current Special Issue launched with the aim of further enlightening important CH areas, inviting researchers to submit original/featured multidisciplinary research works related to heritage crowdsourcing, documentation, management, authoring, storytelling, and dissemination. Audience engagement is considered very important at both sites of the CH production–consumption chain (i.e., push and pull ends). At the same time, sustainability factors are placed at the center of the envisioned analysis. A total of eleven (11) contributions were finally published within this Special Issue, enlightening various aspects of contemporary heritage strategies placed in today’s ubiquitous society. The finally published papers are related but not limited to the following multidisciplinary topics:Digital storytelling for cultural heritage;Audience engagement in cultural heritage;Sustainability impact indicators of cultural heritage;Cultural heritage digitization, organization, and management;Collaborative cultural heritage archiving, dissemination, and management;Cultural heritage communication and education for sustainable development;Semantic services of cultural heritage;Big data of cultural heritage;Smart systems for Historical cities – smart cities;Smart systems for cultural heritage sustainability

Affective Computing

This book provides an overview of state of the art research in Affective Computing. It presents new ideas, original results and practical experiences in this increasingly important research field. The book consists of 23 chapters categorized into four sections. Since one of the most important means of human communication is facial expression, the first section of this book (Chapters 1 to 7) presents a research on synthesis and recognition of facial expressions. Given that we not only use the face but also body movements to express ourselves, in the second section (Chapters 8 to 11) we present a research on perception and generation of emotional expressions by using full-body motions. The third section of the book (Chapters 12 to 16) presents computational models on emotion, as well as findings from neuroscience research. In the last section of the book (Chapters 17 to 22) we present applications related to affective computing

マルチモーダル潜在的ディリクレ配分法の多層化による知識の確率的表現

近年，ロボットと人の共存を目指すための研究が盛んに行われている．現状のロボット技術において，様々なロボットが開発されているが，限られた環境で特定のタスクを実行するものが殆どであり，タスクに必要な行動や入力パターンに対する応答などを人が全て事前に与えなければならない．ロボットが人と自然に暮らすためには，人の言葉を理解する必要があり，その言葉の背後にある潜在的な意味を解釈して行動しなければならない．また，コミュニケーションのために，ロボット自身の意図を言語として創出することが望まれる．旧来の人工知能の研究では，単語を単なる記号として扱い，その記号で閉じた世界の中で言語を理解する努力を続けてきた．自然言語処理・理解は，この流れを強く受けている．これに対して近年のロボティクス・人工知能研究では，いわゆる記号接地問題を基本として，言語の本質的な意味を扱い始めているが，未だに言語の理解や生成の本質的な解決には遠く及ばない．本論文では，ロボットが経験によって得るマルチモーダル情報に基づいて多様な概念を形成し，この概念を基盤とした言語理解・生成を考えることでこの問題を解決する新たな方向性を示す．ここで，概念とはマルチモーダルな情報を分類して形成される「カテゴリ」であり，この概念を通して様々な予測をすることが「理解」であると定義する．さらに言語は，こうした概念と結び付いた音韻ラベルであり，人との自然なインタラクションの中で獲得することが可能である．つまり本論文で提案するモデルは，ロボットが日常の活動によって得ることのできる情報を基盤に概念を形成し，音韻ラベルとの結び付きや語の順番を意味する文法をボトムアップに獲得することで，言語の意味理解や生成を実現するものである．これまで，マルチモーダル情報を用いた物体のカテゴリ分類手法は中村らによって提案されており，実際に，ロボットが経験することによって得た情報をカテゴリ分類することで，人間の感覚に近い物体概念の形成が可能であることを示している．また，形成された概念を利用して未観測情報を予測することができ，ロボットによる物体の理解が前述の定義の範囲で可能であると言える．しかし，より人間のように柔軟な理解をロボットで実現するためには，物体概念の獲得だけでは不十分であることは明らかである．なぜなら，ほとんどの物体はそれを使う人や使う人の動き，使われる場所などが関連しており，これらの情報を予測できない限りその物体を理解したとは言えないためである．つまり，物体概念のみならず人の動き概念や場所概念など多様な概念を学習すると同時に，それらの関係性を獲得する必要がある．このような多様な概念の獲得は，マルチモーダル情報の階層的カテゴリ分類へと発展させることで実現することで可能であり，最終的にはこれがロボットによる「事物の真の理解の計算モデル」となることを明らかにする．これが本論文のゴールである．本論文ではまず，第2章でロボットが家庭環境で作業することを考慮し，これまで著者が開発したヒューマノイドによる掃除タスクを一例として取り上げる．掃除タスクを行うために，「掃除」を定義する必要があり，その定義に従ったタスクの実現に必要な視覚認識システムやタスクの制御などを実装する．これによって定義範囲内の物体認識や把持行動などを実現することができるが，未知な環境に対して柔軟にタスクを行うことができない．この結果を踏まえて，「掃除」の本質的な意味を考察する．例えば，「掃除機をかける」という行動は掃除機を持って細かいごみの上で動かすことであると考え，「掃除機」という物体概念，「何かの上で動かす」という動き概念の相互関係から形成される概念であると考えることができる．すなわち，「掃除」とは多様な概念の階層的な相互依存関係から構成される概念であると考える．こうした多様な概念の形成とそれらの階層的な構造の構築がロボットの知識として重要である．第2章での議論に基づき第3章では，ロボットの確率的知識表現のためのマルチモーダル情報の階層的カテゴリ分類手法を提案する．提案手法は，マルチモーダル潜在的ディリクレ配分法（Multimodal Latent Dirichlet Allocation：MLDA）を階層化した多層マルチモーダル潜在的ディリクレ配分法（multilayered MLDA：mMLDA）である．下層のMLDAでは下位概念である，物体，動き，場所，人物の概念がそれぞれ形成され，上層のMLDA ではこれらの概念を統合する上位概念が形成される．このモデルを用いることで例えば，下位概念としてジュースという物体概念や物を口に運ぶという動き概念，ダイニングという場所概念などが形成される．上位層ではこれらの関係性が学習され，「飲む」という行動概念が形成される．これにより，ジュースを見ることでそれを口に運ぶ「飲む」という行動や，その「飲む」という行動が「ダイニング」という場所で行なわれやすいといった未観測情報の予測を行うことが可能となる．第4章では，形成された多様な概念を利用し，同時に語意や文法を獲得することで，観測したシーンを文章で表現する手法を検討する．ここで扱う問題は，階層的な概念における語意の獲得であり，どの階層のどの概念にどの単語が結び付くかという問題を解く必要がある．本論文では，単語と概念間の相互情報量を用いることで，どの単語が本来どの概念に結び付いているのかを自動的に推定する手法を提案する．これにより単語と概念の結び付きを学習することが可能であり，各単語に対応する，物体，場所や人などといった概念クラスの推定が可能である．従って，教示発話における概念クラスの生起順を学習することで，概念クラスの遷移確率という形で表現される確率文法を学習することができる．これによって，ロボットによる言語の意味理解や生成を実現することが可能となる．一方，実際のコミュニケーションは，背景知識や周辺の状況などといった文脈を考慮しなければ成立しない．つまり，事物に対する理解をより柔軟に行うためには，学んできた多様な概念を活用した上で，様々な文脈を考慮する必要がある．第5章では，ロボットが人と生活する上で，様々な文脈においてどのように行動決定するかを議論する．つまり，獲得した多様な概念と文脈と統合することで，適切な行動を決定する手法を提案する．これにより例えば，人が普段ソファーでテレビを見ているときに，お菓子を食べながらお茶を飲んでいるということを知っていれば，人が「お菓子を持ってきて」と命令した際の音声認識に誤りが生じたとしても，そのときに「ソファーでテレビを見ていてお茶を飲んでいる」という文脈を用いることで，ロボットが適切に判断をして正しい行動をとることができる可能性がある．第6章では，本論文のまとめと今後の課題について述べる．電気通信大学201

Proceedings of the Eighth Italian Conference on Computational Linguistics CliC-it 2021

The eighth edition of the Italian Conference on Computational Linguistics (CLiC-it 2021) was held at Università degli Studi di Milano-Bicocca from 26th to 28th January 2022. After the edition of 2020, which was held in fully virtual mode due to the health emergency related to Covid-19, CLiC-it 2021 represented the first moment for the Italian research community of Computational Linguistics to meet in person after more than one year of full/partial lockdown