Symbol Emergence in Robotics: A Survey

Humans can learn the use of language through physical interaction with their environment and semiotic communication with other people. It is very important to obtain a computational understanding of how humans can form a symbol system and obtain semiotic skills through their autonomous mental development. Recently, many studies have been conducted on the construction of robotic systems and machine-learning methods that can learn the use of language through embodied multimodal interaction with their environment and other systems. Understanding human social interactions and developing a robot that can smoothly communicate with human users in the long term, requires an understanding of the dynamics of symbol systems and is crucially important. The embodied cognition and social interaction of participants gradually change a symbol system in a constructive manner. In this paper, we introduce a field of research called symbol emergence in robotics (SER). SER is a constructive approach towards an emergent symbol system. The emergent symbol system is socially self-organized through both semiotic communications and physical interactions with autonomous cognitive developmental agents, i.e., humans and developmental robots. Specifically, we describe some state-of-art research topics concerning SER, e.g., multimodal categorization, word discovery, and a double articulation analysis, that enable a robot to obtain words and their embodied meanings from raw sensory--motor information, including visual information, haptic information, auditory information, and acoustic speech signals, in a totally unsupervised manner. Finally, we suggest future directions of research in SER.Comment: submitted to Advanced Robotic

Understanding Visualization: A formal approach using category theory and semiotics

This article combines the vocabulary of semiotics and category theory to provide a formal analysis of visualization. It shows how familiar processes of visualization fit the semiotic frameworks of both Saussure and Peirce, and extends these structures using the tools of category theory to provide a general framework for understanding visualization in practice, including: relationships between systems, data collected from those systems, renderings of those data in the form of representations, the reading of those representations to create visualizations, and the use of those visualizations to create knowledge and understanding of the system under inspection. The resulting framework is validated by demonstrating how familiar information visualization concepts (such as literalness, sensitivity, redundancy, ambiguity, generalizability, and chart junk) arise naturally from it and can be defined formally and precisely. This article generalizes previous work on the formal characterization of visualization by, inter alia, Ziemkiewicz and Kosara and allows us to formally distinguish properties of the visualization process that previous work does not

Building a Socio-technical Perspective of Community Resilience with a Semiotic Approach

Situated in the diversity and adversity of real-life contexts facing crisis situations, this research aims at boosting the resilience process within communities supported by digital and social technology. In this paper, eight community leaders in different parts of the world are invited to express their issues and wishes regarding the support of technology to face social challenges. Methods and artefacts based on the Organisational Semiotics (OS) and the Socially-Aware computing have been applied to analyse and consolidate this data. By providing both a systemic view of the problem and also leading to the identification of requirements, the analysis evidences some benefits of the OS-based approach to consolidate perspectives from different real-life scenarios towards building a socio-technical solution

Proceedings ICPW'07: 2nd International Conference on the Pragmatic Web, 22-23 Oct. 2007, Tilburg: NL

Proceedings ICPW'07: 2nd International Conference on the Pragmatic Web, 22-23 Oct. 2007, Tilburg: N

Changing building user attitude and organisational policy towards sustainable resource use in healthcare

Health care provision is significantly impacted by the ability of the health providers to engineer a viable healthcare space to support care stakeholders needs. In this paper we discuss and propose use of organisational semiotics as a set of methods to link stakeholders to systems, which allows us to capture clinician activity, information transfer, and building use; which in tern allows us to define the value of specific systems in the care environment to specific stakeholders and the dependence between systems in a care space. We suggest use of a semantically enhanced building information model (BIM) to support the linking of clinician activity to the physical resource objects and space; and facilitate the capture of quantifiable data, over time, concerning resource use by key stakeholders. Finally we argue for the inclusion of appropriate stakeholder feedback and persuasive mechanism, to incentivise building user behaviour to support organisational level sustainability policy

Specification and implementation of mapping rule visualization and editing : MapVOWL and the RMLEditor

Visual tools are implemented to help users in defining how to generate Linked Data from raw data. This is possible thanks to mapping languages which enable detaching mapping rules from the implementation that executes them. However, no thorough research has been conducted so far on how to visualize such mapping rules, especially if they become large and require considering multiple heterogeneous raw data sources and transformed data values. In the past, we proposed the RMLEditor, a visual graph-based user interface, which allows users to easily create mapping rules for generating Linked Data from raw data. In this paper, we build on top of our existing work: we (i) specify a visual notation for graph visualizations used to represent mapping rules, (ii) introduce an approach for manipulating rules when large visualizations emerge, and (iii) propose an approach to uniformly visualize data fraction of raw data sources combined with an interactive interface for uniform data fraction transformations. We perform two additional comparative user studies. The first one compares the use of the visual notation to present mapping rules to the use of a mapping language directly, which reveals that the visual notation is preferred. The second one compares the use of the graph-based RMLEditor for creating mapping rules to the form-based RMLx Visual Editor, which reveals that graph-based visualizations are preferred to create mapping rules through the use of our proposed visual notation and uniform representation of heterogeneous data sources and data values. (C) 2018 Elsevier B.V. All rights reserved

An aesthetics of touch: investigating the language of design relating to form

How well can designers communicate qualities of touch? This paper presents evidence that they have some capability to do so, much of which appears to have been learned, but at present make limited use of such language. Interviews with graduate designer-makers suggest that they are aware of and value the importance of touch and materiality in their work, but lack a vocabulary to fully relate to their detailed explanations of other aspects such as their intent or selection of materials. We believe that more attention should be paid to the verbal dialogue that happens in the design process, particularly as other researchers show that even making-based learning also has a strong verbal element to it. However, verbal language alone does not appear to be adequate for a comprehensive language of touch. Graduate designers-makers’ descriptive practices combined non-verbal manipulation within verbal accounts. We thus argue that haptic vocabularies do not simply describe material qualities, but rather are situated competences that physically demonstrate the presence of haptic qualities. Such competencies are more important than groups of verbal vocabularies in isolation. Design support for developing and extending haptic competences must take this wide range of considerations into account to comprehensively improve designers’ capabilities

Designing online learning for scientific writing: Collaborations, creations and transformations

This thesis is a multilayered approach to understanding the complex processes involved in designing, developing, implementing and evaluating online learning environments for academic writing in discipline contexts. The study is broadly situated in the field of educational design research (EDR). It brings together theories of pedagogical design, including those of multimodality and educational linguistics, with the practical implementation and evaluation of designs in context. From an applied perspective, the research addresses the problem of providing support for students to improve their academic writing, in particular the writing of the laboratory report genre, a key genre in science and engineering disciplines. For teachers and others involved in the design of online teaching and learning, the aim is to provide design principles to support the process of creating effective resources to teach academic writing online. These principles cover all stages of the process from design to evaluation. The thesis comprises three main stages which focus on the processes involved in the development of an online program for supporting students writing a report in Physiology, the Flexible Electronic Report-writing Tool (FLERT). The first focuses on the collaborations of the design team in creating the online learning resources within a ‘communities of practice’ framework. I use discourse analysis, based on the theory of systemic functional linguistics (SFL), to identify knowledge and relationship building among participants. The second draws on both multimodal social semiotics and SFL to examine how network and screen designs created for laboratory report writing programs in science and engineering have evolved over time. The third uses a multi- and mixed methods approach, together with SFL, to examine two cycles of implementation and evaluation of FLERT to assess how students have transformed their learning through their interactions with the program. The relationships among the outcomes from these three stages provides insights into: • the practice of design for learning; • the meaning making characteristics of the products of design for teaching and learning purposes; • the interactions of student users with the designed products and the influence of design features on student learning; • design principles, both general principles for online learning program design and those, at a more local level, for teaching academic writing online