A graph-based framework for optimal semantic web service composition

Web services are self-described, loosely coupled software components that are network-accessible through standardized web protocols, whose characteristics are described in XML. One of the key promises of Web services is to provide better interoperability and to enable a faster integration between systems. In order to generate robust service oriented architectures, automatic composition algorithms are required in order to combine the functionality of many single services into composite services that are able to respond to demanding user requests, even when there is no single service capable of performing such task. Service composition consists of a combination of single services into composite services that are executed in sequence or in a different order, imposed by a set of control constructions that can be specified using standard languages such as OWL-s or BPEL4WS. In the last years several papers have dealt with composition of web services. Some approaches treat the service composition as a planning problem, where a sequence of actions lead from a initial state to a goal state. However, most of these proposals have some drawbacks: high complexity, high computational cost and inability to maximize the parallel execution of web services. Other approaches consider the problem as a graph search problem, where search algorithms are applied over a web service dependency graph in order to find a solution for a particular request. These proposals are simpler than their counterparts and also many can exploit the parallel execution of web services. However, most of these approaches rely on very complex dependency graphs that have not been optimized to remove data redundancy, which may negatively affect the overall performance and scalability of these techniques in large service registries. Therefore, it is necessary to identify, characterize and optimize the different tasks involved in the automatic service composition process in order to develop better strategies to efficiently obtain optimal solutions. The main goal of this dissertation is to develop a graph-based framework for automatic service composition that generate optimal input-output based compositions not only in terms of complexity of the solutions, but also in terms of overall quality of service solutions. More specifically, the objectives of this thesis are: (1) Analysis of the characteristics of services and compositions. The aim of this objective is to characterize and identify the main steps that are part for the service composition process. (2) Framework for automatic graph-based composition. This objective will focus on developing a framework that enables the efficient input-output based service composition, exploring the integration with other tasks that are part of the composition process, such as service discovery. (3) Development of optimal algorithms for automatic service composition. This objective focuses on the development of a set of algorithms and optimization techniques for the generation of optimal compositions, optimizing the complexity of the solutions and the overall Quality-of- Service. (4) Validation of the algorithms with standard datasets so they can be compared with other proposals

의미론적 환경 이해 기반 인간 로봇 협업

학위논문(박사)--서울대학교 대학원 :공과대학 전기·정보공학부,2020. 2. 이범희.Human-robot cooperation is unavoidable in various applications ranging from manufacturing to field robotics owing to the advantages of adaptability and high flexibility. Especially, complex task planning in large, unconstructed, and uncertain environments can employ the complementary capabilities of human and diverse robots. For a team to be effectives, knowledge regarding team goals and current situation needs to be effectively shared as they affect decision making. In this respect, semantic scene understanding in natural language is one of the most fundamental components for information sharing between humans and heterogeneous robots, as robots can perceive the surrounding environment in a form that both humans and other robots can understand. Moreover, natural-language-based scene understanding can reduce network congestion and improve the reliability of acquired data. Especially, in field robotics, transmission of raw sensor data increases network bandwidth and decreases quality of service. We can resolve this problem by transmitting information in the form of natural language that has encoded semantic representations of environments. In this dissertation, I introduce a human and heterogeneous robot cooperation scheme based on semantic scene understanding. I generate sentences and scene graphs, which is a natural language grounded graph over the detected objects and their relationships, with the graph map generated using a robot mapping algorithm. Subsequently, a framework that can utilize the results for cooperative mission planning of humans and robots is proposed. Experiments were performed to verify the effectiveness of the proposed methods. This dissertation comprises two parts: graph-based scene understanding and scene understanding based on the cooperation between human and heterogeneous robots. For the former, I introduce a novel natural language processing method using a semantic graph map. Although semantic graph maps have been widely applied to study the perceptual aspects of the environment, such maps do not find extensive application in natural language processing tasks. Several studies have been conducted on the understanding of workspace images in the field of computer vision; in these studies, the sentences were automatically generated, and therefore, multiple scenes have not yet been utilized for sentence generation. A graph-based convolutional neural network, which comprises spectral graph convolution and graph coarsening, and a recurrent neural network are employed to generate sentences attention over graphs. The proposed method outperforms the conventional methods on a publicly available dataset for single scenes and can be utilized for sequential scenes. Recently, deep learning has demonstrated impressive developments in scene understanding using natural language. However, it has not been extensively applied to high-level processes such as causal reasoning, analogical reasoning, or planning. The symbolic approach that calculates the sequence of appropriate actions by combining the available skills of agents outperforms in reasoning and planning; however, it does not entirely consider semantic knowledge acquisition for human-robot information sharing. An architecture that combines deep learning techniques and symbolic planner for human and heterogeneous robots to achieve a shared goal based on semantic scene understanding is proposed for scene understanding based on human-robot cooperation. In this study, graph-based perception is used for scene understanding. A planning domain definition language (PDDL) planner and JENA-TDB are utilized for mission planning and data acquisition storage, respectively. The effectiveness of the proposed method is verified in two situations: a mission failure, in which the dynamic environment changes, and object detection in a large and unseen environment.인간과 이종 로봇 간의 협업은 높은 유연성과 적응력을 보일 수 있다는 점에서 제조업에서 필드 로보틱스까지 다양한 분야에서 필연적이다. 특히, 서로 다른 능력을 지닌 로봇들과 인간으로 구성된 하나의 팀은 넓고 정형화되지 않은 공간에서 서로의 능력을 보완하며 복잡한 임무 수행을 가능하게 한다는 점에서 큰 장점을 갖는다. 효율적인 한 팀이 되기 위해서는, 팀의 공통된 목표 및 각 팀원의 현재 상황에 관한 정보를 실시간으로 공유할 수 있어야 하며 함께 의사 결정을 할 수 있어야 한다. 이러한 관점에서, 자연어를 통한 의미론적 환경 이해는 인간과 서로 다른 로봇들이 모두 이해할 수 있는 형태로 환경을 인지한다는 점에서 가장 필수적인 요소이다. 또한, 우리는 자연어 기반 환경 이해를 통해 네트워크 혼잡을 피함으로써 획득한 정보의 신뢰성을 높일 수 있다. 특히, 대량의 센서 데이터 전송에 의해 네트워크 대역폭이 증가하고 통신 QoS (Quality of Service) 신뢰도가 감소하는 문제가 빈번히 발생하는 필드 로보틱스 영역에서는 의미론적 환경 정보인 자연어를 전송함으로써 통신 대역폭을 감소시키고 통신 QoS 신뢰도를 증가시킬 수 있다. 본 학위 논문에서는 환경의 의미론적 이해 기반 인간 로봇 협동 방법에 대해 소개한다. 먼저, 로봇의 지도 작성 알고리즘을 통해 획득한 그래프 지도를 이용하여 자연어 문장과 검출한 객체 및 각 객체 간의 관계를 자연어 단어로 표현하는 그래프를 생성한다. 그리고 자연어 처리 결과를 이용하여 인간과 다양한 로봇들이 함께 협업하여 임무를 수행할 수 있도록 하는 프레임워크를 제안한다. 본 학위 논문은 크게 그래프를 이용한 의미론적 환경 이해와 의미론적 환경 이해를 통한 인간과 이종 로봇 간의 협업 방법으로 구성된다. 먼저, 그래프를 이용한 의미론적 환경 이해 부분에서는 의미론적 그래프 지도를 이용한 새로운 자연어 처리 방법에 대해 소개한다. 의미론적 그래프 지도 작성 방법은 로봇의 환경 인지 측면에서 많이 연구되었지만 이를 이용한 자연어 처리 방법은 거의 연구되지 않았다. 반면 컴퓨터 비전 분야에서는 이미지를 이용한 환경 이해 연구가 많이 이루어졌지만, 연속적인 장면들은 다루는데는 한계점이 있다. 따라서 우리는 그래프 스펙트럼 이론에 기반한 그래프 컨볼루션과 그래프 축소 레이어로 구성된 그래프 컨볼루션 신경망 및 순환 신경망을 이용하여 그래프를 설명하는 문장을 생성한다. 제안한 방법은 기존의 방법들보다 한 장면에 대해 향상된 성능을 보였으며 연속된 장면들에 대해서도 성공적으로 자연어 문장을 생성한다. 최근 딥러닝은 자연어 기반 환경 인지에 있어 급속도로 큰 발전을 이루었다. 하지만 인과 추론, 유추적 추론, 임무 계획과 같은 높은 수준의 프로세스에는 적용이 힘들다. 반면 임무를 수행하는 데 있어 각 에이전트의 능력에 맞게 행위들의 순서를 계산해주는 상징적 접근법(symbolic approach)은 추론과 임무 계획에 있어 뛰어난 성능을 보이지만 인간과 로봇들 사이의 의미론적 정보 공유 방법에 대해서는 거의 다루지 않는다. 따라서, 인간과 이종 로봇 간의 협업 방법 부분에서는 딥러닝 기법들과 상징적 플래너(symbolic planner)를 연결하는 프레임워크를 제안하여 의미론적 이해를 통한 인간 및 이종 로봇 간의 협업을 가능하게 한다. 우리는 의미론적 주변 환경 이해를 위해 이전 부분에서 제안한 그래프 기반 자연어 문장 생성을 수행한다. PDDL 플래너와 JENA-TDB는 각각 임무 계획 및 정보 획득 저장소로 사용한다. 제안한 방법의 효용성은 시뮬레이션을 통해 두 가지 상황에 대해서 검증한다. 하나는 동적 환경에서 임무 실패 상황이며 다른 하나는 넓은 공간에서 객체를 찾는 상황이다.1 Introduction 1 1.1 Background and Motivation 1 1.2 Literature Review 5 1.2.1 Natural Language-Based Human-Robot Cooperation 5 1.2.2 Artificial Intelligence Planning 5 1.3 The Problem Statement 10 1.4 Contributions 11 1.5 Dissertation Outline 12 2 Natural Language-Based Scene Graph Generation 14 2.1 Introduction 14 2.2 Related Work 16 2.3 Scene Graph Generation 18 2.3.1 Graph Construction 19 2.3.2 Graph Inference 19 2.4 Experiments 22 2.5 Summary 25 3 Language Description with 3D Semantic Graph 26 3.1 Introduction 26 3.2 Related Work 26 3.3 Natural Language Description 29 3.3.1 Preprocess 29 3.3.2 Graph Feature Extraction 33 3.3.3 Natural Language Description with Graph Features 34 3.4 Experiments 35 3.5 Summary 42 4 Natural Question with Semantic Graph 43 4.1 Introduction 43 4.2 Related Work 45 4.3 Natural Question Generation 47 4.3.1 Preprocess 49 4.3.2 Graph Feature Extraction 50 4.3.3 Natural Question with Graph Features 51 4.4 Experiments 52 4.5 Summary 58 5 PDDL Planning with Natural Language 59 5.1 Introduction 59 5.2 Related Work 60 5.3 PDDL Planning with Incomplete World Knowledge 61 5.3.1 Natural Language Process for PDDL Planning 63 5.3.2 PDDL Planning System 64 5.4 Experiments 65 5.5 Summary 69 6 PDDL Planning with Natural Language-Based Scene Understanding 70 6.1 Introduction 70 6.2 Related Work 74 6.3 A Framework for Heterogeneous Multi-Agent Cooperation 77 6.3.1 Natural Language-Based Cognition 78 6.3.2 Knowledge Engine 80 6.3.3 PDDL Planning Agent 81 6.4 Experiments 82 6.4.1 Experiment Setting 82 6.4.2 Scenario 84 6.4.3 Results 87 6.5 Summary 91 7 Conclusion 92Docto

A proposal for a global task planning architecture using the RoboEarth cloud based framework

As robotic systems become more and more capable of assisting in human domains, methods are sought to compose robot executable plans from abstract human instructions. To cope with the semantically rich and highly expressive nature of human instructions, Hierarchical Task Network planning is often being employed along with domain knowledge to solve planning problems in a pragmatic way. Commonly, the domain knowledge is specific to the planning problem at hand, impeding re-use. Therefore this paper conceptualizes a global planning architecture, based on the worldwide accessible RoboEarth cloud framework. This architecture allows environmental state inference and plan monitoring on a global level. To enable plan re-use for future requests, the RoboEarth action language has been adapted to allow semantic matching of robot capabilities with previously composed plans

Réseau social pour l’initiation de synergies industrielles

RÉSUMÉ : La recherche présentée dans cette thèse puise dans le domaine de l'écologie industrielle, précisément la symbiose industrielle (SI), en supposant qu'il est possible pour une ou plusieurs entreprises de trouver des débouchés à leurs matières résiduelles. La symbiose qui commence généralement par des relations d’affaires, des opérations de rapprochement et de collaboration entre deux ou plusieurs entreprises (synergies industrielles) aboutit à la mise en œuvre du développement durable à une échelle territoriale. Jusqu’à récemment, la majorité des contributions dans ce domaine portait sur les aspects conceptuels de la symbiose industrielle et les études de cas. Les études de cas avaient pour but de présenter un aperçu général des initiatives ou de projets spécifiques de symbiose industrielle ou d’examiner plusieurs cas au niveau d’une industrie en particulier, d’une région, d’une ville ou même d’une entreprise afin d’effectuer principalement des comparaisons. Plus tard, d'autres études ont commencé à analyser la performance économique et environnementale de la symbiose industrielle et ont ainsi proposé de nouvelles idées ou stratégies pour sa mise en œuvre. Malheureusement, les aspects sociaux sont restés la plupart du temps peu explorés. Bien que les dernières publications aient partiellement abordé ces lacunes, Spekkink, (2016), entre autres, reconnaît le rôle joué par les aspects sociaux, et plus particulièrement l’aspect de la confiance, et bien que la modélisation ait relativement aidé à comprendre l’évolution de la symbiose industrielle et à optimiser les flux de matières en expérimentant des scénarios divers (Cao et al., 2009; Kim et al., 2012), il y a encore peu de compréhension en ce qui concerne l’apport des dimensions sociales et comment elles affectent l'émergence et le fonctionnement des réseaux de symbioses industrielles. L’objectif de ce travail est dans un premier temps de présenter d’une part, la valeur ajoutée des médias sociaux ainsi que leurs fonctions dans l’initiation, la promotion et le développement d’un réseau industriel visant l'identification d'opportunités efficaces de mise en valeur de sous-produits en contexte de symbiose industrielle. Ce travail propose notamment un cadre conceptuel permettant de mieux comprendre la contribution des médias sociaux au développement des symbioses industrielles. Ce travail propose de plus une application du web sémantique au partage des connaissances et à l’identification d’opportunités de synergies industrielles dans le cadre des réseaux sociaux. Finalement, un modèle de simulation à base d’agents est proposé pour illustrer l'influence des relations sociales dans la création des synergies et des symbioses dans un parc industriel selon différentes structures de réseaux (random vs scale-free). À travers ce travail, les conclusions suivantes ont pu être tirées. La première conclusion démontre qu’un réseau social vert enrichi de fonctions sociales peut contribuer à l’émergence de synergies industrielles à travers l’apprentissage, le partage d’informations, le tissage de relations et grâce au rôle de coordination communautaire qu'il peut jouer. Ceci dans le but de soutenir le développement de la symbiose industrielle et de réduire ainsi l’impact collectif des communautés industrielles. La deuxième conclusion démontre qu’il est possible de produire de nouvelles inférences à partir de données issues du web sémantique fondées sur des tags préalablement déterminés et que l’ontologie proposée a la robustesse nécessaire pour réaliser cette tâche. En effet, la structure de l’ontologie exprimée en OWL (Langage interopérable du web sémantique permettant de concevoir des fichiers répondants au vocabulaire et à la sémantique de la logique des descriptions) et son exploitation par les requêtes SPARQL (le langage de requête du web sémantique) a permis de générer des relations validées entre des ressources issues du web afin d’identifier des relations de synergies potentielles entre acteurs industriels. Notons tout de même qu’un certain travail reste à réaliser avant d’envisager de mettre en production une telle ontologie. En effet, il importe d’identifier clairement les sources web d’informations et d'analyser leur qualité d'un point de vue du contenu des connaissances. Pour chacune de ces sources, il importe de méticuleusement choisir les tags appropriés et de les regrouper sous des catégories exploitables. De plus, une analyse plus approfondie des cas d’utilisation de cette approche permettrait aussi de mieux structurer les ontologies et d’orienter leur développement pour la mise en œuvre d’une solution répondant adéquatement aux exigences attendues du système. Finalement, la troisième contribution de ce travail supporte la conclusion que la dynamique et la structure sociale impliquées au sein des parcs industriels influencent le développement des symbioses industrielles. Bien que le modèle de simulation utilisé et les expériences réalisées ne permettent que de déterminer de façon générale leur impact sur la vitesse de développement des synergies industrielles, ces expériences démontrent que dans des conditions générales similaires, la structure et la dynamique des relations sociales influencent la matérialisation des synergies potentielles existantes au sein d’un groupe d’acteurs industriels. Cependant, plus de travail est nécessaire pour valider ces résultats et étudier plus en profondeur à la fois les facteurs sociaux qui favorisent la création des synergies industrielles, mais aussi pour étudier l’impact potentiel de médias sociaux dans divers contextes et dynamiques de symbioses industrielles. De plus, sur le plan méthodologique, les expériences réalisées illustrent comment un tel modèle peut être utilisé comme outil complémentaire aux études de cas empiriques très répandues dans ce domaine. Ces trois contributions ont pour buts à long terme de permettre le développement d’outils informatiques afin de supporter les animateurs de réseaux éco-industriels, les agents de développement économique, les conseils municipaux, les consultants ou les centres de recherche dans l'évaluation des bénéfices économiques et environnementaux potentiels de synergies industrielles. Ce projet a été mené en collaboration avec le Centre de Transfert Technologique en Écologie Industrielle, situé à Sorel- Tracy, Québec, Canada.----------ABSTRACT : The research presented in this thesis draws from the field of industrial ecology, precisely industrial symbiosis (IS), assuming that it is possible for a company or more to find outlets for their waste. The symbiosis that usually starts with business relations, reconciliation of transactions and collaboration between two or more industrial synergies results in the implementation of sustainable development implementation at a territorial level. Until recently, the majority of contributions in this area focused on the conceptual aspects of industrial symbiosis and case studies. The purpose of the case studies was to provide a general overview of a specific industrial symbiosis project or to examine several cases at the level of a particular industry, region, city or even a company to perform mainly comparisons. Later, other studies began to analyze the economic and environmental performance of industrial symbiosis and thus proposed new ideas or strategies for its implementation. Unfortunately, the social aspects have remained little explored. Although recent publications have partially addressed these gaps, Spekkink (2016), among others, acknowledges the role played by social aspects, particularly the aspect of trust. Although modeling has helped to understand the evolution of industrial symbiosis and the optimization of material flows by experimenting with various scenarios (Cao et al., 2009, Kim et al., 2012), there is still little understanding of the contribution of dimensions and how they affect the emergence and functioning of industrial symbiosis networks. On one hand, the objective of this work is to present the added value of social media and their functions in the initiation, promotion and development of an industrial network that is aimed at identifying opportunities as efficient by-product development in the context of industrial symbiosis. This work proposes in particular a conceptual framework to better understand the contribution of social media to the development of industrial symbiosis. On the other hand, this work also proposes an application of the semantic web to the sharing of knowledge and the identification of opportunities for industrial synergies within the framework of the social networks. Finally, an agent-based simulation model is proposed to illustrate the influence of social relations in the creation of synergies and symbiosis in an industrial park according to different network structures (random vs scale-free). Through this work, the following conclusions can be drawn. The first conclusion demonstrates that a green social network enriched with social functions can contribute to the emergence of industrial synergies through learning, information sharing, relationship building and community coordination. This emergence supports the development of industrial symbiosis and thus reduces the collective impact of industrial communities. The second conclusion demonstrates that it is possible to produce new inferences from semantic web data based on predetermined tags and that the proposed ontology has the necessary robustness to perform this task. Indeed, the structure of the ontology expressed in OWL (Semantic web interoperable language allowed to design files that respond to the vocabulary and semantics of descriptive logic) and its use by SPARQL queries (the query language of the web Semantics) enabled the generation of validated relationships between resources from the web in order to identify potential synergies between industrial players. Let us note, however, that some work remains to be realized before considering putting such an ontology into production. Indeed, it is crucial to clearly identify web sources of information and to analyze their quality from a knowledge content perspective. For each of these sources, it is important to carefully select the appropriate tags and group them into exploitable categories. Moreover, a more in-depth analysis of the use of this approach would also make it possible to better structure the ontologies and guide their development in order to implement a solution that meets the expected requirements of the system. Finally, the third contribution of this work supports the conclusion that the dynamics and social structure involved in industrial parks’ influence the development of industrial symbiosis. Although the simulation model used and the experiments carried out only allow us to determine in general their impact on the rate of development of industrial synergies, these experiments show that under similar general conditions the structure and dynamics of social relations influence the potential synergies within a group of industrial players. However, more work is needed to validate these findings and to explore more deeply both the social factors that favor the creation of industrial synergies and also to study the potential impact of social media in various contexts and dynamics of industrial symbiosis. Moreover, from the methodological point of view, the experiments carried out illustrate how such a model can be used as a complementary tool to the empirical case studies that are widely used in this field. The three long-term objectives of these three contributions are to enable the development of IT tools to support eco-industrial network leaders, economic development officers, municipal councils, consultants or research centers in the evaluation of economic and environmental benefits of industrial synergies. This project was conducted in collaboration with the Technology Transfer Centre in Industrial Ecology, located in Sorel-Tracy, Quebec, Canada

An Integrated Method for Information and Communication Technology (ICT) Supported Energy Efficiency Evaluation and Optimization in Manufacturing: Knowledge-based Approach and Energy Performance Indicators (EnPI) to Support Evaluation and Optimization of Energy Efficiency

This thesis develops a holistic evaluation and optimization of energy efficiency in manufacturing. The innovation of this thesis consists in the integrated method applying an expressive and adaptive ontology knowledge base capable of learning, and sector-independent, straightforward energy performance indicator for evaluating different processes and units within a company. This thesis also develops a hyper-heuristics-based energy-optimized and flexible production scheduling

Knowledge representation and exploitation for interactive and cognitive robots

L'arrivée des robots dans notre vie quotidienne fait émerger le besoin pour ces systèmes d'avoir accès à une représentation poussée des connaissances et des capacités de raisonnements associées. Ainsi, les robots doivent pouvoir comprendre les éléments qui composent l'environnement dans lequel ils évoluent. De plus, la présence d'humains dans ces environnements et donc la nécessité d'interagir avec eux amènent des exigences supplémentaires. Ainsi, les connaissances ne sont plus utilisées par le robot dans le seul but d'agir physiquement sur son environnement mais aussi dans un but de communication et de partage d'information avec les humains. La connaissance ne doit plus être uniquement compréhensible par le robot lui-même mais doit aussi pouvoir être exprimée. Dans la première partie de cette thèse, nous présentons Ontologenius. C'est un logiciel permettant de maintenir des bases de connaissances sous forme d'ontologie, de raisonner dessus et de les gérer dynamiquement. Nous commençons par expliquer en quoi ce logiciel est adapté aux applications d'interaction humain-robot (HRI), notamment avec la possibilité de représenter la base de connaissances du robot mais aussi une estimation des bases de connaissances des partenaires humains ce qui permet d'implémenter les mécanismes de théorie de l'esprit. Nous poursuivons avec une présentation de ses interfaces. Cette partie se termine par une analyse des performances du système ainsi développé. Dans une seconde partie, cette thèse présente notre contribution à deux problèmes d'exploration des connaissances: l'un ayant trait au référencement spatial et l'autre à l'utilisation de connaissances sémantiques. Nous commençons par une tâche de description d'itinéraires pour laquelle nous proposons une ontologie permettant de décrire la topologie d'environnements intérieurs et deux algorithmes de recherche d'itinéraires. Nous poursuivons avec une tâche de génération d'expression de référence. Cette tâche vise à sélectionner l'ensemble optimal d'informations à communiquer afin de permettre à un auditeur d'identifier l'entité référencée dans un contexte donné. Ce dernier algorithme est ensuite affiné pour y ajouter les informations sur les activités passées provenant d'une action conjointe entre un robot et un humain, afin de générer des expressions encore plus pertinentes. Il est également intégré à un planificateur de tâches symbolique pour estimer la faisabilité et le coût des futures communications. Cette thèse se termine par la présentation de deux architectures cognitives, la première utilisant notre contribution concernant la description d'itinéraire et la seconde utilisant nos contributions autour de la Génération d'Expression de Référence. Les deux utilisent Ontologenius pour gérer la base de connaissances sémantique. À travers ces deux architectures, nous présentons comment nos travaux ont amené la base de connaissances a progressivement prendre un rôle central, fournissant des connaissances à tous les composants du système.As robots begin to enter our daily lives, we need advanced knowledge representations and associated reasoning capabilities to enable them to understand and model their environments. Considering the presence of humans in such environments, and therefore the need to interact with them, this need comes with additional requirements. Indeed, knowledge is no longer used by the robot for the sole purpose of being able to act physically on the environment but also to communicate and share information with humans. Therefore knowledge should no longer be understandable only by the robot itself, but should also be able to be narrative-enabled. In the first part of this thesis, we present our first contribution with Ontologenius. This software allows to maintain knowledge bases in the form of ontology, to reason on them and to manage them dynamically. We start by explaining how this software is suitable for \acrfull{hri} applications. To that end, for example to implement theory of mind abilities, it is possible to represent the robot's knowledge base as well as an estimate of the knowledge bases of human partners. We continue with a presentation of its interfaces. This part ends with a performance analysis, demonstrating its online usability. In a second part, we present our contribution to two knowledge exploration problems around the general topic of spatial referring and the use of semantic knowledge. We start with the route description task which aims to propose a set of possible routes leading to a target destination, in the framework of a guiding task. To achieve this task, we propose an ontology allowing us to describe the topology of indoor environments and two algorithms to search for routes. The second knowledge exploration problem we tackle is the \acrfull{reg} problem. It aims at selecting the optimal set of piece of information to communicate in order to allow a hearer to identify the referred entity in a given context. This contribution is then refined to use past activities coming from joint action between a robot and a human, in order to generate new kinds of Referring Expressions. It is also linked with a symbolic task planner to estimate the feasibility and cost of future communications. We conclude this thesis by the presentation of two cognitive architectures. The first one uses the route description contribution and the second one takes advantage of our Referring Expression Generation contribution. Both of them use Ontologenius to manage the semantic Knowledge Base. Through these two architectures, we present how our contributions enable Knowledge Base to gradually take a central role, providing knowledge to all the components of the architectures

EG-ICE 2021 Workshop on Intelligent Computing in Engineering

The 28th EG-ICE International Workshop 2021 brings together international experts working at the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolutions to support multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways