MaestROB: A Robotics Framework for Integrated Orchestration of Low-Level Control and High-Level Reasoning

This paper describes a framework called MaestROB. It is designed to make the robots perform complex tasks with high precision by simple high-level instructions given by natural language or demonstration. To realize this, it handles a hierarchical structure by using the knowledge stored in the forms of ontology and rules for bridging among different levels of instructions. Accordingly, the framework has multiple layers of processing components; perception and actuation control at the low level, symbolic planner and Watson APIs for cognitive capabilities and semantic understanding, and orchestration of these components by a new open source robot middleware called Project Intu at its core. We show how this framework can be used in a complex scenario where multiple actors (human, a communication robot, and an industrial robot) collaborate to perform a common industrial task. Human teaches an assembly task to Pepper (a humanoid robot from SoftBank Robotics) using natural language conversation and demonstration. Our framework helps Pepper perceive the human demonstration and generate a sequence of actions for UR5 (collaborative robot arm from Universal Robots), which ultimately performs the assembly (e.g. insertion) task.Comment: IEEE International Conference on Robotics and Automation (ICRA) 2018. Video: https://www.youtube.com/watch?v=19JsdZi0TW

BCI Integration: Application Interfaces

Natural disaster

Multiple Robot Simulation in a Virtual Reality Environment

Nowadays, robotics is becoming increasingly important in people's daily lives. However, the process of learning and training in robotics is not always easy. In fact, in most cases, proper training is linked to direct interaction with these devices. This is usually not possible for the vast majority of people, as they may not have access to a robot. Nevertheless, thanks to the emergence of different technologies such as Virtual Reality (VR) it is possible to do things that were considered unimaginable before. Therefore, this project aims to make the most of both technologies, creating an alternative way of interacting with robots to understand how they behave, thus flattening the robotics learning curve. To this end, a software that allows the simulation and control of various robots in VR has been developed

Neurokernel: An Open Source Platform for Emulating the Fruit Fly Brain

We have developed an open software platform called Neurokernel for collaborative development of comprehensive models of the brain of the fruit fly Drosophila melanogaster and their execution and testing on multiple Graphics Processing Units (GPUs). Neurokernel provides a programming model that capitalizes upon the structural organization of the fly brain into a fixed number of functional modules to distinguish between these modules’ local information processing capabilities and the connectivity patterns that link them. By defining mandatory communication interfaces that specify how data is transmitted between models of each of these modules regardless of their internal design, Neurokernel explicitly enables multiple researchers to collaboratively model the fruit fly’s entire brain by integration of their independently developed models of its constituent processing units. We demonstrate the power of Neurokernel’s model integration by combining independently developed models of the retina and lamina neuropils in the fly’s visual system and by demonstrating their neuroinformation processing capability. We also illustrate Neurokernel’s ability to take advantage of direct GPU-to-GPU data transfers with benchmarks that demonstrate scaling of Neurokernel’s communication performance both over the number of interface ports exposed by an emulation’s constituent modules and the total number of modules comprised by an emulation

A Cloud-Based Extensible Avatar For Human Robot Interaction

Adding an interactive avatar to a human-robot interface requires the development of tools that animate the avatar so as to simulate an intelligent conversation partner. Here we describe a toolkit that supports interactive avatar modeling for human-computer interaction. The toolkit utilizes cloud-based speech-to-text software that provides active listening, a cloud-based AI to generate appropriate textual responses to user queries, and a cloud-based text-to-speech generation engine to generate utterances for this text. This output is combined with a cloud-based 3D avatar animation synchronized to the spoken response. Generated text responses are embedded within an XML structure that allows for tuning the nature of the avatar animation to simulate different emotional states. An expression package controls the avatar's facial expressions. The introduced rendering latency is obscured through parallel processing and an idle loop process that animates the avatar between utterances. The efficiency of the approach is validated through a formal user study

Architectural modelling for robotics: RoboArch and the CorteX example

The need for robotic systems to be verified grows as robots are increasingly used in complex applications with safety implications. Model-driven engineering and domain-specific languages (DSLs) have proven useful in the development of complex systems. RoboChart is a DSL for modelling robot software controllers using state machines and a simple component model. It is distinctive in that it has a formal semantics and support for automated verification. Our work enriches RoboChart with support for modelling architectures and architectural patterns used in the robotics domain. Support is in the shape of an additional DSL, RoboArch, whose primitive concepts encapsulate the notion of a layered architecture and architectural patterns for use in the design of the layers that are only informally described in the literature. A RoboArch model can be used to generate automatically a sketch of a RoboChart model, and the rules for automatic generation define a semantics for RoboArch. Additional patterns can be formalised by extending RoboArch. In this paper, we present RoboArch, and give a perspective of how it can be used in conjunction with CorteX, a software framework developed for the nuclear industry

Live Coding as a Model for Cultural Practice & Cultural-Epistemological Aspects of Live Coding

This report documents the program and the outcomes of Dagstuhl Seminar 13382 “Collaboration and learning through live coding”. Live coding is improvised interactive programming, typically to create electronic music and other digital media, done live with an audience. Our seminar was motivated by the phenomenon and experience of live coding. Our conviction was that those represent an important and broad, but seldom articulated, set of opportunities for computer science and the arts and humanities. The seminar participants included a broad range of scholars, researchers, and practitioners spanning fields from music theory to software engineering. We held live coding performances, and facilitated discussions on three main perspectives, the humanities, computing education, and software engineering. The main outcome of our seminar was better understanding of the potential of live coding for informing cross-disciplinary scholarship and practice, connecting the arts, cultural studies, and computing. The report is edited by Alan Blackwell and Alex McLean and James Noble and Julian Rohrhuber