Instructing Hierarchical Tasks to Robots by Verbal Commands

Natural language is an effective tool for communication, as information can be expressed in different ways and at different levels of complexity. Verbal commands, utilized for instructing robot tasks, can therefor replace traditional robot programming techniques, and provide a more expressive means to assign actions and enable collaboration. However, the challenge of utilizing speech for robot programming is how actions and targets can be grounded to physical entities in the world. In addition, to be time-efficient, a balance needs to be found between fine- and course-grained commands and natural language phrases. In this work we provide a framework for instructing tasks to robots by verbal commands. The framework includes functionalities for single commands to actions and targets, as well as longer-term sequences of actions, thereby providing a hierarchical structure to the robot tasks. Experimental evaluation demonstrates the functionalities of the framework by human collaboration with a robot in different tasks, with different levels of complexity. The tools are provided open-source at https://petim44.github.io/voice-jogger/Comment: 7 pages, accepted to 16th IEEE/SICE International Symposium on System Integratio

Multi-label Annotation for Visual Multi-Task Learning Models

Deep learning requires large amounts of data, and a well-defined pipeline for labeling and augmentation. Current solutions support numerous computer vision tasks with dedicated annotation types and formats, such as bounding boxes, polygons, and key points. These annotations can be combined into a single data format to benefit approaches such as multi-task models. However, to our knowledge, no available labeling tool supports the export functionality for a combined benchmark format, and no augmentation library supports transformations for the combination of all. In this work, these functionalities are presented, with visual data annotation and augmentation to train a multi-task model (object detection, segmentation, and key point extraction). The tools are demonstrated in two robot perception use cases.Comment: 5 pages, accepted to IEEE International Conference on Robotic Computin

Co-speech gestures for human-robot collaboration

Collaboration between human and robot requires effective modes of communication to assign robot tasks and coordinate activities. As communication can utilize different modalities, a multi-modal approach can be more expressive than single modal models alone. In this work we propose a co-speech gesture model that can assign robot tasks for human-robot collaboration. Human gestures and speech, detected by computer vision and speech recognition, can thus refer to objects in the scene and apply robot actions to them. We present an experimental evaluation of the multi-modal co-speech model with a real-world industrial use case. Results demonstrate that multi-modal communication is easy to achieve and can provide benefits for collaboration with respect to single modal tools.Comment: 5 pages, accepted to IEEE International Conference on Robotics Computin

Noise and dynamical pattern selection

In pattern forming systems such as Rayleigh-Benard convection or directional solidification, a large number of linearly stable, patterned steady states exist when the basic, simple steady state is unstable. Which of these steady states will be realized in a given experiment appears to depend on unobservable details of the system's initial conditions. We show, however, that weak, Gaussian white noise drives such a system toward a preferred wave number which depends only on the system parameters and is independent of initial conditions. We give a prescription for calculating this wave number, analytically near the onset of instability and numerically otherwise.Comment: 12 pages, REVTEX, no figures. Submitted to Phys. Rev. Let

Anion Distribution, Structural Distortion, and Symmetry-Driven Optical Band Gap Bowing in Mixed Halide Cs₂SnX₆ Vacancy Ordered Double Perovskites

Mixed anion compounds in the Fm-3m vacancy ordered perovskite structure were synthesised and characterised experimentally and computationally with a focus on compounds where A = Cs+. Pure anion Cs2SnX6 compounds were formed with X = Cl, Br and I using a room temperature solution phase method. Mixed anion compounds were formed as solid solutions of Cs2SnCl6 and Cs2SnBr6 and a second series from Cs2SnBr6 and Cs2SnI6. Single phase structures formed across the entirety of both composition series, with no evidence of long range anion ordering observed by diffraction. A distortion of the cubic A2BX6 structure was identified in which the spacing of the BX6 octahedra changes to accommodate the A site cation without reduction of overall symmetry. Optical band gap values varied with anion composition between 4.89 eV in Cs2SnCl6 to 1.35 eV in Cs2SnI6, but proved highly non-linear with changes in composition. In mixed halide compounds it was found that lower energy optical transitions appeared that were not present in the pure halide compounds, and this could be attributed to lowering of the local symmetry within the tin halide octahedra. The electronic structure was characterised by photoemission spectroscopy, and Raman spectroscopy revealed vibrational modes in the mixed halide compounds that could be assigned to particular mixed halide octahedra. This analysis was used to determine the distribution of octahedra types in mixed anion compounds, which was found to be consistent with a near-random distribution of halide anions throughout the structure, although some deviations from random halide distribution were noted in mixed iodide-bromide compounds, where the larger iodide anions preferentially adopted trans configurations