iviz: A ROS Visualization App for Mobile Devices

In this work, we introduce iviz, a mobile application for visualizing ROS data. In the last few years, the popularity of ROS has grown enormously, making it the standard platform for open source robotic programming. A key reason for this success is the availability of polished, general-purpose modules for many tasks, such as localization, mapping, path planning, and quite importantly, data visualization. However, the availability of the latter is generally restricted to PCs with the Linux operating system. Thus, users that want to see what is happening in the system with a smartphone or a tablet are stuck with solutions such as screen mirroring or using web browser versions of rviz, which are difficult to interact with from a mobile interface. More importantly, this makes newer visualization modalities such as Augmented Reality impossible. Our application iviz, based on the Unity engine, addresses these issues by providing a visualization platform designed from scratch to be usable in mobile platforms, such as iOS, Android, and UWP, and including native support for Augmented Reality for all three platforms. If desired, it can also be used in a PC with Linux, Windows, or macOS without any changes.Comment: This work has 7 pages and 7 figures. The repository of the project can be found in https://github.com/KIT-ISAS/iviz/tree/deve

Robot Remote Control based on Augmented Reality Glasses

Τα τελευταία χρόνια, η ραγδαία ανάπτυξη στον τομέα της ρομποτικής έχει φέρει εντυπωσιακές δυνατότητες στο περιβάλλον της τεχνολογίας. Έχει αναπτυχθεί ένα ευρύ φάσμα εφαρμογών με συσκευές κόμβους ώστε να εξυπηρετείται ο έλεγχος πληροφορίας σε απομακρυσμένες τοποθεσίες, αλλά και μέρη όπου το ανθρώπινο σώμα δεν μπορεί καν να πλησιάσει. Αυτή η εξέλιξη έχει δημιουργήσει την ανάγκη για πρωτότυπα ταιριάσματα νέων τεχνολογιών και καινοτόμες μεθόδους στον έλεγχο ρομπότ. Σε αυτή τη διπλωματική εργασία θα επικεντρωθούμε στον τηλεχειρισμό ενός ρομπότ με την τεχνολογία της επαυξημένης πραγματικότητας. Συγκεκριμένα, θα λάβουμε στην AR συσκευή Hololens της Microsoft μία ζωντανή ροή βίντεο από ένα ρομπότ με λειτουρικό ROS, και θα το πλοηγήσουμε από τα γυαλιά μέσω ενός διακομιστή Kafka. Τα πειράματα υποστηρίχθηκαν από το λειτουργικό σύστημα Ubuntu 16.04, τους προσομοιωτές Gazebo και Rviz, ένα Turtlebot 2 με raspberry 3 που εκτελεί το λειτουργικό σύστημα ROS, καθώς και έναν αισθητήρα XBOX Kinect με έγχρωμη κάμερα και αισθητήρα βάθουςIn recent years, the rapid development in the field of robotics has brought an impressive potential to the technology environment. It has been developed a wide range of applications for node devices in order to serve the control of information in remote locations but also in places where the human body can not even approach. This development has created the need for new matchings of new technologies and innovative methods in robot control. In this thesis, we will focus on the remote control of a robot with the technology of augmented reality. Specifically, we will be receiving a live video stream from a ROS robot to Microsoft’s AR glasses, Hololens and we will navigate back the robot from them through a Kafka Server. The experiments were supported by the Ubuntu 16.04 operating system, the Gazebo, and Rviz simulators, a Turtlebot 2 with a raspberry 3 that is running the ROS operating system, as well as a XBOX Kinect sensor with a color camera and a depth sensor

Novel Architecture of OneM2M-Based Convergence Platform for Mixed Reality and IoT

There have been numerous works proposed to merge augmented reality/mixed reality (AR/MR) and Internet of Things (IoT) in various ways. However, they have focused on their specific target applications and have limitations on interoperability or reusability when utilizing them to different domains or adding other devices to the system. This paper proposes a novel architecture of a convergence platform for AR/MR and IoT systems and services. The proposed architecture adopts the oneM2M IoT standard as the basic framework that converges AR/MR and IoT systems and enables the development of application services used in general-purpose environments without being subordinate to specific systems, domains, and device manufacturers. We implement the proposed architecture utilizing the open-source oneM2M-based IoT server and device platforms released by the open alliance for IoT standards (OCEAN) and Microsoft HoloLens as an MR device platform. We also suggest and demonstrate the practical use cases and discuss the advantages of the proposed architecture

Explorando ferramentas de modelação digital, aumentada e orientada por dados em engenharia e design de produto

Tools are indispensable for all diligent professional practice. New concepts and possibilities for paradigm shifting are emerging with recent computational technological developments in digital tools. However, new tools from key concepts such as “Big-Data”, “Accessibility” and “Algorithmic Design” are fundamentally changing the input and position of the Product Engineer and Designer. After the context introduction, this dissertation document starts by extracting three pivotal criteria from the Product Design Engineering's State of the Art analysis. In each one of those criteria the new emergent, more relevant and paradigmatic concepts are explored and later on are positioned and compared within the Product Lifecycle Management wheel scheme, where the potential risks and gaps are pointed to be explored in the experience part. There are two types of empirical experiences: the first being of case studies from Architecture and Urban Planning — from the student's professional experience —, that served as a pretext and inspiration for the experiments directly made for Product Design Engineering. First with a set of isolated explorations and analysis, second with a hypothetical experience derived from the latter and, finally, a deliberative section that culminate in a listing of risks and changes concluded from all the previous work. The urgency to reflect on what will change in that role and position, what kind of ethical and/or conceptual reformulations should exist for the profession to maintain its intellectual integrity and, ultimately, to survive, are of the utmost evidence.As ferramentas são indispensáveis para toda a prática diligente profissional. Novos conceitos e possibilidades de mudança de paradigma estão a surgir com os recentes progressos tecnológicos a nível computacional nas ferramentas digitais. Contudo, novas ferramentas originadas sobre conceitos-chave como “Big Data”, “Acessibilidade” e “Design Algorítmico” estão a mudar de forma fundamental o contributo e posição do Engenheiro e Designer de Produto. Esta dissertação, após uma primeira introdução contextual, começa por extrair três conceitos-eixo duma análise ao Estado da Arte actual em Engenharia e Design de Produto. Em cada um desses conceitos explora-se os novos conceitos emergentes mais relevantes e paradigmáticos, que então são comparados e posicionados no círculo de Gestão de Ciclo de Vida de Produto, apontando aí potenciais riscos e falhas que possam ser explorados em experiências. As experiências empíricas têm duas índoles: a primeira de projetos e casos de estudo de arquitetura e planeamento urbanístico — experiência em contexto de trabalho do aluno —, que serviu de pretexto e inspiração para as experiências relacionadas com Engenharia e Design de Produto. Primeiro com uma série de análises e experiências isoladas, segundo com uma formulação hipotética com o compêndio dessas experiências e, finalmente, com uma secção de reflexão que culmina numa série de riscos e mudanças induzidas do trabalho anterior. A urgência em refletir sobre o que irá alterar nesse papel e posição, que género de reformulações éticas e/ou conceptuais deverão existir para que a profissão mantenha a sua integridade intelectual e, em última instância, sobreviva, são bastante evidentes.Mestrado em Engenharia e Design de Produt

User Interface for ARTable and Microsoft Hololens

Tato práce se zaměřuje na použitelnost brýlí Microsoft HoloLens pro rozšířenou realitu v prototypu pracoviště pro spolupráci člověka s robotem - "ARTable". Použití brýlí je demonstrováno vytvořeným uživatelským rozhraním, které pomáhá uživatelům lépe a rychleji porozumět systému ARTable. Umožňuje prostorově vizualizovat naučené programy, aniž by bylo nutné spouštět samotného robota. Uživatel je veden 3D animací jednotlivých programů a hlasem zařízení, což mu pomůže získat jasnou představu o tom, co by se stalo, pokud by program spustil přímo na robotovi. Implementované řešení také umožňuje interaktivně provést uživatele celým procesem programování robota. Použití brýlí umožňuje mimo jiné zobrazit cenné prostorové informace, například vidění robota, tedy zvýraznit ty objekty, které jsou robotem detekovány.This thesis focuses on usability of mixed reality head-mounted display -   Microsoft HoloLens - in a human-robot collaborative workspace - the ARTable. Use of the headset is demonstrated by created user interface which helps regular workers to better and faster understand the ARTable system. It allows to spatially visualize learned programs, without the necessity to run the robot itself. The user is guided by 3D animation of individual programs and by device voice, which helps him to get a clear idea of what will happen if he runs the program directly on the robot. The solution also provides interactive guidance for the user when programming the robot. Using mixed reality displays also enables to visualize valuable spatial information, such as robot perception.