Augmented reality experiments with industrial robot in industry 4.0 environment

The role of human in the Industrie 4.0 vision is still considered as irreplaceable. Therefore, user interfaces of cyber-physical systems involved in the production automation need to be well designed and taking into consideration the industrial application requirements. With the advances in augmented and virtual reality data visualization and novel interaction techniques like mid-air gestures, these approaches seem to be suitable for integration into the industry environment. This paper describes the implementation of an augmented reality application for smart glasses with mid-air gestures and smart phone with touch interaction to compare and evaluate the usage of such interfaces in a production cell comprising an industrial robot.This research has been (partially) supported by the Technology Agency of the Czech Republic under the research program TE01020415 (V3C – Visual Computing Competence Center). This work is also (partially) funded by the Operational Programme for Competitiveness and Internationalisation –COMPETE 2020 and by FCT – Portuguese Foundation for Science and Technology.info:eu-repo/semantics/publishedVersio

Creating the Internet of Augmented Things: An Open-Source Framework to Make IoT Devices and Augmented and Mixed Reality Systems Talk to Each Other

[Abstract] Augmented Reality (AR) and Mixed Reality (MR) devices have evolved significantly in the last years, providing immersive AR/MR experiences that allow users to interact with virtual elements placed on the real-world. However, to make AR/MR devices reach their full potential, it is necessary to go further and let them collaborate with the physical elements around them, including the objects that belong to the Internet of Things (IoT). Unfortunately, AR/MR and IoT devices usually make use of heterogeneous technologies that complicate their intercommunication. Moreover, the implementation of the intercommunication mechanisms requires involving specialized developers with have experience on the necessary technologies. To tackle such problems, this article proposes the use of a framework that makes it easy to integrate AR/MR and IoT devices, allowing them to communicate dynamically and in real time. The presented AR/MR-IoT framework makes use of standard and open-source protocols and tools like MQTT, HTTPS or Node-RED. After detailing the inner workings of the framework, it is illustrated its potential through a practical use case: a smart power socket that can be monitored and controlled through Microsoft HoloLens AR/MR glasses. The performance of such a practical use case is evaluated and it is demonstrated that the proposed framework, under normal operation conditions, enables to respond in less than 100 ms to interaction and data update requests.Xunta de Galicia; IN853B-2018/0

Towards Context-aware Process Guidance in Cyber-Physical Systems with Augmented Reality

Assembly, configuration, maintenance, and repair processes in cyber-physical systems (e.g., a press line in a plant) comprise a multitude of complex tasks, whose execution needs to be controlled, coordinated and monitored. Amongst others, a process-centric guidance of users (e.g. service operators) is required, taking the high variability in the assembly of cyber-physical systems (e.g. press line variability) into account. Moreover, the tasks to be performed along these processes may be related to physical components, sensors and actuators, which need to be properly recognized, integrated and operated. In order to digitize cyber-physical processes as well as to guide users in a process-centric way, therefore, we suggest integrating process management technology, sensor/actuator interfaces, and augmented reality techniques. The paper discusses fundamental requirements for such an integration and presents an approach for process-centric user guidance that combines context and process management with augmented reality enhanced tasks. For evaluation purposes, we analyzed the cyber-physical processes of pharmaceutical packaging machines and implemented selected ones based on the approach. Overall, we are able to demonstrate the usefulness of context-aware process management for the flexible support of cyber-physical processes in the Industrial Internet of Things

Experimentation of a New TeleAssistance System Using Augmented Reality

ISBN : 2-9515730-9-XInternational audienceT.A.C system was built in order to propose a new mode of remote communication. When somebody needs to be assisted on a manual task, classical technical support methods such as phone or visioconference rapidly show their limits in terms of human interaction. By allowing a distant expert to see exactly what an operator is seeing, he could interact with him in interactive time thanks to augmented reality via an adequate interaction paradigm named "Picking Outlining Adding". By "simulating" co-presence of the expert close to the operator through visual guidance information, ambiguity of language and difficulty of communication are avoided, and operations are easily performed. Scientific experimentation we have conducted and we describe in this article shows the teaching interest and the efficiency of this new mode of communication. The operator learns and operates more rapidly, in complete serenity, increasing reliability of his tasks. Moreover, throughout this paper, we show that the developed principles are sufficiently general to be applied to other use cases of tele-assistance that go beyond the originally planned industrial maintenance

Experimentation of a New TeleAssistance System Using Augmented Reality

ISBN : 2-9515730-9-XInternational audienceT.A.C system was built in order to propose a new mode of remote communication. When somebody needs to be assisted on a manual task, classical technical support methods such as phone or visioconference rapidly show their limits in terms of human interaction. By allowing a distant expert to see exactly what an operator is seeing, he could interact with him in interactive time thanks to augmented reality via an adequate interaction paradigm named "Picking Outlining Adding". By "simulating" co-presence of the expert close to the operator through visual guidance information, ambiguity of language and difficulty of communication are avoided, and operations are easily performed. Scientific experimentation we have conducted and we describe in this article shows the teaching interest and the efficiency of this new mode of communication. The operator learns and operates more rapidly, in complete serenity, increasing reliability of his tasks. Moreover, throughout this paper, we show that the developed principles are sufficiently general to be applied to other use cases of tele-assistance that go beyond the originally planned industrial maintenance

Integrated Display and Environmental Awareness System - System Architecture Definition

The Integrated Display and Environmental Awareness System (IDEAS) is an interdisciplinary team project focusing on the development of a wearable computer and Head Mounted Display (HMD) based on Commercial-Off-The-Shelf (COTS) components for the specific application and needs of NASA technicians, engineers and astronauts. Wearable computers are on the verge of utilization trials in daily life as well as industrial environments. The first civil and COTS wearable head mounted display systems were introduced just a few years ago and they probed not only technology readiness in terms of performance, endurance, miniaturization, operability and usefulness but also maturity of practice in perspective of a socio-technical context. Although the main technical hurdles such as mass and power were addressed as improvements on the technical side, the usefulness, practicality and social acceptance were often noted on the side of a broad variety of humans' operations. In other words, although the technology made a giant leap, its use and efficiency still looks for the sweet spot. The first IDEAS project started in January 2015 and was concluded in January 2017. The project identified current COTS systems' capability at minimum cost and maximum applicability and brought about important strategic concepts that will serve further IDEAS-like system development

Augmented reality marker-based technology for augmenting newspaper advertisement

In this research, we describe an augmented reality android-based idea in which we utilize augmented reality marker-based technology for augmenting newspaper advertisement with electronic information that does not modify the format of the newspaper document and remains exactly the same, substantially improves the utility of paper by reducing the portion of the printed Ad on the newspaper. An implementation on a camera phone is discussed that lets users retrieve data and access links from newspaper advertisements to electronic data. We carefully examined over twenty people of different ages and occupations who participated in the newspaper-based AR and we got a significant overall response. Further analysis implies that this may assist students in understanding the complex 3D objects, which they can manipulate, learn tasks and improve skills