Smart operators: How augmented and virtual technologies are affecting the worker's performance in manufacturing contexts

Purpose: The correct interaction between the workforce and augmented, virtual, and mixed reality technologies represents a crucial aspect of the success of the smart factory. This interaction is, indeed, affected by the variability of human behavior and its reliability, which can strongly influence the quality, safety, and productivity standards. For this reason, this paper aims to provide a clear and complete analysis of the impacts of these technologies on the performance of operators. Design/methodology/approach: A Systematic Literature Review (SLR) was conducted to identify peer-reviewed papers that focused on the implementation of augmented and virtual technologies in manufacturing systems and their effects on human performance. Findings: In total, 61 papers were selected and thoroughly analyzed. The findings of this study reveal that Augmented, Virtual and Mixed Reality can be applied for several applications in manufacturing systems with different types of devices, that involve various advantages and disadvantages. The worker’s performance that are influencing by the use of these technologies are above all time to complete a task, error rate and mental and physical workload. Originality/value: Over the years Augmented, Virtual and Mixed Reality technologies in manufacturing systems have been investigated by researchers. Several studies mostly focused on technological issues, have been conducted. The role of the operator, whose tasks may be influenced positively or negatively by the use of new devices, has been hardly ever analyzed and a deep analysis of human performance affected by these technologies is missing. This study represents a preliminary analysis to fill this gap. The results obtained from the SLR allowed us to develop a conceptual framework that investigates the current state-of-the-art knowledge about the topic and highlights gaps in the current researchPeer Reviewe

An innovative user-centred support tool for Augmented Reality maintenance systems design: a preliminary study

Augmented Reality (AR) technologies in maintenance are demonstrated to positively impact on technicians’ work and performance. Current research in this area is mainly focused on solving technical challenges related with AR in industrial environments. Limited attention has been put into the user perception, ergonomics and usability aspects of AR systems design. This paper proposes an innovative user-centred design support tool for AR systems in maintenance contexts. The tool is based on the Analytic Hierarchy Process (AHP), which is a well-established multi-criteria decision-making approach. In this research, AHP is utilised for guiding designers in the evaluation of application-contexts and AR-technologies for selecting the most suitable and effective AR interaction solution. The tool’s validation has been conducted with twelve maintenance-experts in a design workshop using two case studies. The quantitative results obtained in both case studies reveal the applicability of the AHP model, as well as the effectiveness of the design support tool for complex decisions in AR for maintenance. The use of AHP methods for AR design enable experts to deal with complex and contrasting concepts and express a preference among them with a subjective judgement based on their personal understanding of the problem. Therefore, simplifying the design of AR systems for complex maintenance contexts

Augmented reality usage for prototyping speed up

The first part of the article describes our approach for solution of this problem by means of Augmented Reality. The merging of the real world model and digital objects allows streamline the work with the model and speed up the whole production phase significantly. The main advantage of augmented reality is the possibility of direct manipulation with the scene using a portable digital camera. Also adding digital objects into the scene could be done using identification markers placed on the surface of the model. Therefore it is not necessary to work with special input devices and lose the contact with the real world model. Adjustments are done directly on the model. The key problem of outlined solution is the ability of identification of an object within the camera picture and its replacement with the digital object. The second part of the article is focused especially on the identification of exact position and orientation of the marker within the picture. The identification marker is generalized into the triple of points which represents a general plane in space. There is discussed the space identification of these points and the description of representation of their position and orientation be means of transformation matrix. This matrix is used for rendering of the graphical objects (e. g. in OpenGL and Direct3D).Comment: Keywords: augmented reality, prototyping, pose estimation, transformation matri

Potential of mobile applications in human-centric production and logistics management

With the increasing market penetration of smart devices (smartphones, smartwatches, and tablets), various mobile applications (apps) have been developed to fulfill tasks in daily life. Recently, efforts have been made to develop apps to support human operators in industrial work. When apps installed on commercial devices are utilized, tasks that were formerly done purely manually or with the help of investment-intensive specific devices can be performed more efficiently and/or at a lower cost and with reduced errors. Despite their advantages, smart devices have limitations because embedded sensors (e.g., accelerometers) and components (e.g., cameras) are usually designed for nonindustrial use. Hence, validation experiments and case studies for industrial applications are needed to ensure the reliability of app usage. In this study, a systematic literature review was employed to identify the state of knowledge about the use of mobile apps in production and logistics management. The results show how apps can support human centricity based on the enabling technologies and components of smart devices. An outlook for future research and applications is provided, including the need for proper validation studies to ensure the diversity and reliability of apps and more research on psychosocial aspects of human-technology interaction

Comparative Evaluation of Augmented Reality-based Assistance for Procedural Tasks: A Simulated Control Room Study

This is an Accepted Manuscript of an article published by Taylor & Francis in Behaviour & Information Technology in 2019, available online: https://doi.org/10.1080/0144929X.2019.1660805This research explores the design, implementation, and evaluation of a prototype augmented reality application that assists operators in performing procedural tasks in control room settings. Our prototype uses a tablet display to supplement an operator’s natural view of existing control panel elements with sequences of interactive visual and attention guiding cues. An experiment, conducted using a nuclear power plant simulator, examined university students completing both standard and emergency operating procedures. The augmented reality condition was compared against two other conditions – a paper-based procedure condition using paper manuals and a computer-based procedure condition using digital procedures presented on a desktop display. The results demonstrated that the augmented reality -based procedure system had benefits in terms of reduced mental workload in comparison to the other two conditions. Regarding task completion time, accuracy, and situation awareness, the augmented reality condition had no significant difference when compared against the computer-based procedure condition but performed better than the paper-based procedure condition. It was also found that the augmented reality condition resulted in fewer intra-team inquiry communication exchanges in comparison to both paper-based and computer-based conditions. The augmented reality condition, however, yielded poorer memory retention score when assessed against the other two conditions

Augmented reality experiments with industrial robot in industry 4.0 environment

The role of human in the Industrie 4.0 vision is still considered as irreplaceable. Therefore, user interfaces of cyber-physical systems involved in the production automation need to be well designed and taking into consideration the industrial application requirements. With the advances in augmented and virtual reality data visualization and novel interaction techniques like mid-air gestures, these approaches seem to be suitable for integration into the industry environment. This paper describes the implementation of an augmented reality application for smart glasses with mid-air gestures and smart phone with touch interaction to compare and evaluate the usage of such interfaces in a production cell comprising an industrial robot.This research has been (partially) supported by the Technology Agency of the Czech Republic under the research program TE01020415 (V3C – Visual Computing Competence Center). This work is also (partially) funded by the Operational Programme for Competitiveness and Internationalisation –COMPETE 2020 and by FCT – Portuguese Foundation for Science and Technology.info:eu-repo/semantics/publishedVersio

Camera 3D positioning mixed reality-based interface to improve worker safety, ergonomics and productivity

[EN] This research develops a new mixed reality-based worker interface for industrial camera 3D positioning, which is intuitive and easy to manage, in order to enhance the worker safety, ergonomics and productivity. An experimental prototype to be used in the car body quality control is developed in the paper. The benefits and drawbacks of the proposed interface are discussed along the paper and sustained through several usability tests conducted with users familiar and not-familiar with mixed reality devices. Furthermore, the feasibility of the proposed approach is demonstrated by tests made in an industrial environment with skilled workers from Alfatec Sistemas company.This work was supported in part by the Spanish Government under the Project DPI2017-87656-C2-1-R.Muñoz García, A.; Martí Testón, A.; Mahiques, X.; Gracia Calandin, LI.; Solanes Galbis, JE.; Tornero Montserrat, J. (2020). Camera 3D positioning mixed reality-based interface to improve worker safety, ergonomics and productivity. CIRP Journal of Manufacturing Science and Technology (Online). 28:24-37. https://doi.org/10.1016/j.cirpj.2020.01.004S24372