Study of Augmented Reality based manufacturing for further integration of quality control 4.0: a systematic literature review

Augmented Reality (AR) has gradually become a mainstream technology enabling Industry 4.0 and its maturity has also grown over time. AR has been applied to support different processes on the shop-floor level, such as assembly, maintenance, etc. As various processes in manufacturing require high quality and near-zero error rates to ensure the demands and safety of end-users, AR can also equip operators with immersive interfaces to enhance productivity, accuracy and autonomy in the quality sector. However, there is currently no systematic review paper about AR technology enhancing the quality sector. The purpose of this paper is to conduct a systematic literature review (SLR) to conclude about the emerging interest in using AR as an assisting technology for the quality sector in an industry 4.0 context. Five research questions (RQs), with a set of selection criteria, are predefined to support the objectives of this SLR. In addition, different research databases are used for the paper identification phase following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) methodology to find the answers for the predefined RQs. It is found that, in spite of staying behind the assembly and maintenance sector in terms of AR-based solutions, there is a tendency towards interest in developing and implementing AR-assisted quality applications. There are three main categories of current AR-based solutions for quality sector, which are AR-based apps as a virtual Lean tool, AR-assisted metrology and AR-based solutions for in-line quality control. In this SLR, an AR architecture layer framework has been improved to classify articles into different layers which are finally integrated into a systematic design and development methodology for the development of long-term AR-based solutions for the quality sector in the future

Research and Development of Simulation Service System with Cloud Manufacturing for Industrial Robot

伴随着工业信息化技术的不断提高，在自动化生产中工业机器人施展出的作用愈加明显。机器人仿真作为机器人性能和工作特点的有效研究工具，在机器人研究领域占据重要地位，为生产制造提供有效的参考依据。计算机技术与互联网技术的兴起，尤其是虚拟现实、云计算、大数据等新兴技术，为机器人仿真研究提供了新的思路。 本文在现有机器人仿真研究的基础上，基于虚拟现实与云制造技术，对机器人加工仿真、机器人示教、机器人在线选型等功能进行研究开发与集成，初步搭建了一个面向工业机器人的云制造仿真服务原型系统。主要研究内容如下： （1）对工业机器人的运动学进行推导，求解ABBIRB4600-60/2.05型号机器人的正逆运动学...As industrial information technology is continuously developing, industrial robot plays an even more obvious role in automatic production. Robot simulation occupies an important place in the field of robot research, not only it’s an effective tool to study robot performance and working characteristics, but also it can provide reference for robot production. With the development of computer technol...学位：工程硕士院系专业：航空航天学院_工程硕士(机械工程)学号：1992014115291

Intuitive robot programming using augmented reality

The demands on companies caused by the markets are becoming more and more fast-moving and complex. Reasons are the increasing number of variants of products as well as reduced product life cycle times. This is particularly relevant for assembly tasks, since a very high flexibility and adaptability to varying ambient condition must be ensured in this area. Therefore, a lot of steps are currently being carried out manually. However, if an automation is attempted in the field of assembly, companies are often facing a trade-off between a high degree of automation and flexibility of the production system. A concept to increase the automation rate in assembly tasks can be seen in the use of collaborative robots, so that the benefits of both, humans and robots, can be combined to accomplish the task. In doing so, one key issue is to simplify and thus to accelerate the programming process so that the necessary programming skills of employees can be reduced. Although some of today’s collaborative robots already offer good programming approaches like kinesthetic teaching, this article introduces a new and more intuitive programming method which is based on Augmented Reality. For this purpose, components of an assembly group are virtually linked with CAD models by using optical markers. The operator can then virtually assemble the components according to the assembly sequence. Results of first tests indicate, that once the assembly process is recorded, the robot can accomplish the assembly in reality. Finally, possibilities for future developments are presented

A Framework for Extended Reality System Development in Manufacturing

This paper presents a framework for developing extended reality (XR) systems within manufacturing context. The aim of this study is to develop a systematic framework to improve the usability and user acceptance of future XR systems. So that manufacturing industry can move from the “wow effect” of XR demonstrators into the stage whereas XR systems can be successfully integrated and improve the conventional work routines. It is essential to ensure the usability and user acceptance of XR systems for the wider adoption in manufacturing. The proposed framework was developed through six case studies that covered different XR system developments for different application areas of manufacturing. The framework consists of five iterative phases: (1) requirements analysis, (2) solution selection, (3) data preparation, (4) system implementation and (5) system evaluation. It is validated through one empirical case and seven identified previous studies, which partly aligned with the proposed framework. The proposed framework provides a clear guideline on the steps needed to integrate XR in manufacturing and it extends the XR usage with increased usability and user acceptance. Furthermore, it strengthens the importance of user-centered approach for XR system development in manufacturing

Monitoring companion for industrial robotic processes

For system integrators, optimizing complex industrial robotic applications (e.g. robotised welding) is a difficult and time-consuming task. This procedure is rendered tedious and often very hard to achieve when the operator cannot access the robotic system once in operation, perhaps because the installation is far away or because of the operational environment. In these circumstances, as an alternative to physically visiting the installation site, the system integrator may rely on additional nearby sensors to remotely acquire the necessary process information. While it is hard to completely replace this trial and error approach, it is possible to provide a way to gather process information more effectively that can be used in several robotic installations.This thesis investigates the use of a "monitoring robot" in addition to the task robot(s) that belong to the industrial process to be optimized. The monitoring robot can be equipped with several different sensors and can be moved into close proximity of any installed task robot so that it can be used to collect information from that process during and/or after the operation without interfering. The thesis reviews related work in the industry and in the field of teleoperation to identify the most important challenges in remote monitoring and teleoperation. From the background investigation it is clear that two very important issues are: i) the nature of the teleoperator’s interface and; ii) the efficiency of the shared control between the human operator and the monitoring system. In order to investigate these two issues efficiently it was necessary to create experimental scenarios that operate independently from any application scenario, so an abstract problem domain is created. This way the monitoring system's control and interface can be evaluated in a context that presents challenges that are typical of a remote monitoring task but are not application domain specific. Therefore the validity of the proposed approach can be assessed from a generic and, therefore, more powerful and widely applicable perspective. The monitoring framework developed in this thesis is described, both in the shared control design choices based on virtual fixtures (VF) and the implementation in a 3D visualization environment. The monitoring system developed is evaluated with a usability study with user participants. The usability study aims at assessing the system's performance along with its acceptance and ease of use in a static monitoring task, accompanied by user\hyp{}filled TLX questionnaires. Since future work will apply this system in real robotic welding scenarios, this thesis finally reports some preliminary work in such an application

An Augmented Interface to Display Industrial Robot Faults

Technology advancement is changing the way industrial factories have to face an increasingly complex and competitive market. The fourth industrial revolution (known as industry 4.0) is also changing how human workers have to carry out tasks and actions. In fact, it is no longer impossible to think of a scenario in which human operators and industrial robots work side-by-side, sharing the same environment and tools. To realize a safe work environment, workers should trust robots as well as they trust human operators. Such goal is indeed complex to achieve, especially when workers are under stress conditions, such as when a fault occurs and the human operators are no longer able to understand what is happening in the industrial manipulator. Indeed, Augmented Reality (AR) can help workers to visualize in real-time robots’ faults. This paper proposes an augmented system that assists human workers to recognize and visualize errors, improving their awareness of the system. The system has been tested using both an AR see-through device and a smartphone

A Common Digital Twin Platform for Education, Training and Collaboration

The world is in transition driven by digitalization; industrial companies and educational institutions are adopting Industry 4.0 and Education 4.0 technologies enabled by digitalization. Furthermore, digitalization and the availability of smart devices and virtual environments have evolved to pro- duce a generation of digital natives. These digital natives whose smart devices have surrounded them since birth have developed a new way to process information; instead of reading literature and writing essays, the digital native generation uses search engines, discussion forums, and on- line video content to study and learn. The evolved learning process of the digital native generation challenges the educational and industrial sectors to create natural training, learning, and collaboration environments for digital natives. Digitalization provides the tools to overcome the aforementioned challenge; extended reality and digital twins enable high-level user interfaces that are natural for the digital natives and their interaction with physical devices. Simulated training and education environments enable a risk-free way of training safety aspects, programming, and controlling robots. To create a more realistic training environment, digital twins enable interfacing virtual and physical robots to train and learn on real devices utilizing the virtual environment. This thesis proposes a common digital twin platform for education, training, and collaboration. The proposed solution enables the teleoperation of physical robots from distant locations, enabling location and time-independent training and collaboration in robotics. In addition to teleoperation, the proposed platform supports social communication, video streaming, and resource sharing for efficient collaboration and education. The proposed solution enables research collaboration in robotics by allowing collaborators to utilize each other’s equipment independent of the distance between the physical locations. Sharing of resources saves time and travel costs. Social communication provides the possibility to exchange ideas and discuss research. The students and trainees can utilize the platform to learn new skills in robotic programming, controlling, and safety aspects. Cybersecurity is considered from the planning phase to the implementation phase. Only cybersecure methods, protocols, services, and components are used to implement the presented platform. Securing the low-level communication layer of the digital twins is essential to secure the safe teleoperation of the robots. Cybersecurity is the key enabler of the proposed platform, and after implementation, periodic vulnerability scans and updates enable maintaining cybersecurity. This thesis discusses solutions and methods for cyber securing an online digital twin platform. In conclusion, the thesis presents a common digital twin platform for education, training, and collaboration. The presented solution is cybersecure and accessible using mobile devices. The proposed platform, digital twin, and extended reality user interfaces contribute to the transitions to Education 4.0 and Industry 4.0

Augmented reality ARC welding learning application to enhance student’s motivation

This study introduces a mobile application integrated with augmented reality that can be used by Mechanical Engineering students to learn welding subject. Moreover, a study of perception among a sample of respondents who have used the app has been conducted. Normally, students will learn the theoretical concepts of welding in the class followed by instruction-based training in the workshop. However, students have difficulty to follow everything that has been taught in the class in a limited time. Welding is dangerous for beginners and the welding environments are harmful and injurious to the health. Furthermore, the preliminary study revealed that the students lacked sufficient materials to assist them during revision and need additional materials for welding learning. This study proposes the Mobile Arc Welding Learning (MAWL) app which use mobile app integrated with augmented reality technology to study user’s perceptions based on ease of use, engagement, learnability, satisfaction and usefulness. The research methodology in this study was adapted from Kuechler and Vaishnavi which consists of five phases: awareness of problems, suggestions, development, evaluation, and conclusion. The study incorporates the ARCS motivation model and constructivist theory to provide effective learning material. The evaluation results showed that the independent variables (Engagement, Satisfaction and Usefulness) are significantly related to the dependent variable (motivation). The results of the perception study indicate that the users strongly agreed on ease of use, learnability, satisfaction, usefulness and motivation, while they agreed on engagement. The main contribution of this study is the MAWL app which is a supplementary learning tool with the purpose to motivate students in welding learning. The results show that the ARCS motivation model and constructivist theory can be applied to develop effective learning materials found in the Mobile Arc Welding Learning (MAWL) app. Besides that, engagement, satisfaction and usefulness have positive influence on motivation. Thus, when designing and developing AR mobile-based welding learning app, factors including engagement, satisfaction and usefulness need to be integrated into the app to improve the student’s motivation towards the tool

Towards immersive designing of production processes using virtual reality techniques

The article provides a novel approach to the implementation of virtual reality within planning and design of manual processes and systems. The use of hardware and software required to perform different production - especially assembly - tasks in a virtual environment, using CAD parts as interactive elements, is presented. Considering the CAD parts, the format conversion problem is comprehensively described and solved using format conversion software to overcome the present poor data connectivity between the CAD system and VR hardware and software. Two examples of work processes have been made in a virtual environment: peg-in-hole and wall socket assembly. In the latter case, the traditional planning approach of manual assembly tasks using predetermined motion time system MTM-2 has been compared with a modern approach in which the assembly task is fully performed within a virtual environment. The comparison comprises a discussion on the assembly task execution times. In addition, general and specific advantages and disadvantages that arise in the immersive designing of production processes using virtual reality are presented, as well as reflections on teamwork and collaborative man-machine work. Finally, novel technologies are proposed to overcome the main problems that occur when implementing VR, such as time-consuming scene defining or tedious CAD software data conversion