Augmented Reality in Industry 4.0

Since the origins of Augmented Reality (AR), industry has always been one of its prominent application domains. The recent advances in both portable and wearable AR devices and the new challenges introduced by the fourth industrial revolution (renowned as industry 4.0) further enlarge the applicability of AR to improve the productiveness and to enhance the user experience. This paper provides an overview on the most important applications of AR regarding the industry domain. Key among the issues raised in this paper are the various applications of AR that enhance the user's ability to understand the movement of mobile robot, the movements of a robot arm and the forces applied by a robot. It is recommended that, in view of the rising need for both users and data privacy, technologies which compose basis for Industry 4.0 will need to change their own way of working to embrace data privacy

Study of Augmented Reality based manufacturing for further integration of quality control 4.0: a systematic literature review

Augmented Reality (AR) has gradually become a mainstream technology enabling Industry 4.0 and its maturity has also grown over time. AR has been applied to support different processes on the shop-floor level, such as assembly, maintenance, etc. As various processes in manufacturing require high quality and near-zero error rates to ensure the demands and safety of end-users, AR can also equip operators with immersive interfaces to enhance productivity, accuracy and autonomy in the quality sector. However, there is currently no systematic review paper about AR technology enhancing the quality sector. The purpose of this paper is to conduct a systematic literature review (SLR) to conclude about the emerging interest in using AR as an assisting technology for the quality sector in an industry 4.0 context. Five research questions (RQs), with a set of selection criteria, are predefined to support the objectives of this SLR. In addition, different research databases are used for the paper identification phase following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) methodology to find the answers for the predefined RQs. It is found that, in spite of staying behind the assembly and maintenance sector in terms of AR-based solutions, there is a tendency towards interest in developing and implementing AR-assisted quality applications. There are three main categories of current AR-based solutions for quality sector, which are AR-based apps as a virtual Lean tool, AR-assisted metrology and AR-based solutions for in-line quality control. In this SLR, an AR architecture layer framework has been improved to classify articles into different layers which are finally integrated into a systematic design and development methodology for the development of long-term AR-based solutions for the quality sector in the future

Registro espacial 2D–3D para a inspeção remota de subestações de energia

Remote inspection and supervisory control are critical features for smart factories, civilian surveillance, power systems, among other domains. For reducing the time to make decisions, operators must have both a high situation awareness, implying a considerable amount of data to be presented, and minimal sensory load. Recent research suggests the adoption of computer vision techniques for automatic inspection, as well as virtual reality (VR) as an alternative to traditional SCADA interfaces. Nevertheless, although VR may provide a good representation of a substation’s state, it lacks some real-time information, available from online field cameras and microphones. Since these two sources of information (VR and field information) are not integrated into one single solution, we miss the opportunity of using VR as a SCADA-aware remote inspection tool, during operation and disaster-response routines. This work discusses a method to augment virtual environments of power substations with field images, enabling operators to promptly see a virtual representation of the inspected area's surroundings. The resulting environment is integrated with an image-based state inference machine, continuously checking the inferred states against the ones reported by the SCADA database. Whenever a discrepancy is found, an alarm is triggered and the virtual camera can be immediately teleported to the affected region, speeding up system reestablishment. The solution is based on a client-server architecture and allows multiple cameras deployed in multiple substations. Our results concern the quality of the 2D–3D registration and the rendering framerate for a simple scenario. The collected quantitative metrics suggest good camera pose estimations and registrations, as well as an arguably optimal rendering framerate for substations' equipment inspection.CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível SuperiorCEMIG - Companhia Energética de Minas GeraisCNPq - Conselho Nacional de Desenvolvimento Científico e TecnológicoFAPEMIG - Fundação de Amparo a Pesquisa do Estado de Minas GeraisTese (Doutorado)A inspeção remota e o controle supervisório são requisitos críticos para fábricas modernas, vigilância de civis, sistemas de energia e outras áreas. Para reduzir o tempo da tomada de decisão, os operadores precisam de uma elevada consciência da situação em campo, o que implica em uma grande quantidade de dados a serem apresentados, mas com menor carga sensorial possível. Estudos recentes sugerem a adoção de técnicas de visão computacional para inspeção automática, e a Realidade Virtual (VR) como uma alternativa às interfaces tradicionais do SCADA. Entretanto, apesar de fornecer uma boa representação do estado da subestação, os ambientes virtuais carecem de algumas informações de campo, provenientes de câmeras e microfones. Como essas duas fontes de dados (VR e dispositivos de captura) não são integrados em uma única solução, perde-se a oportunidade de usar VR como uma ferramenta de inspeção remota conectada ao SCADA, durante a operação e rotinas de respostas a desastres. Este trabalho trata de um método para aumentar ambientes virtuais de subestações com imagens de campo, permitindo aos operadores a rápida visualização de uma representação virtual do entorno da área monitorada. O ambiente resultante é integrado com uma máquina de inferência estados por imagens, comparando continuamente os estados inferidos com aqueles reportados pela base SCADA. Na ocasião de uma discrepância, um alarme é gerado e possibilita que a câmera virtual seja imediatamente teletransportada para a região afetada, acelerando o processo de retomada do sistema. A solução se baseia em uma arquitetura cliente-servidor e permite múltiplas câmeras presentes em múltiplas subestações. Os resultados dizem respeito à qualidade do registro 2D–3D e à taxa de renderização para um cenário simples. As métricas quantitativas coletadas sugerem bons níveis de registro e estimativa de pose de câmera, além de uma taxa ótima de renderização para fins de inspeção de equipamentos em subestações

Probabilistic RGB-D Odometry based on Points, Lines and Planes Under Depth Uncertainty

This work proposes a robust visual odometry method for structured environments that combines point features with line and plane segments, extracted through an RGB-D camera. Noisy depth maps are processed by a probabilistic depth fusion framework based on Mixtures of Gaussians to denoise and derive the depth uncertainty, which is then propagated throughout the visual odometry pipeline. Probabilistic 3D plane and line fitting solutions are used to model the uncertainties of the feature parameters and pose is estimated by combining the three types of primitives based on their uncertainties. Performance evaluation on RGB-D sequences collected in this work and two public RGB-D datasets: TUM and ICL-NUIM show the benefit of using the proposed depth fusion framework and combining the three feature-types, particularly in scenes with low-textured surfaces, dynamic objects and missing depth measurements.Comment: Major update: more results, depth filter released as opensource, 34 page

Digital Factory and Virtual Reality: Teaching Virtual Reality Principles with Game Engines

Virtual reality (VR) is widely used in various industrial applications. All leading industrial manufacturing companies today have a strategy called the ‘concept of a digital factory’ where all aspects of manufacturing are digitally verified on digital mock-ups prior to physical manufacturing. Other than that, it is a rapidly developing new medium and further development of VR and IT will open up new possibilities. The new concept of Industry 4.0 is based on using approaches like the Internet of Things, Cloud Computing, Cyber-Physical Systems and Virtual Reality. With the decreasing cost of VR devices, even smaller businesses are able to implement such technologies. It is therefore crucial that mechanical engineering graduates are familiar with these new technologies and trends. We had to use unconventional methods to educate mechanical engineering students in the latest trends in IT and VR. Back in 2010, there were almost no tools available for teaching how to create industry-themed VR environments, which did not require complicated coding, so we decided to make our own. To simplify the development, we used Source Engine as the core and enhanced it with a library of textures, models and scripts we called DigiTov. Although Source Engine is a game engine, the master logic of VR development is the same as for professional SW products. In autumn 2015, a group of 10 students modified the DigiTov for Unity3D, forming a team made up of different roles

Investigating the Feasibility, Procedures, Benefits, and Challenges of Using Mixed Reality in Construction Projects: An Electrical Layout Case Study

Innovation and development of technology in the field of visualization have affected the way of designing, presenting, and interacting of the design in the architecture and construction industry. With the invention of the Mixed Reality (MR) device, HoloLens, 2016, many researchers and construction practitioners have marched their research of implementing the MR application in the construction industry. However, there is scarce research on the MR application review, the workflow of using BIM models with the HoloLens, its implementation in the site, and the benefits and challenges of the MR application in the construction site. The gap in the research needs yet to be filled, and this research paper will help to bridge the gap. For this research, the MR application that was available in the market at the time of research were identified through internet research and literature review. The MR applications were installed in two HoloLens and were tested on-site and lab for the construction of the electrical layout. The capabilities of the MR application were studied in the lab and site. The features including scaling, alignment, move, drag, rotate, team collaboration, navigation, and measurement utilities were studied. From the study, the MR application provides several benefits in on-site layout and as-built verification of electrical layout including 3D visualization, less manual work, instruction and demonstration of the construction process, error detection, remote support, quick decision-making, and easy documentation, and efficient progress tracking. Along with the benefits, the MR application has a few challenges while applying it on the site including stud locations, typical rooms, and model drifting. In addition, the limitations of using the MR application were studied. Future research will focus on solving one of the challenges, model drifting, through automatic alignment with survey point references

Advancing the Decarbonization of the Construction Sector: Lifecycle Quality and Performance Assurance of Nearly Zero-Energy Buildings

Dealing with and maintaining high-quality standards in the design and construction phases is challenging, especially for on-site construction. Issues like improper implementation of building components and poor communication can widen the gap between design specifications and actual conditions. To prevent this, particularly for energy-efficient buildings, it is’s vital to develop resili-ent, sustainable strategies. These should optimize resource use, minimize environmental impact, and enhance livability, contributing to carbon neutrality by 2050 and climate change mitigation. Traditional post-occupancy evaluations, which identify defects after construction, are impractical for addressing energy performance gaps. A new, real-time inspection approach is necessary throughout the construction process. This paper suggests an innovative guideline for prefabricated buildings, emphasizing digital ‘self-instruction’ and ‘self-inspection’. These procedures ensure ac-tivities impacting quality adhere to specific instructions, drawings, and 3D models, incorporating the relevant acceptance criteria to verify completion. This methodology, promoting alignment with planned energy-efficient features, is supported by BIM-based software and Augmented Reality (AR) tools, embodying Industry 4.0 principles. BIM (Building Information Modelling) and AR bridge the gap between virtual design and actual construction, improving stakeholder communication and enabling real-time monitoring and adjustments. This integration fosters accuracy and efficiency, which are key for energy-efficient and nNearly zZero-eEnergy bBuildings, marking a shift towards a more precise, collaborative, and environmentally sensible construction industry