A cyber-physical machine tools platform using OPC UA and MTConnect

Cyber-Physical Machine Tools (CPMT) represent a new generation of machine tools that are smarter, well connected, widely accessible, more adaptive and more autonomous. Development of CPMT requires standardized information modelling method and communication protocols for machine tools. This paper proposes a CPMT Platform based on OPC UA and MTConnect that enables standardized, interoperable and efficient data communication among machine tools and various types of software applications. First, a development method for OPC UA-based CPMT is proposed based on a generic OPC UA information model for CNC machine tools. Second, to address the issue of interoperability between OPC UA and MTConnect, an MTConnect to OPC UA interface is developed to transform MTConnect information model and its data to their OPC UA counterparts. An OPC UA-based CPMT prototype is developed and further integrated with a previously developed MTConnect-based CPMT to establish a common CPMT Platform. Third, different applications are developed to demonstrate the advantages of the proposed CPMT Platform, including an OPC UA Client, an advanced AR-assisted wearable Human-Machine Interface and a conceptual framework for CPMT powered cloud manufacturing environment. Experimental results have proven that the proposed CPMT Platform can significantly improve the overall production efficiency and effectiveness in the shop floor

Towards Designing a Planet Walk Simulation in a Controlled Environment

Planet Walk Simulation involves several parametric readings like physical human body conditions, geographic and atmospheric conditions, etc. In this work, we have set goals including system state monitoring, terrain mapping, and user navigation. Specific objectives are displaying the user’s telemetry, elevating their spatial awareness through short range object detection, and displaying their location relative to origin using wi-fi routers. The experimental results showed navigation motion paths for astronauts and identify obstacles in the path with the help of LiDAR and Hololens

INDUSTRIAL SAFETY USING AUGMENTED REALITY AND ARTIFICIAL INTELLIGENCE

Industrialization brought benefits to the development of societies, albeit at the cost of the safety of industrial workers. Industrial operators were often severely injured or lost their lives during the working process. The causes can be cuts or lacerations resulting from moving machine parts, burns or scalds resulting from touch, or mishandling of thermal, electrical, and chemical objects. Fatigue, distraction, or inattention can exacerbate the risk of industrial accidents. The accidents can cause service downtime of manufacturing machinery, leading to lower productivity and significant financial losses. Therefore, regulations and safety measures were formulated and overseen by the government and local authorities. Safety measures include effective training of workers, an inspection of the workplace, safety rules, safeguarding, and safety warning systems. For instance, safeguarding prevents contact with hazardous moving parts by isolating or stopping them, whereas a safety warning system detects accident risks and issues an alert warning. Warning systems were mostly mounted detection sensors and alerting systems. Mobile alerting devices can be gadgets such as phones, tablets, smartwatches, or smart glasses. Smart goggles can be utilized for industrial safety to protect, detect, and warn about potential risks. Adopting new technologies such as augmented reality and artificial intelligence can enhance the safety of workers in the industry. Augmented reality systems developed for head-mounted displays can extend workers’ perception of the environment. Artificial intelligence utilizing state-of-the-art sensors can improve industrial safety by making workers aware of potential hazards in the environment. For instance, thermal or infrared sensors can detect hot objects in the workplace. Built-in infrared sensors in smart glasses can detect the state of attention of users. Using smart glasses, potential hazards can be conveyed to industrial workers using various modalities, such as audial, visual, or tactile. We have successfully developed advanced safety systems for industrial workers. Our innovative approach incorporates cutting-edge technologies such as eye tracking, spatial mapping, and thermal imaging. By utilizing eye tracking, we are able to identify instances of user inattention, while spatial mapping allows us to analyze the user’s behavior and surroundings. Furthermore, the integration of thermal imaging enables us to detect hot objects within the user’s field of view. The first system we developed is a warning system that harnesses the power of augmented reality and artificial intelligence. This system effectively issues alerts and presents holographic warnings to combat instances of inattention or distraction. By utilizing visual cues and immersive technology, we aim to proactively prevent accidents and promote worker safety. The second safety system we designed involves the integration of a third-party thermal imaging system into smart glasses. Through this integration, our safety system overlays false-color holograms onto hot objects, enabling workers to easily identify and avoid potential hazards. To evaluate the effectiveness of our systems, we conducted comprehensive experiments with human participants. These experiments involved both qualitative and quantitative measurements, and we further conducted semi-structured interviews with the participants to gather their insights. The results and subsequent discussions from our experiments have provided valuable insights for the future implementation of safety systems. Through this research, we envision the continued advancement and refinement of safety technologies to further enhance worker safety in industrial settings

Study of Augmented Reality based manufacturing for further integration of quality control 4.0: a systematic literature review

Augmented Reality (AR) has gradually become a mainstream technology enabling Industry 4.0 and its maturity has also grown over time. AR has been applied to support different processes on the shop-floor level, such as assembly, maintenance, etc. As various processes in manufacturing require high quality and near-zero error rates to ensure the demands and safety of end-users, AR can also equip operators with immersive interfaces to enhance productivity, accuracy and autonomy in the quality sector. However, there is currently no systematic review paper about AR technology enhancing the quality sector. The purpose of this paper is to conduct a systematic literature review (SLR) to conclude about the emerging interest in using AR as an assisting technology for the quality sector in an industry 4.0 context. Five research questions (RQs), with a set of selection criteria, are predefined to support the objectives of this SLR. In addition, different research databases are used for the paper identification phase following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) methodology to find the answers for the predefined RQs. It is found that, in spite of staying behind the assembly and maintenance sector in terms of AR-based solutions, there is a tendency towards interest in developing and implementing AR-assisted quality applications. There are three main categories of current AR-based solutions for quality sector, which are AR-based apps as a virtual Lean tool, AR-assisted metrology and AR-based solutions for in-line quality control. In this SLR, an AR architecture layer framework has been improved to classify articles into different layers which are finally integrated into a systematic design and development methodology for the development of long-term AR-based solutions for the quality sector in the future

Optimization of computer-assisted intraoperative guidance for complex oncological procedures

Mención Internacional en el título de doctorThe role of technology inside the operating room is constantly increasing, allowing surgical procedures previously considered impossible or too risky due to their complexity or limited access. These reliable tools have improved surgical efficiency and safety. Cancer treatment is one of the surgical specialties that has benefited most from these techniques due to its high incidence and the accuracy required for tumor resections with conservative approaches and clear margins. However, in many cases, introducing these technologies into surgical scenarios is expensive and entails complex setups that are obtrusive, invasive, and increase the operative time. In this thesis, we proposed convenient, accessible, reliable, and non-invasive solutions for two highly complex regions for tumor resection surgeries: pelvis and head and neck. We explored how the introduction of 3D printing, surgical navigation, and augmented reality in these scenarios provided high intraoperative precision. First, we presented a less invasive setup for osteotomy guidance in pelvic tumor resections based on small patient-specific instruments (PSIs) fabricated with a desktop 3D printer at a low cost. We evaluated their accuracy in a cadaveric study, following a realistic workflow, and obtained similar results to previous studies with more invasive setups. We also identified the ilium as the region more prone to errors. Then, we proposed surgical navigation using these small PSIs for image-to-patient registration. Artificial landmarks included in the PSIs substitute the anatomical landmarks and the bone surface commonly used for this step, which require additional bone exposure and is, therefore, more invasive. We also presented an alternative and more convenient installation of the dynamic reference frame used to track the patient movements in surgical navigation. The reference frame is inserted in a socket included in the PSIs and can be attached and detached without losing precision and simplifying the installation. We validated the setup in a cadaveric study, evaluating the accuracy and finding the optimal PSI configuration in the three most common scenarios for pelvic tumor resection. The results demonstrated high accuracy, where the main source of error was again incorrect placements of PSIs in regular and homogeneous regions such as the ilium. The main limitation of PSIs is the guidance error resulting from incorrect placements. To overcome this issue, we proposed augmented reality as a tool to guide PSI installation in the patient’s bone. We developed an application for smartphones and HoloLens 2 that displays the correct position intraoperatively. We measured the placement errors in a conventional and a realistic phantom, including a silicone layer to simulate tissue. The results demonstrated a significant reduction of errors with augmented reality compared to freehand placement, ensuring an installation of the PSI close to the target area. Finally, we proposed three setups for surgical navigation in palate tumor resections, using optical trackers and augmented reality. The tracking tools for the patient and surgical instruments were fabricated with low-cost desktop 3D printers and designed to provide less invasive setups compared to previous solutions. All setups presented similar results with high accuracy when tested in a 3D-printed patient-specific phantom. They were then validated in the real surgical case, and one of the solutions was applied for intraoperative guidance. Postoperative results demonstrated high navigation accuracy, obtaining optimal surgical outcomes. The proposed solution enabled a conservative surgical approach with a less invasive navigation setup. To conclude, in this thesis we have proposed new setups for intraoperative navigation in two complex surgical scenarios for tumor resection. We analyzed their navigation precision, defining the optimal configurations to ensure accuracy. With this, we have demonstrated that computer-assisted surgery techniques can be integrated into the surgical workflow with accessible and non-invasive setups. These results are a step further towards optimizing the procedures and continue improving surgical outcomes in complex surgical scenarios.Programa de Doctorado en Ciencia y Tecnología Biomédica por la Universidad Carlos III de MadridPresidente: Raúl San José Estépar.- Secretario: Alba González Álvarez.- Vocal: Simon Droui

Smart Technologies for Precision Assembly

This open access book constitutes the refereed post-conference proceedings of the 9th IFIP WG 5.5 International Precision Assembly Seminar, IPAS 2020, held virtually in December 2020. The 16 revised full papers and 10 revised short papers presented together with 1 keynote paper were carefully reviewed and selected from numerous submissions. The papers address topics such as assembly design and planning; assembly operations; assembly cells and systems; human centred assembly; and assistance methods in assembly

Opportunities for using eye tracking technology in manufacturing and logistics: Systematic literature review and research agenda

Workers play essential roles in manufacturing and logistics. Releasing workers from routine tasks and enabling them to focus on creative, value-adding activities can enhance their performance and wellbeing, and it is also key to the successful implementation of Industry 4.0. One technology that can help identify patterns of worker-system interaction is Eye Tracking (ET), which is a non-intrusive technology for measuring human eye movements. ET can provide moment-by-moment insights into the cognitive state of the subject during task execution, which can improve our understanding of how humans behave and make decisions within complex systems. It also enables explorations of the subject’s interaction mode with the working environment. Earlier research has investigated the use of ET in manufacturing and logistics, but the literature is fragmented and has not yet been discussed in a literature review yet. This article therefore conducts a systematic literature review to explore the applications of ET, summarise its benefits, and outline future research opportunities of using ET in manufacturing and logistics. We first propose a conceptual framework to guide our study and then conduct a systematic literature search in scholarly databases, obtaining 71 relevant papers. Building on the proposed framework, we systematically review the use of ET and categorize the identified papers according to their application in manufacturing (product development, production, quality inspection) and logistics. Our results reveal that ET has several use cases in the manufacturing sector, but that its application in logistics has not been studied extensively so far. We summarize the benefits of using ET in terms of process performance, human performance, and work environment and safety, and also discuss the methodological characteristics of the ET literature as well as typical ET measures used. We conclude by illustrating future avenues for ET research in manufacturing and logistics

Visualization and Human-Machine Interaction

The digital age offers a lot of challenges in the eld of visualization. Visual imagery has been effectively used to communicate messages through the ages, to express both abstract and concrete ideas. Today, visualization has ever-expanding applications in science, engineering, education, medicine, entertainment and many other areas. Different areas of research contribute to the innovation in the eld of interactive visualization, such as data science, visual technology, Internet of things and many more. Among them, two areas of renowned importance are Augmented Reality and Visual Analytics. This thesis presents my research in the fields of visualization and human-machine interaction. The purpose of the proposed work is to investigate existing solutions in the area of Augmented Reality (AR) for maintenance. A smaller section of this thesis presents a minor research project on an equally important theme, Visual Analytics. Overall, the main goal is to identify the most important existing problems and then design and develop innovative solutions to address them. The maintenance application domain has been chosen since it is historically one of the first fields of application for Augmented Reality and it offers all the most common and important challenges that AR can arise, as described in chapter 2. Since one of the main problem in AR application deployment is reconfigurability of the application, a framework has been designed and developed that allows the user to create, deploy and update in real-time AR applications. Furthermore, the research focused on the problems related to hand-free interaction, thus investigating the area of speech-recognition interfaces and designing innovative solutions to address the problems of intuitiveness and robustness of the interface. On the other hand, the area of Visual Analytics has been investigated: among the different areas of research, multidimensional data visualization, similarly to AR, poses specific problems related to the interaction between the user and the machine. An analysis of the existing solutions has been carried out in order to identify their limitations and to point out possible improvements. Since this analysis delineates the scatterplot as a renowned visualization tool worthy of further research, different techniques for adapting its usage to multidimensional data are analyzed. A multidimensional scatterplot has been designed and developed in order to perform a comparison with another multidimensional visualization tool, the ScatterDice. The first chapters of my thesis describe my investigations in the area of Augmented Reality for maintenance. Chapter 1 provides definitions for the most important terms and an introduction to AR. The second chapter focuses on maintenance, depicting the motivations that led to choose this application domain. Moreover, the analysis concerning open problems and related works is described along with the methodology adopted to design and develop the proposed solutions. The third chapter illustrates how the adopted methodology has been applied in order to assess the problems described in the previous one. Chapter 4 describes the methodology adopted to carry out the tests and outlines the experimental results, whereas the fifth chapter illustrates the conclusions and points out possible future developments. Chapter 6 describes the analysis and research work performed in the eld of Visual Analytics, more specifically on multidimensional data visualizations. Overall, this thesis illustrates how the proposed solutions address common problems of visualization and human-machine interaction, such as interface de- sign, robustness of the interface and acceptance of new technology, whereas other problems are related to the specific research domain, such as pose tracking and reconfigurability of the procedure for the AR domain