Using virtual reality and 3D industrial numerical models for immersive interactive checklists

At the different stages of the PLM, companies develop numerous checklist-based procedures involving prototype inspection and testing. Besides, techniques from CAD, 3D imaging, animation and virtual reality now form a mature set of tools for industrial applications. The work presented in this article develops a unique framework for immersive checklist-based project reviews that applies to all steps of the PLM. It combines immersive navigation in the checklist, virtual experiments when needed and multimedia update of the checklist. It provides a generic tool, independent of the considered checklist, relies on the integration of various VR tools and concepts, in a modular way, and uses an original gesture recognition. Feasibility experiments are presented, validating the benefits of the approach

Asynchronous industrial collaboration : How virtual reality and virtual tools aid the process of maintenance method development and documentation creation

In the light of Industry 4.0, the field of Industrial Maintenance faces a large digital transformation, adopting Extended Reality (XR) technologies to aid industrial operations. For the manufacturing corporations that provide maintenance services, the efficiency of industrial maintenance plays a crucial role in the competitiveness and is tightly related to the technical documentation supporting maintenance. However, the process of documentation creation faces several challenges due to lack of access to the physical equipment and difficulties in remote communication between globally distributed departments. To address these shortcomings, this research investigates the utilization of Virtual Reality (VR) to facilitate asynchronous collaboration of globally dispersed departments involved in the pipeline of maintenance method and documentation creation. The presented proof-of-concept (the COVE-VR platform) has been developed as an academia-industry collaboration and evaluated iteratively with subject matter experts. The proposed VR platform consists of two virtual environments and eight virtual tools, which allow interaction with virtual prototypes (3D CAD models) and means of digital content creation. Our findings show the high relevance of the developed solution for the needs of industrial departments and the ability to support asynchronous collaboration among them. This article delivers qualitative findings on the value of VR technology and presents guidelines on how to develop virtual tools for digital content creation within VR, adaptable to other industrial contexts. We suggest providing embedded guidance and design consistency to ensure smooth interactions with virtual tools and further discuss the importance of proper positioning, the transparency of operations and the information property of generated content.publishedVersionPeer reviewe

Development of design tools for the evaluation of complex CAD models

The goal of this thesis is to invent means to enable 3D-models, created in SOLIDWORKS CAD-software, to be explored in a low-cost head-mounted display device Oculus Rift. Such a design tool would make possible to study 3D CAD-models in Virtual Reality with a very low cost and share design information for people who are not so familiar with CAD-models. As a result this can lower the product development effort for new products enabling, lower working time and improve both the quality of the final product and customer satisfaction. CAD-models as well as exported graphics format from SOLIDWORKS can’t directly be imported to Oculus Rift device. Therefore transformation pipeline need to be invented between the SOLIDWORKS and Oculus Rift. There are some options for this and their issues are studied in this thesis. As a best option it was found to use a conversion plugin tool to SOLIDWORKS which convert CAD-models to FBX-format. This format can be imported to Unity game engine software which is supported by Oculus Rift device. Graphics file conversion process in known to be imperfect. Therefore conversion of three SOLIDWORKS CAD-models have been analyzed; a Crane, a Pipe system, and a Hydraulic cylinder. This testing gave a good overview of possibilities and limitations of the graphics pipeline. It was found that ordinary CAD-models can be transferred to Oculus Rift without problems. However, there are some special features in CAD models which cause problems for in graphics file conversion

Elevator installation in virtual reality - How practicing in virtual reality affects learning experience and could it enhance the transfer of training

Purpose of this study was to find out how practicing in virtual reality affects learning experience and could it enhance the transfer of training. Study was carried out in industrial setting in collaboration with KONE corporation. Effects of the simulation training were evaluated with a survey and theme interviews. Based on this case study it is safe to assume that practicing in virtual reality affects learning experience positively, and enhances the conditions of transfer of training to happen by for example providing a clear context for the task trained and enabling concrete “doing by hands” experience. Simulation training also seems to have a positive effect to trainee confidence which affects the transfer process by adding personal motivation

Primena tehnologije proširene stvarnosti za poboljšanje bezbednosti i efikasnosti rada u industrijskim okruženjima

This doctoral dissertation presents a system based on augmented reality (AR) technology for improvement of safety and efficiency at work in industrial environments. The AR system is a solution intended to assign work tasks to workers, provide necessary instructions for implementing them, and issue related occupational safety instructions directly at the workplace. At the same time, the system provides ability to record all the issued instructions and control their proper execution. It is formulated a model of a typical technological process broad enough that can be applied in different industrial environments. Based on that, the functionality and the structure of the intended AR system is determined. A practical implementation of the system is done, and its evaluation performed in two concrete electroenergetic industrial environments. In these implementations different AR technologies are used and tested. An evaluation of the proposed AR system is done, and it is concluded that the proposed AR system improves the human resources management and ensures performing of occupational safety measures

Méthode interactive et par l'apprentissage pour la generation de trajectoire en conception du produit

The accessibility is an important factor considered in the validation and verification phase of the product design and usually dominates the time and costs in this phase. Defining the accessibility verification as the motion planning problem, the sampling based motion planners gained success in the past fifteen years. However, the performances of them are usually shackled by the narrow passage problem arising when complex assemblies are composed of large number of parts, which often leads to scenes with high obstacle densities. Unfortunately, humans’ manual manipulations in the narrow passage always show much more difficulties due to the limitations of the interactive devices or the cognitive ability. Meanwhile, the challenges of analyzing the end users’ response in the design process promote the integration with the direct participation of designers.In order to accelerate the path planning in the narrow passage and find the path complying with user’s preferences, a novel interactive motion planning method is proposed. In this method, the integration with a random retraction process helps reduce the difficulty of manual manipulations in the complex assembly/disassembly tasks and provide local guidance to the sampling based planners. Then a hypothesis is proposed about the correlation between the topological structure of the scenario and the motion path in the narrow passage. The topological structure refers to the medial axis (2D) and curve skeleton (3D) with branches pruned. The correlation runs in an opposite manner to the sampling based method and provide a new perspective to solve the narrow passage problem. The curve matching method is used to explore this correlation and an interactive motion planning framework that can learn from experience is constructed in this thesis. We highlight the performance of our framework on a challenging problem in 2D, in which a non-convex object passes through a cluttered environment filled with randomly shaped and located non-convex obstacles.L'accessibilitéest un facteur important pris en compte dans la validation et la vérification en phase de conception du produit et augmente généralement le temps et les coûts de cette phase. Ce domaine de recherche a eu un regain d’intérêt ces quinze dernières années avec notamment de nouveaux planificateurs de mouvement. Cependant, les performances de ces méthodes sont généralement très faibles lorsque le problème se caractérise par des passages étroits des assemblages complexes composées d'un grand nombre de pièces. Cela conduit souvent àdes scènes àforte densitéd'obstacles. Malheureusement, les manipulations manuelles des humains dans le passage étroit montrent toujours beaucoup de difficultés en raison des limitations des dispositifs interactifs ou la capacitécognitive. Pendant ce temps, les défis de l'analyse de la réponse finale des utilisateurs dans le processus de conception promeut l'intégration avec la participation directe des concepteurs.Afin d'accélérer la planification dans le passage étroit et trouver le chemin le plus conforme aux préférences de l'utilisateur, une nouvelle méthode de planification de mouvement interactif est proposée. Nous avons soulignéla performance de notre algorithme dans certains scénarios difficiles en 2D et 3D environnement.Ensuite, une hypothèse est proposésur la corrélation entre la structure topologique du scénario et la trajectoire dans le passage étroit. La méthode basée sur les courbures est utilisée pour explorer cette corrélation et un cadre de planification de mouvement interactif qui peut apprendre de l'expérience est construit dans cette thèse. Nous soulignons la performance de notre cadre sur un problème difficile en 2D, dans lequel un objet non-convexe passe à travers un environnement encombrérempli d'obstacles non-convexes de forme aléatoire et situés

The Benefits of Extended Reality for Technical Communication : Utilizing XR for Maintenance Documentation Creation and Delivery

The main goal of this dissertation is to explore the benefits of extended reality for technical communication. Both of these fields offer opportunities and also pose challenges to each other, and this dissertation provides insight into this relationship. The research was initiated by the author’s personal interest in both fields and also human-technology interaction and user needs in general. Even though this is an academic dissertation, it is first and foremost a practitioner’s view of these evolving technologies and their potential uses in industry and, specifically, in industrial maintenance and technical communication. Under the umbrella of extended reality and technical communication, this dissertation focuses on two main themes. The first part studies virtual reality as a technology to facilitate collaboration and digital content creation for technical documentation in industrial companies, and the second part explores the possibilities of augmented reality and smart glasses as a delivery channel for maintenance instructions. The developed concepts were tested by domain experts in user tests. The overall results of testing were positive, and domain experts expressed enthusiasm toward the concepts and technologies in general. The technical documentation process is an inherently collaborative process involving stakeholders from different teams and organizations, and virtual reality was evaluated to have a positive effect on that process, especially in the case of globally scattered teams. The developed tools were also rated positively for digital content creation. Therefore, virtual reality offers many benefits for technical documentation creation, an area where it has not been utilized until now. On the augmented reality side, domain experts were generally enthusiastic about the use of smart glasses even though the technologies are not yet mature enough for field use in industrial maintenance. Furthermore, the results show that content created in the technical communications industry standard, DITA XML, works well when delivered to smart glasses, and the same content can be single sourced to other delivery channels. The use of DITA XML, therefore, eliminates the need to tailor content for each delivery channel separately, and offers an effective way to create and update content for AR applications in industrial companies. This, in turn, can advance the use of AR technologies and related devices in field operations in industrial companies. In conclusion, the findings of this dissertation show that the fields of technical communication and extended reality have a significant amount of synergy. In this dissertation I establish use cases and guidelines for these areas

Analyse et simulation de mouvements d'atteinte contraints en position et orientation pour un humanoïde de synthèse

La simulation du geste humain est une thématique de recherche importante et trouve notamment une application dans l'analyse ergonomique pour l'aide à la conception de postes de travail. Le sujet de cette thèse concerne la génération automatique de tâches d'atteinte dans le plan horizontal pour un humanoïde. Ces dernières, à partir d'un objectif exprimé dans l'espace de la tâche, requièrent une coordination de l'ensemble des liaisons. L'une des principales difficultés rencontrées lors de la simulation de gestes réalistes est liée à la redondance naturelle de l'humain. Notre démarche est focalisée principalement sur deux aspects : - le mouvement de la main dans l'espace opérationnel (trajectoire spatiale et profil temporel), - la coordination des différentes sous-chaînes cinématiques. Afin de caractériser le mouvement humain, nous avons mené une campagne de capture de mouvements pour des gestes d'atteinte contraignant la position et l'orientation de la main dans le plan horizontal. Ces acquisitions nous ont permis de connaître l'évolution spatiale et temporelle de la main dans l'espace de la tâche, en translation et en rotation. Ces données acquises couplées à une méthode de rejeu ont également permis d'analyser les relations intrinsèques qui lient l'espace de la tâche à l'espace articulaire du mannequin. Le schéma de génération automatique de mouvements réalistes est basé sur une pile de tâche avec une approche cinématique. L'hypothèse retenue pour simuler le geste est de suivre le chemin le plus court dans l'espace de la tâche tout en bornant le coût dans l'espace articulaire. Un ensemble de paramètres permet de régler le schéma. Il en résulte une cartographie de réglages qui permet de simuler une classe de mouvements réalistes. Enfin, ce schéma de génération automatique de mouvements réalistes est validé par une comparaison quantitative et qualitative entre la simulation et le geste humain. ABSTRACT : The simulation of human movement is an active theme of research, particularly in ergonomic analysis to aid in the design of workstations. The aim of this thesis concerns the automatic generation of reaching tasks in the horizontal plane for a virtual humanoid. An objective expressed in the task space, requires coordination of all joints of the mannequin. The main difficulties encountered in the simulation of realistic movements is related to the natural redundancy of the human. Our approach is focused mainly on two aspects: - Motion of the hand's operator in the task space (spatial and temporal aspect), - Coordination of all kinematic chains. To characterize human movement, we conducted a set of motion capture with position and orientation constraints of the hand in the horizontal plane. These acquisitions allowed to know the spatial and temporal evolution of the hand in the task space, for translation and rotation aspects. These acquired data were coupled with a playback method to analyze the intrinsic relations that link the task space to joint space of the model. The automatic generation scheme of realistic motion is based on a stack of task with a kinematic approach. The assumption used to simulate the action is to follow the shortest path in the task space while limiting the cost in the joint space. The scheme is characterized by a set of parameters. A global map of parameter adjustment enables the simulation of a class of realistic movements. Finally, this scheme is validated quantitatively and qualitatively with comparison between the simulation and the human gesture

Vers une méthodologie de développement d'outils de réalité virtuelle pour faciliter la convergence métiers en conception de produits centrée sur l'homme

La conception concourante de produits matériels centrée sur l homme est basée sur une collaboration entre le concepteur mécanicien, l ergonome et le designer industriel. Cette collaboration souvent difficile peut être facilitée par l utilisation d objets intermédiaires de conception, tels que la Réalité Virtuelle (RV). Néanmoins, bien que largement utilisée dans l industrie, la RV souffre d un déficit d acceptation de la part des concepteurs de produits. Dans le cadre de ces travaux, nous proposons d utiliser la RV sous la forme d outils immersifs d assistance à la convergence multidisciplinaire développés selon une démarche anthropocentrée en fonction des besoins spécifiques à chaque projet de conception de produits. Afin d optimiser les délais de développement, nous proposons une méthodologie de conception d applications immersive dédiée : la méthodologie ASAP(As Soon As Possible). Une première série expérimentale a été conduite dans le cadre de contrats industriels d études et de recherche afin de valider la faisabilité de la méthodologie et l efficacité des outils développés.Une deuxième série expérimentale a été effectuée sur plus de 50 sujets dans le cadre de projets, cette fois, pédagogiques qui ont nécessité le développement de 12 applications. Elle a permis de valider quantitativement l influence des outils immersifs sur l efficacité perçue des phases de convergence interdisciplinaires ainsi que l influence de l approche proposée sur l acceptation de la RV par les concepteurs de produits. Ces travaux de thèse présentent une première approche qui, selon nous, permettra à terme, de faire évoluer l usage de la RV vers une intégration plus forte au sein des processus de conception de produits avec, par exemple, une plus large utilisation des applications immersives de modélisation 3D, réelles sources d innovation.Human centered and concurring product design is based on the collaboration between mechanical engineers, human factor experts and industrial designers. This collaboration is often difficult and can be eased through the use of intermediary objects such as Virtual Reality (VR). Nevertheless, even though VR is widely used in the industry, it suffers from a lack of acceptance by product designers. In the context of this research work, we suggest to use VR in the form of immersive multidisciplinary convergence support tools. These tools are developed in accordance with an anthropocentered approach, as a function of each specific product design project s needs. In order to optimize development times, we propose a dedicated immersive software design methodology : the ASAP methodology (As Soon As Possible). A first experiment, aiming to demonstrate the feasibility of the ASAP methodology and the effectiveness of the implemented immersive tools, has been conducted in the context of industrial product design projects. A second experiment, involving more than 50 participants, has been conducted in the context of educational product design projects and led to the development of 12 immersivetools. It demonstrated quantitatively the contribution of immersive tools to the perceived effectiveness of interdisciplinary convergence phases and the contribution of the ASAP methodology on the acceptation of VR by product designers. This research work describes a first approach that could, according to us, allow abetter integration of VR within product design processes.BELFORT-UTBM-SEVENANS (900942101) / SudocSudocFranceF