Product conceptualization through a 3D natural interface considering in real-time spatial and ergonomic restrictions

Currently, the conceptualization of products whose shape and configuration depends on the context is a highly time-consuming process since it has to be achieved asynchronously between the real environment of the product’s usage and the design office -- In general, the designer manually maps the context in order to create a 3D model of it and then to start the product design process -- The literature presents some proposals to digitalize the context without the need of a manual mapping -- However, these approaches are mainly computer-centric tools where the designer is desk-bound and (s)he does not have a clear spatial perception since the interactions with the 3D models are usually based on 2D interfaces -- On this research we aimed to prove that conceptualization of context-dependent products directly over its real environment through gesture-based modeling tools, allows the designer to consider spatial and ergonomic restrictions that the context imposes to the product, through the real-time analysis of the interaction user-context -- In order to prove that, we developed a tool called Äir-Modeling, in which the designer is able to create virtual conceptual products quickly and efficiently, taking advantage of hand gestures meanwhile (s)he is interacting directly with the real scenario in an Augmented Reality (AR) environment -- Air-Modeling also allows a continuous evaluation of the user postures involved in the product usage and assembly in order to analyze ergonomic risks, and perform the necessary changes in the product shape or configuration from early stages of the design process -- A test was carried out to prove the effects of the use of the proposed tool in the design process in comparison with the traditional way through traditional CAD packages -- We found that the real context can be used as an information input in real-time during product conceptualization -- Beside this, we could notice that virtual parts creation is more efficient from a 3D input than a 2D interface such as a mouse or a keyboard -- This was reflected in the experiment carried out in which 21 users conceptualized a bookcase for a given context using both Air-Modeling and a commercial CAD tool -- It was obtained a reduction in the modeling time using our tool on 76% of the cases with a final average reduction of 44%. Finally, we concluded that 3D modeling in AR environments using the hands as interface and the context as an information input in realtime, allows the designer to conceptualize potential solutions in quick and efficient manner, exploiting as much as possible, inspirational instants -- On the other hand, modeling in a natural scale directly over the real scene prevents the designer to draw his/her attention on dimensional details but allows him/her to focus on the product itself and its relation with the environment -- Besides, developing 3D models in natural scale allows analyzing the interaction between the user, the context and the virtual model for determining ergonomic issues related with the product usage or assembly process -- We believe that this kind of technologies makes the development of customized products more efficient by adding spatial and ergonomic restrictions to the conceptualization process in real-time -- This facilitates the convergence to the design solution, possibly avoiding some iteration in the design proces

Computational interaction techniques for 3D selection, manipulation and navigation in immersive VR

3D interaction provides a natural interplay for HCI. Many techniques involving diverse sets of hardware and software components have been proposed, which has generated an explosion of Interaction Techniques (ITes), Interactive Tasks (ITas) and input devices, increasing thus the heterogeneity of tools in 3D User Interfaces (3DUIs). Moreover, most of those techniques are based on general formulations that fail in fully exploiting human capabilities for interaction. This is because while 3D interaction enables naturalness, it also produces complexity and limitations when using 3DUIs. In this thesis, we aim to generate approaches that better exploit the high potential human capabilities for interaction by combining human factors, mathematical formalizations and computational methods. Our approach is focussed on the exploration of the close coupling between specific ITes and ITas while addressing common issues of 3D interactions. We specifically focused on the stages of interaction within Basic Interaction Tasks (BITas) i.e., data input, manipulation, navigation and selection. Common limitations of these tasks are: (1) the complexity of mapping generation for input devices, (2) fatigue in mid-air object manipulation, (3) space constraints in VR navigation; and (4) low accuracy in 3D mid-air selection. Along with two chapters of introduction and background, this thesis presents five main works. Chapter 3 focusses on the design of mid-air gesture mappings based on human tacit knowledge. Chapter 4 presents a solution to address user fatigue in mid-air object manipulation. Chapter 5 is focused on addressing space limitations in VR navigation. Chapter 6 describes an analysis and a correction method to address Drift effects involved in scale-adaptive VR navigation; and Chapter 7 presents a hybrid technique 3D/2D that allows for precise selection of virtual objects in highly dense environments (e.g., point clouds). Finally, we conclude discussing how the contributions obtained from this exploration, provide techniques and guidelines to design more natural 3DUIs

Simulation-Based Countermeasures Towards Accident Prevention : Virtual Reality Utilization in Industrial Processes and Activities

Despite growing industrial interests in fully immersive virtual reality (VR) applications for safety countermeasures, there is scanty research on the subject in the context of accident prevention during manufacturing processes and plant maintenance activities. This dissertation aims to explore and experiment with VR for accident prevention by targeting three workplace safety countermeasures: fire evacuation drills, hazard identification and risk assessments (HIRA), and emergency preparedness and response (EPR) procedures. Drawing on the virtual reality accident causation model (VR-ACM) (i.e., 3D modelling and simulation, accident causation, and safety drills) and the fire evacuation training model, two industrial 3D simulation models were utilized for the immersive assessment and training. These were a lithium-ion battery (LIB) manufacturing factory and a gas power plant (GPP). In total, five studies (publications) were designed to demonstrate the potential of VR in accident prevention during the manufacturing processes and maintenance activities at the facility conceptual stages. Two studies were with the LIB factory simulation to identify inherent hazards and assess risks for redesigning the factory to ensure workplace safety compliance. The other three studies constituted fire hazard identifications, emergency evacuations and hazard control/mitigations during the maintenance activity in the GPP simulation. Both study models incorporated several participants individually immersed in the virtual realm to experience the accident phenomena intuitively. These participants provided feedback for assessing the research objectives. Results of the studies indicated that several inherent hazards in the LIB factory were identified and controlled/mitigated. Secondly, the GPP experiment results suggested that although the maintenance activity in the virtual realm increased the perception of presence, a statistically significant delay was recorded at the pre-movement stage due to the lack of situational safety awareness. Overall, the study demonstrates that participants immersed in a VR plant maintenance activity and manufacturing factory process simulation environments can experience real-time emergency scenarios and conditions necessary for implementing the essential safety countermeasures to prevent accidents.Vaikka kiinnostus virtuaalitodellisuuden (VR) käyttöön turvallisuuden varotoimissa teollisuudessa on kasvanut, tutkimuksia ei ole juurikaan tehty onnettomuuksien ehkäisystä valmistus- ja kunnossapitotoiminnassa. Tämän väitöskirjan tavoitteena on tutkia ja kokeilla VR:ää tapaturmien ehkäisyssä kohdistuen kolmeen työpaikan turvallisuuden varotoimeen: paloharjoitukset, riskien arvioinnit sekä hätätilanteiden valmiusmenettelyt ja toimintasuunnitelmat (EPR). Kokemuksellisessa ja uppouttavassa koulutuksessa hyödynnettiin kahta teollisuuden 3D-simulointimallia, jotka nojautuvat virtuaalitodellisuuden onnettomuuksien aiheutumismalliin (VR-ACM) (eli 3D-mallinnus- ja simulointi, onnettomuussyy- ja turvallisuuskoulutus) sekä paloharjoitusmalliin. Nämä 3D-simulointimallit ovat litiuminoniakkuja (LIB) valmistava tehdas, joka rakennettiin Visual Components 3D-simulointiohjelmistolla (versio 4.0) ja kaasuvoimala (GPP) Unrealin reaaliaikaisella pelimoottorilla (versio 4.2). Yhteensä viisi tutkimusta (julkaisua) suunniteltiin havainnollistamaan VR:n potentiaalia tapaturmien ehkäisyssä valmistusprosessin layout-suunnittelun ja tehtaan konseptivaiheissa tehtävän kunnossapidon aikana. Kaksi tutkimusta tehtiin LIB-tehdassimulaatiolla vaarojen tunnistamiseksi sekä riskien arvioimiseksi. Tutkimukset tehtiin tehtaan uudelleensuunnittelua varten, työturvallisuuden noudattamisen varmistamiseksi. Muut kolme tutkimusta käsittelevät palovaaran tunnistamista, hätäevakuointia ja riskien vähentämistä huoltotoiminnan aikana GPP-simulaatiossa. Molemmissa tutkimusmalleissa oli useita virtuaalimaailmaan uppoutuneita osallistujia, jotka saivat kokea onnettomuudet yksilöllisesti ja intuitiivisesti. Osallistujat antoivat palautetta kokeen jälkeisessä kyselyssä. Kyselyn tuloksien avulla LIB-tehtaassa tunnistettiin ja lievennettiin useita vaaroja. GPP-kokeilun tulokset viittasivat siihen, että vaikka ylläpitotoiminta virtuaalimaailmassa lisäsi teleläsnäoloa, tilastollisesti merkittävä viive kirjattiin liikettä edeltävässä vaiheessa turvallisuustietoisuuden puuteen vuoksi. Kaiken kaikkiaan tutkimus osoittaa, että VR-laitoksen kunnossapitotoimintaan ja tuotantotehtaan prosessisimulaatioympäristöihin uppoutuvat osallistujat voivat kokea reaaliaikaisia hätäskenaarioita ja olosuhteita, jotka ovat välttämättömiä olennaisten turvallisuustoimien toteuttamiseksi.fi=vertaisarvioitu|en=peerReviewed

An approach based on VR to design industrial human-robot collaborative workstations

This paper presents an integrated approach for the design of human-robot collaborative workstations in industrial shop floors. In particular, the paper presents how to use virtual reality (VR) technologies to support designers in the creation of interactive workstation prototypes and in early validation of design outcomes. VR allows designers to consider and evaluate in advance the overall user experience, adopting a user-centered perspective. The proposed approach relies on two levels: the first allows designers to have an automatic generation and organization of the workstation physical layout in VR, starting from a conceptual description of its functionalities and required tools; the second aims at supporting designers during the design of human-machine interfaces (HMIs) by interaction mapping, HMI prototyping and testing in VR. The proposed approach has been applied on two realistic industrial case studies related to the design of an intensive warehouse and a collaborative assembly workstation for automotive industry, respectively. The two case studies demonstrate how the approach is suited for early prototyping of complex environments and human-machine interactions by taking into account the user experience from the early phases of design

Scalable and Extensible Augmented Reality with Applications in Civil Infrastructure Systems.

In Civil Infrastructure System (CIS) applications, the requirement of blending synthetic and physical objects distinguishes Augmented Reality (AR) from other visualization technologies in three aspects: 1) it reinforces the connections between people and objects, and promotes engineers’ appreciation about their working context; 2) It allows engineers to perform field tasks with the awareness of both the physical and synthetic environment; 3) It offsets the significant cost of 3D Model Engineering by including the real world background. The research has successfully overcome several long-standing technical obstacles in AR and investigated technical approaches to address fundamental challenges that prevent the technology from being usefully deployed in CIS applications, such as the alignment of virtual objects with the real environment continuously across time and space; blending of virtual entities with their real background faithfully to create a sustained illusion of co- existence; integrating these methods to a scalable and extensible computing AR framework that is openly accessible to the teaching and research community, and can be readily reused and extended by other researchers and engineers. The research findings have been evaluated in several challenging CIS applications where the potential of having a significant economic and social impact is high. Examples of validation test beds implemented include an AR visual excavator-utility collision avoidance system that enables spotters to ”see” buried utilities hidden under the ground surface, thus helping prevent accidental utility strikes; an AR post-disaster reconnaissance framework that enables building inspectors to rapidly evaluate and quantify structural damage sustained by buildings in seismic events such as earthquakes or blasts; and a tabletop collaborative AR visualization framework that allows multiple users to observe and interact with visual simulations of engineering processes.PHDCivil EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/96145/1/dsuyang_1.pd

Pre-Prototyping Framework: An Early Design Prototyping Methodology for Human-Centered Products and Workplaces

Prototyping is a crucial step in product design and development, but it is also known as the highest sunk cost. The top 20 Research and Development (R&D) department spends 142 billion dollars, yet 40 to 46 percent of this money goes into developing products that cannot even make to the market. Furthermore, the lack of comprehensive and widely accepted prototyping strategies and guidelines challenges the success of design teams in selecting options among a plethora of prototyping methodologies, techniques, and resources. The first shortcoming of current prototyping strategies is that they focus on the prototyping experience or the hands-on prototyping process without paying adequate attention to theoretical guidelines about prototyping factors and theories. The second limitation is the sparse guidelines of practical ‘know-how’ or what tools to use while building the prototype. The third shortcoming is that the existing prototyping strategies do not adequately incorporate the Human Factor Engineering (HFE) guidelines into the design of human-centered products. To address these shortcomings, this dissertation aims to formulate a Pre-Prototyping framework that aids designers in exploring prototyping strategies for human-centered products during the early design process. The overall objectives of this study are split into primary and secondary research objectives. Three secondary objectives are developed as building blocks of the overarching research objectives. First of all, Chapter 2 addresses the first secondary research objective by exploring how Performance Shaping Factors (PSFs) affect human performance and what prototyping strategies should be employed to capture it. Next, Chapter 3 addresses the second secondary research objective by proposing a computational prototyping method, which assists the designer in exploring the design space to integrate HFE design principles in the conceptual design process. Finally, the third secondary research objective is presented in Chapter 5, which explores the levels of human product interaction and the fidelity that plays a role in prototyping strategies. The secondary research objectives helped to gain deeper insight into prototyping for HCD based products. These insights are building blocks to address the three primary research questions and develop the Pre-Prototyping framework. The methodology to develop the Pre-Prototyping framework follows a similar step-by-step approach and workflows common to other prototyping frameworks in the prototyping literature; however, the proposed Pre-Prototyping framework adds HCD guidelines and proactive prototyping strategies by injecting HFE design principles. The distinction between this work and the existing prototyping framework is that the focus is on a Pre-Prototyping strategy rather than a hands-on prototyping activity. The second distinction is that this prototyping methodology is developed by focusing on the human-centered design since most of the existing methodologies concentrate solely on the prototyping experience. The proposed methodology comprises Prototyping Categories, Prototyping Dimensions, and Prototyping Toolbox along with Human Factor Guidelines. These different areas are combined in a framework that is currently presented using MS Excel User-Form. Designers can use this framework via Excel User-Form to conceptualize Pre-Prototyping strategies based on the specific HCD requirements. The proposed methodology is validated by an experiment that conducts 12 prototyping problems between the Intervention group and the Control group. Independent t-tests are be performed between the two groups. It is found that participants who use the proposed framework develop better Pre-Prototyping strategies than those who do not. In addition to the quantitative test, qualitative analysis is carried out by capturing the prototyping experience and attitude of the designers. Likert Scale and screen recordings data are used to gain further insight into participants' evaluation of the framework. It is discovered that the participants perceived the Pre-Prototyping framework to be helpful and they agree to use the Pre-Prototyping framework for prototyping human-centered products

Development of a toolkit for component-based automation systems

From the earliest days of mass production in the automotive industry there has been a progressive move towards the use of flexible manufacturing systems that cater for product variants that meet market demands. In recent years this market has become more demanding with pressures from legislation, globalisation and increased customer expectations. This has lead to the current trends of mass customisation in production. In order to support this manufacturing systems are not only becoming more flexible† to cope with the increased product variants, but also more agile‡ such that they may respond more rapidly to market changes. Modularisation§ is widely used to increase the agility of automation systems, such that they may be more readily reconfigured¶. Also with globalisation into India and Asia semi-automatic machines (machines that interact with human operators) are more frequently used to reduce capital outlay and increase flexibility. There is an increasing need for tools and methodologies that support this in order to improve design robustness, reduce design time and gain a competitive edge in the market. The research presented in this thesis is built upon the work from COMPAG/COMPANION (COMponent- based Paradigm for AGile automation, and COmmon Model for PArtNers in automatION), and as part of the BDA (Business Driven Automation), SOCRADES (Service Oriented Cross-layer infrastructure for Distributed smart Embedded deviceS), and IMC-AESOP (ArchitecturE for Service- Oriented Process – monitoring and control) projects conducted at Loughborough University UK. This research details the design and implementation of a toolkit for building and simulating automation systems comprising components with behaviour described using Finite State Machines (FSM). The research focus is the development of the engineering toolkit that can support the automation system lifecycle from initial design through commissioning to maintenance and reconfiguration as well as the integration of a virtual human. This is achieved using a novel data structure that supports component definitions for control, simulation, maintenance and the novel integration of a virtual human into the automation system operation

Flexible Automation and Intelligent Manufacturing: The Human-Data-Technology Nexus

This is an open access book. It gathers the first volume of the proceedings of the 31st edition of the International Conference on Flexible Automation and Intelligent Manufacturing, FAIM 2022, held on June 19 – 23, 2022, in Detroit, Michigan, USA. Covering four thematic areas including Manufacturing Processes, Machine Tools, Manufacturing Systems, and Enabling Technologies, it reports on advanced manufacturing processes, and innovative materials for 3D printing, applications of machine learning, artificial intelligence and mixed reality in various production sectors, as well as important issues in human-robot collaboration, including methods for improving safety. Contributions also cover strategies to improve quality control, supply chain management and training in the manufacturing industry, and methods supporting circular supply chain and sustainable manufacturing. All in all, this book provides academicians, engineers and professionals with extensive information on both scientific and industrial advances in the converging fields of manufacturing, production, and automation