Developing augmented reality capabilities for industry 4.0 small enterprises: Lessons learnt from a content authoring case study

Augmented reality (AR) has been proposed as a disruptive and enabling technology within the Industry 4.0 manufacturing paradigm. The complexity of the AR content creation process results in an inability for Small Enterprise (SE)to create bespoke,flexibleARtraining support “in-house” and is a potential barrier to industrial adoption of AR. Presently, AR content creation requires a range of specialist knowledge (e.g. 3D modelling, interface design, programming and spatial tracking) and may involve infrastructure changes (e.g. fiducial markers, cameras) and disruption to workflow. The research reported in this paper concerns the development and deployment of an Augmented Repair Training Application (ARTA); a templatebased interface to support end user (shop floor) AR content creation. The proposed methodology and implementation are discussed and evaluated in a real-world industrial case study in collaboration with a Small Enterprise (SE) in the Used and Waste Electronic and Electrical Equipment sector (UEEE/WEEE). The need for end user friendly templates is presented in the conclusion alongside further related work

Context-Enabled Visualization Strategies for Automation Enabled Human-in-the-loop Inspection Systems to Enhance the Situation Awareness of Windstorm Risk Engineers

Insurance loss prevention survey, specifically windstorm risk inspection survey is the process of investigating potential damages associated with a building or structure in the event of an extreme weather condition such as a hurricane or tornado. Traditionally, the risk inspection process is highly subjective and depends on the skills of the engineer performing it. This dissertation investigates the sensemaking process of risk engineers while performing risk inspection with special focus on various factors influencing it. This research then investigates how context-based visualizations strategies enhance the situation awareness and performance of windstorm risk engineers. An initial study investigated the sensemaking process and situation awareness requirements of the windstorm risk engineers. The data frame theory of sensemaking was used as the framework to carry out this study. Ten windstorm risk engineers were interviewed, and the data collected were analyzed following an inductive thematic approach. The themes emerged from the data explained the sensemaking process of risk engineers, the process of making sense of contradicting information, importance of their experience level, internal and external biases influencing the inspection process, difficulty developing mental models, and potential technology interventions. More recently human in the loop systems such as drones have been used to improve the efficiency of windstorm risk inspection. This study provides recommendations to guide the design of such systems to support the sensemaking process and situation awareness of windstorm visual risk inspection. The second study investigated the effect of context-based visualization strategies to enhance the situation awareness of the windstorm risk engineers. More specifically, the study investigated how different types of information contribute towards the three levels of situation awareness. Following a between subjects study design 65 civil/construction engineering students completed this study. A checklist based and predictive display based decision aids were tested and found to be effective in supporting the situation awareness requirements as well as performance of windstorm risk engineers. However, the predictive display only helped with certain tasks like understanding the interaction among different components on the rooftop. For remaining tasks, checklist alone was sufficient. Moreover, the decision aids did not place any additional cognitive demand on the participants. This study helped us understand the advantages and disadvantages of the decision aids tested. The final study evaluated the transfer of training effect of the checklist and predictive display based decision aids. After one week of the previous study, participants completed a follow-up study without any decision aids. The performance and situation awareness of participants in the checklist and predictive display group did not change significantly from first trial to second trial. However, the performance and situation awareness of participants in the control condition improved significantly in the second trial. They attributed this to their exposure to SAGAT questionnaire in the first study. They knew what issues to look for and what tasks need to be completed in the simulation. The confounding effect of SAGAT questionnaires needs to be studied in future research efforts

Gestaltungskonzept für Augmented Reality unterstütztes Training an manuellen Montagearbeitsplätzen

Forschungsvorhaben – Die Komplexitätssteigerung manueller Montageprozesse erfordert hochqualifizierte Arbeitskräfte, um die diffizilen Prozesse zu beherrschen. Um eine effektive und effiziente Ausbildung dieser Personen sicherzustellen, wird der Einsatz AR-basierter Trainingssysteme zunehmend relevant für die Industrie. Momentan mangelt es allerdings an wissenschaftlichen Untersuchungen im industriellen Nutzungskontext. Aufgrund dessen fehlen notwendige Erkenntnisse zur Gestaltung und Evaluation dieser Assistenzsysteme. Dem Ziel, diese Limitation zu schließen, widmet sich die vorliegende Dissertation. Forschungsrahmen – Das Vorgehen zur Erarbeitung der genannten Zielsetzung orientiert sich an der gestaltungsorientierten Forschung. Dieses Paradigma forciert die iterative Gestaltung von problemlösungsorientierten Artefakten mit Hilfe einer stringenten Anwendung wissenschaftlicher Methoden. Forschungsergebnisse – Durch Anwendung des zuvor erwähnten Forschungsparadigmas werden drei Artefakte gestaltet. Die Grundlage dafür liefern die Ergebnisse einer umfangreichen strukturierten Literaturrecherche und die Analyse domänenspezifischer Anforderungen. Basierend auf diesen wird zunächst ein menschzentriertes Vorgehensmodell zur Analyse, Gestaltung und Evaluation von AR-basierten Trainingssystemen im industriellen Nutzungskontext erarbeitet (Artefakt 1), welches seine reproduzierbare Verwendbarkeit durch ausgewählte Methoden und Empfehlungen in allen Ablaufphasen sicherstellt. Darauf aufbauend wird eine HMD-basierte Trainingssoftware instanziiert (Artefakt 2) und durch geeignete Evaluation systematisch zu einer gebrauchstauglichen Anwendung weiterentwickelt. Diese wird einem abschließenden Test unterzogen und im Vergleich zu zwei etablierten Trainingskonzepten empirisch erprobt. Die daraus resultierenden Ergebnisse verdeutlichen sowohl die Nützlichkeit des Vorgehensmodells als auch die Gebrauchstauglichkeit des innovativen Trainingssystems. Beruhend auf den zahlreichen Erfahrungen und Erkenntnissen dieser Dissertation, werden abschließende Empfehlungen (Artefakt 3) dargelegt, welche die erfolgreiche Durchführung ähnlicher wissenschaftlicher Arbeiten sicherstellen. Einschränkungen – Die umfangreichen multimodalen Funktionalitäten der Trainingssoftware wurden für eine konkrete prozedurale Montagetätigkeit entwickelt und am Beispiel eines industriellen Referenzarbeitsplatzes mit potentiellen Anwendern erprobt. Eine Anpassung der Software auf weitere Anwendungsfälle ist aufgrund des immensen Programmieraufwands sehr zeitaufwändig. Dadurch ist die Skalierbarkeit der Software stark limitiert. Angehende Forschungsprojekte sollten daher den Einsatz von Autorenwerkzeugen untersuchen, um eine effiziente Content-Erstellung zu gewährleisten. Darüber hinaus wurden keine Einflüsse der AR-Technologie auf das Langzeitgedächtnis erforscht. Diese Limitation eröffnet ein weiteres interessantes Forschungsfeld für zukünftige Untersuchungen. Implikationen – Diese Dissertation liefert sowohl wissenschaftliche als auch praxisbezogene Implikationen. Demnach schließen die Erkenntnisse zur Gestaltung und Evaluation AR-basierter Trainingssysteme bestehende Forschungslücken und gewährleisten eine reproduzierbare Instanziierung weiterer solcher Assistenzsysteme. Eine gebrauchstaugliche HMD-basierte Trainingssoftware bietet Industrieunternehmen mit manuellen Montageprozessen zudem Einsparungspotentiale durch eine vollkommen neuartige und hocheffektive Ausbildungsmöglichkeit.Purpose – The growing complexity of manual assembly processes require highly skilled workers to deal with such challenging tasks. Therefore, AR-based learning systems become more and more interesting for the industry promising to ensure effective and efficient learning processes. However, scientific research in the field of AR-based learning, especially in the industrial domain, is still very limited. For this reason, necessary knowledge for the design and evaluation of such assistive systems is lacking. This dissertation aims to close these limitations. Approach – The framework of the current scientific work is based on design science research (DSR). This research paradigm attempts to solve practical problems by developing purpose-oriented artifacts with rigorous scientific methods. Findings – Three artifacts are designed using the DSR technique. Hereof, the results of a comprehensive literature survey and an analysis of domain specific requirements provide the foundation. Based on this, a human-centered framework for analyzing, designing and evaluating AR-based learning systems in the industrial context is elaborated (artifact 1). Through well-chosen methods and recommendations in all three phases, a reproducible approach can be guaranteed. By applying this framework, a HMD-based learning software (artifact 2) is developed through several iterative evaluations with potential users using the example of a real internal combustion engine assembly task in order to ensure a high usability. Finally, the software is compared to two traditional approaches (paper-based and trainer-based learning). The results validate the utility of the framework as well as that of the innovative HMD-based learning approach. Based on numerous findings and empirical knowledge, several recommendations are derived to conclude this dissertation and facilitate forthcoming research. Limitations – The elaborate multimodal functionalities of the training software are developed and systematically improved using a concrete procedural internal combustion engine assembly task. This leads to a tremendous and time consuming programming effort as soon as individual software adjustments or assignments are requested by the industrial domain. Therefore, the software scalability is very limited. Due to the previously mentioned limitation, future research should concentrate its investigation into developing authoring tools which enable an efficient AR content creation. Furthermore, the impact of AR on the shortterm memory is only analyzed in this dissertation which opens up an additional interesting research area for future explorations. Implications – This current thesis provides scientific as well as practical oriented implications. The results regarding the design and evaluation of AR-based learning systems ensure a reproducible scientific procedure to instantiate these assistive systems further. In addition, initial insights regarding the use of these systems in the industrial domain are presented, therefore closing current research gaps. A HMD-based learning software offers the opportunity for companies with manual assembly tasks to conserve money due to a completely new and highly-effective training possibility