Holdable Haptic Device for 4-DOF Motion Guidance

Hand-held haptic devices can allow for greater freedom of motion and larger workspaces than traditional grounded haptic devices. They can also provide more compelling haptic sensations to the users' fingertips than many wearable haptic devices because reaction forces can be distributed over a larger area of skin far away from the stimulation site. This paper presents a hand-held kinesthetic gripper that provides guidance cues in four degrees of freedom (DOF). 2-DOF tangential forces on the thumb and index finger combine to create cues to translate or rotate the hand. We demonstrate the device's capabilities in a three-part user study. First, users moved their hands in response to haptic cues before receiving instruction or training. Then, they trained on cues in eight directions in a forced-choice task. Finally, they repeated the first part, now knowing what each cue intended to convey. Users were able to discriminate each cue over 90% of the time. Users moved correctly in response to the guidance cues both before and after the training and indicated that the cues were easy to follow. The results show promise for holdable kinesthetic devices in haptic feedback and guidance for applications such as virtual reality, medical training, and teleoperation.Comment: Submitted to IEEE World Haptics Conference 201

Leveraging Multimodal Interaction and Adaptive Interfaces for Location-based Augmented Reality Islamic Tourism Application

A Location-based Augmented Reality (LBAR) application leveraging multimodal interaction and adaptive interface based on Islamic tourism information was proposed to enhance user experience while travelling. LBAR has the potential to improve tourist experience and help tourists to access relevant information, thus improving their knowledge regarding touristic destination while increasing levels of their entertainment throughout the process. In LBAR application, Point of Interest (POI) displayed are exposed to the “occlusion problem” where the AR contents are visually redundant and overlapping with one another causing the users to loose valuable information. Previous research have suggested the design of AR POI which help user to see the augmented POI clearly. The user can click on the desired POI but it still displays a large amount of POI. From our best study, there is limitation of research studying on how to minimize the amount of displayed POI based on user’s current needs. Therefore, in this paper we suggest to use an adaptive user interface and multimodal interaction to solve this problem. We discussed the process of analysing and designing the user interfaces of previous studies. The proposed mobile solution was presented by explaining the application contents, the combination of adaptive multimodal inputs, system’s flow chart and multimodal task definition. Then the user evaluation was conducted to measure the level of satisfaction in terms of the usability of the application. A total of 24 Islamic tourists have participated in this study. The findings revealed that the average SUS score of 75.83 of respondents agree in terms of satisfaction of the LBAR application to be utilized while traveling. Finally, we conclude this paper by providing the suggestion of future works

Conceptualization and Design of a Workflow Management System Front End for Augmented Reality Headsets

A currently discussed approach to increase efficiency during task execution, inter alia to reduce error rates and execution times, is the utilization of headset-based augmented reality systems (HARS). Additional to direct task support, HARSes can offer workflow management and control functions. However, these are only covered very limitedly by existing design-oriented approaches. Thus, users have to resort to additional devices, decreasing efficiency, and usability. Based on a three-step systematic literature analysis and two focus groups, we present a novel tentative design theory for HARSes supporting the full range of workflow management and control functions. Our design theory consists of four design requirements and nine design principles and is the basis for a software artifact prototype. Both our tentative design theory and software artifact are formatively evaluated by a third focus group. Our contributions add to the prescriptive knowledge base of the community and may be adapted by researchers and practitioners

Multi-3D-Models Registration-Based Augmented Reality (AR) Instructions for Assembly

This paper introduces a novel, markerless, step-by-step, in-situ 3D Augmented Reality (AR) instruction method and its application - BRICKxAR (Multi 3D Models/M3D) - for small parts assembly. BRICKxAR (M3D) realistically visualizes rendered 3D assembly parts at the assembly location of the physical assembly model (Figure 1). The user controls the assembly process through a user interface. BRICKxAR (M3D) utilizes deep learning-trained 3D model-based registration. Object recognition and tracking become challenging as the assembly model updates at each step. Additionally, not every part in a 3D assembly may be visible to the camera during the assembly. BRICKxAR (M3D) combines multiple assembly phases with a step count to address these challenges. Thus, using fewer phases simplifies the complex assembly process while step count facilitates accurate object recognition and precise visualization of each step. A testing and heuristic evaluation of the BRICKxAR (M3D) prototype and qualitative analysis were conducted with users and experts in visualization and human-computer interaction. Providing robust 3D AR instructions and allowing the handling of the assembly model, BRICKxAR (M3D) has the potential to be used at different scales ranging from manufacturing assembly to construction

Augmented Reality: Emergent Applications and Opportunities for Industry 4.0

As the industry evolves further into digitalization, companies grow interested in new solutions made possible by the technologies of the Industry 4.0. One such paradigm is that of Augmented Reality (AR), which has seen significant growth in recent years, but still with great room for improvement, and still with many people unaware of its potential benefits. In its implementations, AR has generated value across a wide range of applications that aim at reducing the frequency of human error, decreasing the training time of professionals, and improving workflow. The purpose of this study is to understand the existing uses of AR across different fields, including its current state-of-the-art applications in the industrial sector, and find ways in which its systems in production lines could be improved, with a focus on the interaction between the user and the system, presenting a series of proposed solutions. To conclude, possible opportunities that could aid industry professionals and operators of manufacturing systems supported by AR are presented and discussed

Augmented reality: emergent applications and opportunities for industry 4.0

As the industry evolves further into digitalization, companies grow interested in new solutions made possible by the technologies of the Industry 4.0. One such paradigm is that of Augmented Reality (AR), which has seen significant growth in recent years, but still with great room for improvement, and still with many people unaware of its potential benefits. In its implementations, AR has generated value across a wide range of applications that aim at reducing the frequency of human error, decreasing the training time of professionals, and improving workflow. The purpose of this study is to understand the existing uses of AR across different fields, including its current state-of-the-art applications in the industrial sector, and find ways in which its systems in production lines could be improved, with a focus on the interaction between the user and the system, presenting a series of proposed solutions. To conclude, possible opportunities that could aid industry professionals and operators of manufacturing systems supported by AR are presented and discussed.publishe

A Reference Architecture for a Workflow Management System Front End Designed for Augmented Reality Headsets

A well-known approach to managing and controlling workflows in organizations is the workflow management system (WFMS). Recently, approaches utilizing augmented reality headsets as WFMS front ends have been discussed, promising higher efficiency, effectiveness, and usability for certain application scenarios. However, existing design-oriented approaches lack actionable guidance for implementation. A well-known approach to address such knowledge gaps is a reference architecture, which inter alia reduces development times and risks and facilitates collaboration between developers. Based on an existing tentative design theory for an augmented reality-based WFMS front end, we contribute a reference architecture containing an extended design theory, user interface design, and models for use cases, components, classes, and sequence flows in the unified modeling language. The reference architecture was successfully operationalized in a prototype and positively evaluated via a survey of potential users

Model-based training of manual procedures in automated production systems

Maintenance engineers deal with increasingly complex automated production systems (aPSs). Such systems are characterized by an increasing computerization or the addition of robots that collaborate with human workers. The effects of changing or replacing components of such systems are difficult to assess since there are complex interdependencies between process parameters and the state of the components. This paper proposes a model-based training system that visualizes these interdependencies using domain-independent SysML models. The training system consists of a virtual training system for initial training and an online support system for assistance during maintenance or changeover procedures. Both systems use structural SysML models to visualize the state of the machine at a certain step of a procedure. An evaluation of the system in a changeover procedure against a paper-based manual showed promising results regarding effectiveness, usability and attractiveness.Comment: 25 pages, https://www.sciencedirect.com/science/article/pii/S095741581830080

Adopting augmented reality in the age of industrial digitalisation

Industrial augmented reality (IAR) is one of the key pillars of the industrial digitalisation concepts, which connects workers with the physical world through overlaying digital information. Augmented reality (AR) market is increasing but still its adoption levels are low in industry. While companies strive to learn and adopt AR, there are chances that they fail in such endeavours due to lack of understanding key challenges and success factors in this space. This study identifies critical success factors and challenges for IAR implementation projects based on field experiments. The broadly used technology, organisation, environment (TOE) framework was used as a theoretical basis for the study, while 22 experiments were conducted for validation. It is found that, while technological aspects are of importance, organisational issues are more relevant for industry, which has not been reflected to the same extent in the literature.No funding source. 22 experiments were conducted with in-kind support (employee time and company access) from Beckhoff Automation, Herman Miller and fluiconnecto as well as University of Cambridge students (see Table 1)

Smart operators: How augmented and virtual technologies are affecting the worker's performance in manufacturing contexts

Purpose: The correct interaction between the workforce and augmented, virtual, and mixed reality technologies represents a crucial aspect of the success of the smart factory. This interaction is, indeed, affected by the variability of human behavior and its reliability, which can strongly influence the quality, safety, and productivity standards. For this reason, this paper aims to provide a clear and complete analysis of the impacts of these technologies on the performance of operators. Design/methodology/approach: A Systematic Literature Review (SLR) was conducted to identify peer-reviewed papers that focused on the implementation of augmented and virtual technologies in manufacturing systems and their effects on human performance. Findings: In total, 61 papers were selected and thoroughly analyzed. The findings of this study reveal that Augmented, Virtual and Mixed Reality can be applied for several applications in manufacturing systems with different types of devices, that involve various advantages and disadvantages. The worker’s performance that are influencing by the use of these technologies are above all time to complete a task, error rate and mental and physical workload. Originality/value: Over the years Augmented, Virtual and Mixed Reality technologies in manufacturing systems have been investigated by researchers. Several studies mostly focused on technological issues, have been conducted. The role of the operator, whose tasks may be influenced positively or negatively by the use of new devices, has been hardly ever analyzed and a deep analysis of human performance affected by these technologies is missing. This study represents a preliminary analysis to fill this gap. The results obtained from the SLR allowed us to develop a conceptual framework that investigates the current state-of-the-art knowledge about the topic and highlights gaps in the current researchPeer Reviewe