Opportunities for using eye tracking technology in manufacturing and logistics: Systematic literature review and research agenda

Workers play essential roles in manufacturing and logistics. Releasing workers from routine tasks and enabling them to focus on creative, value-adding activities can enhance their performance and wellbeing, and it is also key to the successful implementation of Industry 4.0. One technology that can help identify patterns of worker-system interaction is Eye Tracking (ET), which is a non-intrusive technology for measuring human eye movements. ET can provide moment-by-moment insights into the cognitive state of the subject during task execution, which can improve our understanding of how humans behave and make decisions within complex systems. It also enables explorations of the subject’s interaction mode with the working environment. Earlier research has investigated the use of ET in manufacturing and logistics, but the literature is fragmented and has not yet been discussed in a literature review yet. This article therefore conducts a systematic literature review to explore the applications of ET, summarise its benefits, and outline future research opportunities of using ET in manufacturing and logistics. We first propose a conceptual framework to guide our study and then conduct a systematic literature search in scholarly databases, obtaining 71 relevant papers. Building on the proposed framework, we systematically review the use of ET and categorize the identified papers according to their application in manufacturing (product development, production, quality inspection) and logistics. Our results reveal that ET has several use cases in the manufacturing sector, but that its application in logistics has not been studied extensively so far. We summarize the benefits of using ET in terms of process performance, human performance, and work environment and safety, and also discuss the methodological characteristics of the ET literature as well as typical ET measures used. We conclude by illustrating future avenues for ET research in manufacturing and logistics

Enhancing depth cues with AR visualization for forklift operation assistance in warehouse.

With warehouse operations contributing to the major part of logistics, architects tend to utilize every inch of the space allocated to maximize the stacking space. Increasing the height of the aisles and narrowing down the aisle-aisle space are major design issues in doing so. Even though forklift manufacturing companies introduced high reach trucks and forklifts for narrow aisles, forklift operators face many issues while working with heavy pallets. This thesis focused on developing a systemthat uses Augmented Reality(AR) to aid forklift operators in performing their pallet racking and pick up tasks. It used AR technology to superimpose virtual cues over the real world specifying the pallets to be picked up and moved and also assist in operating the forklift using depth cues. This aims to increase the productivity of the forklift operators in the warehouse. Depth cues are overlaid on a live video feed from a camera attached to the front of the forklift which was displayed using a laptop to the participants. To evaluate the usability of the system designed, an experiment was conducted and the performance results and the feedback from the participants was evaluated. A remote controlled toy forklift was used to conduct the experiment and a motion tracking system was set-up to track the cab and pallet. Simple pallet handling tasks were designed for the participants and their performance and feedback was collected and analysed. This thesis shows how AR offers a simple and effecient solution for the problems faced by forklift operators while performing pallet handling tasks in warehouse

Project Blue Ocean report

A project was undertaken for Synlait Milk Limited in partial fulfilment of the Master in Engineering Management degree at the University of Canterbury. Project Blue Ocean aimed to discover opportunities for new technology adoption to improve the business operation. The project was initiated to drive the Manufacturing Excellence framework which contains three strong pillars: Safety, Reliability and People. The project began with the discovery of the current issues, mainly focused on manual handling (critical risk activities), repetitive and low-value tasks. The technology solutions were generated respectively to each issue and a high level concept study was developed for each of the top three technology solutions. Design Thinking methodology was applied throughout the project to understand the problems, define the underlying issues, generate unconstrained technology ideas, and prototype the most feasible solution. Justification methods such as the NTCP Diamond Model, the Total Application Model and the Technology Category Model were combined to create an evaluation matrix to find out the top three technology solutions: Vacuum System at Fluid Bed, Collaborative Robots and Fob Key Integration. Preliminary economic evaluation and recommendation plans were made, based on a high level concept study of each solution

How to Communicate Robot Motion Intent: A Scoping Review

Robots are becoming increasingly omnipresent in our daily lives, supporting us and carrying out autonomous tasks. In Human-Robot Interaction, human actors benefit from understanding the robot's motion intent to avoid task failures and foster collaboration. Finding effective ways to communicate this intent to users has recently received increased research interest. However, no common language has been established to systematize robot motion intent. This work presents a scoping review aimed at unifying existing knowledge. Based on our analysis, we present an intent communication model that depicts the relationship between robot and human through different intent dimensions (intent type, intent information, intent location). We discuss these different intent dimensions and their interrelationships with different kinds of robots and human roles. Throughout our analysis, we classify the existing research literature along our intent communication model, allowing us to identify key patterns and possible directions for future research.Comment: Interactive Data Visualization of the Paper Corpus: https://rmi.robot-research.d

Exploring Natural User Abstractions For Shared Perceptual Manipulator Task Modeling & Recovery

State-of-the-art domestic robot assistants are essentially autonomous mobile manipulators capable of exerting human-scale precision grasps. To maximize utility and economy, non-technical end-users would need to be nearly as efficient as trained roboticists in control and collaboration of manipulation task behaviors. However, it remains a significant challenge given that many WIMP-style tools require superficial proficiency in robotics, 3D graphics, and computer science for rapid task modeling and recovery. But research on robot-centric collaboration has garnered momentum in recent years; robots are now planning in partially observable environments that maintain geometries and semantic maps, presenting opportunities for non-experts to cooperatively control task behavior with autonomous-planning agents exploiting the knowledge. However, as autonomous systems are not immune to errors under perceptual difficulty, a human-in-the-loop is needed to bias autonomous-planning towards recovery conditions that resume the task and avoid similar errors. In this work, we explore interactive techniques allowing non-technical users to model task behaviors and perceive cooperatively with a service robot under robot-centric collaboration. We evaluate stylus and touch modalities that users can intuitively and effectively convey natural abstractions of high-level tasks, semantic revisions, and geometries about the world. Experiments are conducted with \u27pick-and-place\u27 tasks in an ideal \u27Blocks World\u27 environment using a Kinova JACO six degree-of-freedom manipulator. Possibilities for the architecture and interface are demonstrated with the following features; (1) Semantic \u27Object\u27 and \u27Location\u27 grounding that describe function and ambiguous geometries (2) Task specification with an unordered list of goal predicates, and (3) Guiding task recovery with implied scene geometries and trajectory via symmetry cues and configuration space abstraction. Empirical results from four user studies show our interface was much preferred than the control condition, demonstrating high learnability and ease-of-use that enable our non-technical participants to model complex tasks, provide effective recovery assistance, and teleoperative control

Simulation-Based Countermeasures Towards Accident Prevention : Virtual Reality Utilization in Industrial Processes and Activities

Despite growing industrial interests in fully immersive virtual reality (VR) applications for safety countermeasures, there is scanty research on the subject in the context of accident prevention during manufacturing processes and plant maintenance activities. This dissertation aims to explore and experiment with VR for accident prevention by targeting three workplace safety countermeasures: fire evacuation drills, hazard identification and risk assessments (HIRA), and emergency preparedness and response (EPR) procedures. Drawing on the virtual reality accident causation model (VR-ACM) (i.e., 3D modelling and simulation, accident causation, and safety drills) and the fire evacuation training model, two industrial 3D simulation models were utilized for the immersive assessment and training. These were a lithium-ion battery (LIB) manufacturing factory and a gas power plant (GPP). In total, five studies (publications) were designed to demonstrate the potential of VR in accident prevention during the manufacturing processes and maintenance activities at the facility conceptual stages. Two studies were with the LIB factory simulation to identify inherent hazards and assess risks for redesigning the factory to ensure workplace safety compliance. The other three studies constituted fire hazard identifications, emergency evacuations and hazard control/mitigations during the maintenance activity in the GPP simulation. Both study models incorporated several participants individually immersed in the virtual realm to experience the accident phenomena intuitively. These participants provided feedback for assessing the research objectives. Results of the studies indicated that several inherent hazards in the LIB factory were identified and controlled/mitigated. Secondly, the GPP experiment results suggested that although the maintenance activity in the virtual realm increased the perception of presence, a statistically significant delay was recorded at the pre-movement stage due to the lack of situational safety awareness. Overall, the study demonstrates that participants immersed in a VR plant maintenance activity and manufacturing factory process simulation environments can experience real-time emergency scenarios and conditions necessary for implementing the essential safety countermeasures to prevent accidents.Vaikka kiinnostus virtuaalitodellisuuden (VR) käyttöön turvallisuuden varotoimissa teollisuudessa on kasvanut, tutkimuksia ei ole juurikaan tehty onnettomuuksien ehkäisystä valmistus- ja kunnossapitotoiminnassa. Tämän väitöskirjan tavoitteena on tutkia ja kokeilla VR:ää tapaturmien ehkäisyssä kohdistuen kolmeen työpaikan turvallisuuden varotoimeen: paloharjoitukset, riskien arvioinnit sekä hätätilanteiden valmiusmenettelyt ja toimintasuunnitelmat (EPR). Kokemuksellisessa ja uppouttavassa koulutuksessa hyödynnettiin kahta teollisuuden 3D-simulointimallia, jotka nojautuvat virtuaalitodellisuuden onnettomuuksien aiheutumismalliin (VR-ACM) (eli 3D-mallinnus- ja simulointi, onnettomuussyy- ja turvallisuuskoulutus) sekä paloharjoitusmalliin. Nämä 3D-simulointimallit ovat litiuminoniakkuja (LIB) valmistava tehdas, joka rakennettiin Visual Components 3D-simulointiohjelmistolla (versio 4.0) ja kaasuvoimala (GPP) Unrealin reaaliaikaisella pelimoottorilla (versio 4.2). Yhteensä viisi tutkimusta (julkaisua) suunniteltiin havainnollistamaan VR:n potentiaalia tapaturmien ehkäisyssä valmistusprosessin layout-suunnittelun ja tehtaan konseptivaiheissa tehtävän kunnossapidon aikana. Kaksi tutkimusta tehtiin LIB-tehdassimulaatiolla vaarojen tunnistamiseksi sekä riskien arvioimiseksi. Tutkimukset tehtiin tehtaan uudelleensuunnittelua varten, työturvallisuuden noudattamisen varmistamiseksi. Muut kolme tutkimusta käsittelevät palovaaran tunnistamista, hätäevakuointia ja riskien vähentämistä huoltotoiminnan aikana GPP-simulaatiossa. Molemmissa tutkimusmalleissa oli useita virtuaalimaailmaan uppoutuneita osallistujia, jotka saivat kokea onnettomuudet yksilöllisesti ja intuitiivisesti. Osallistujat antoivat palautetta kokeen jälkeisessä kyselyssä. Kyselyn tuloksien avulla LIB-tehtaassa tunnistettiin ja lievennettiin useita vaaroja. GPP-kokeilun tulokset viittasivat siihen, että vaikka ylläpitotoiminta virtuaalimaailmassa lisäsi teleläsnäoloa, tilastollisesti merkittävä viive kirjattiin liikettä edeltävässä vaiheessa turvallisuustietoisuuden puuteen vuoksi. Kaiken kaikkiaan tutkimus osoittaa, että VR-laitoksen kunnossapitotoimintaan ja tuotantotehtaan prosessisimulaatioympäristöihin uppoutuvat osallistujat voivat kokea reaaliaikaisia hätäskenaarioita ja olosuhteita, jotka ovat välttämättömiä olennaisten turvallisuustoimien toteuttamiseksi.fi=vertaisarvioitu|en=peerReviewed

Smart working technologies in industry 4.0 : contributions to different manufacturing activities and workers’ skills

A Indústria 4.0 é considerada a quarta revolução industrial porque utiliza uma ampla integração de tecnologias de informação e de operação na fabricação industrial. Apesar dessa perspectiva tecnológica, diversos estudos vêm evidenciando a importância de considerar o fator humano para o desenvolvimento de um sistema de manufatura inteligente. Nesse sentido, a dimensão denominada como Smart Working precisa ser melhor investigada, uma vez que entender como as tecnologias afetam os trabalhadores e as habilidades desses são cruciais para o bom desempenho das fábricas. Em razão disso, o objetivo desta dissertação foi entender como as Smart Working Technologies (SWT) podem contribuir para as atividades e as habilidades dos trabalhadores da manufatura. Para tanto, primeiramente foi realizada uma análise abrangente da literatura para identificar as SWT e seus impactos nas capacidades dos trabalhadores em suas atividades de manufatura. Deste modo, foram analisados 80 artigos que relacionam as SWT em oito atividades de manufatura. Posteriormente, foi selecionada uma das SWT mais relevantes conforme a literatura, os robôs colaborativos, para identificar os efeitos das tecnologias nas habilidades dos trabalhadores. Deste modo, foram analisados 138 casos de aplicação reportados por uma das empresas fornecedoras líderes mundiais, bem como três entrevistas com empresas adotantes da tecnologia. Os resultados apontam que existem 15 SWT que podem ser implementadas nas atividades de manufatura e relacionadas às capacidades dos trabalhadores. Além disso, os resultados também apontam que podem existir quatro efeitos das SWT nas habilidades dos trabalhadores. Estes achados demonstram que de acordo com a estratégia da empresa uma SWT pode impactar de diferentes formas os trabalhadores.Industry 4.0 is considered the fourth industrial revolution because it uses a broad integration of information and operating technologies in industrial manufacturing. Despite this technological perspective, several studies have highlighted the importance of considering the human factor to develop a smart manufacturing system. In this sense, the Smart Working dimension needs to be further investigated since understanding how technologies affect workers and their skills are crucial for factories' good performance. Therefore, the objective of this dissertation was to understand how Smart Working Technologies (SWT) can contribute to the activities and skills of manufacturing workers. To this end, firstly a systematic literature review was carried out to identify SWTs and their impacts on workers' capabilities in their manufacturing activities. Thus, 80 articles relating to SWT in eight manufacturing activities were analyzed. Subsequently, one of the most relevant SWTs according to the literature, collaborative robots, was selected to identify the effects of technologies on workers' skills. In this way, 138 application cases reported by one of the world's leading supplier companies were analyzed, as well as three interviews with companies that adopted the technology. The results show that there are 15 SWT that can be implemented in manufacturing activities and related to workers' capabilities. In addition, the results also point out that there may be four effects of SWT on workers' skills. According to the company's strategy, these findings demonstrate that an SWT can impact workers in different ways