Haptic-Guided Teleoperation of a 7-DoF Collaborative Robot Arm With an Identical Twin Master

In this article, we describe two techniques to enable haptic-guided teleoperation using 7-DoF cobot arms as master and slave devices. A shortcoming of using cobots as master-slave systems is the lack of force feedback at the master side. However, recent developments in cobot technologies have brought in affordable, flexible, and safe torque-controlled robot arms, which can be programmed to generate force feedback to mimic the operation of a haptic device. In this article, we use two Franka Emika Panda robot arms as a twin master-slave system to enable haptic-guided teleoperation. We propose a two layer mechanism to implement force feedback due to 1) object interactions in the slave workspace, and 2) virtual forces, e.g. those that can repel from static obstacles in the remote environment or provide task-related guidance forces. We present two different approaches for force rendering and conduct an experimental study to evaluate the performance and usability of these approaches in comparison to teleoperation without haptic guidance. Our results indicate that the proposed joint torque coupling method for rendering task forces improves energy requirements during haptic guided telemanipulation, providing realistic force feedback by accurately matching the slave torque readings at the master side

Gamification of assembly planning in virtual environment

Purpose: The purpose of this paper is to study the effect of the gamification of virtual assembly planning on the user performance, user experience and engagement. / Design/methodology/approach: A multi-touch table was used to manipulate virtual parts and gamification features were integrated into the virtual assembly environment. An experiment was conducted in two conditions: a gamified and a non-gamified virtual environment. Subjects had to assemble a virtual pump. The user performance was evaluated in terms of the number of errors, the feasibility of the generated assembly sequence and the user feedback. / Findings: The gamification reduced the number of errors and increased the score representing the number of right decisions. The results of the subjective and objective analysis showed that the number of errors decreased with engagement in the gamified assembly. The increase in the overall user experience reduced the number of errors. The subjective evaluation showed a significant difference between the gamified and the non-gamified assembly in terms of the level of engagement, the learning usability and the overall experience. / Research limitations/implications: The effective learning retention after training has not been tested, and longitudinal studies are necessary. The effect of the used gamification elements has been evaluated as a whole; further work could isolate the most beneficial features and add other elements that might be more beneficial for learning. / Originality/value: The research reported in this paper provides valuable insights into the gamification of virtual assembly using a low-cost multi-touch interface. The results are promising for training operators to assemble a product at the design stage

Disassembly Sequence Evaluation: A User Study Leveraging Immersive Computing Technologies

As interest in product recovery, reuse, and recycling rises, planning and evaluating disassembly sequences are becoming increasingly important. The manner in which a product can be taken apart strongly influences end-of-life (EOL) operations and costs. Early disassembly planning can also inform non-EOL processes including repair and routine maintenance. Recently, research has concentrated on creating optimization algorithms which automatically generate disassembly sequences. These algorithms often require data that are unavailable or estimated with high uncertainty. Furthermore, industries often employ CAD modeling software to evaluate disassembly sequences during the design stage. The combination of these methods result in mathematically generated solutions, however, the solutions may not account for attributes that are difficult to quantify (human interaction). To help designers better explore and understand disassembly sequence opportunities, the research presented in this paper combines the value of mathematical modeling with the benefits of immersive computing technologies (ICT) to aid in early design decision making. For the purposes of this research, an ICT application was developed. The application displays both 3D geometry of a product and an interactive graph visualization of existing disassembly sequences. The user can naturally interact with the geometric models and explore sequences outlined in the graph visualization. The calculated optimal path can be highlighted allowing the user to quickly compare the optimal sequence against alternatives. The application has been implemented in a three wall immersive projection environment. A user study involving a hydraulic pump assembly was conducted. The results suggest that this approach may be a viable method of evaluating disassembly sequences early in design

Automated freeform assembly of threaded fasteners

Over the past two decades, a major part of the manufacturing and assembly market has been driven by its customer requirements. Increasing customer demand for personalised products create the demand for smaller batch sizes, shorter production times, lower costs, and the flexibility to produce families of products - or different parts - with the same sets of equipment. Consequently, manufacturing companies have deployed various automation systems and production strategies to improve their resource efficiency and move towards right-first-time production. However, many of these automated systems, which are involved with robot-based, repeatable assembly automation, require component- specific fixtures for accurate positioning and extensive robot programming, to achieve flexibility in their production. Threaded fastening operations are widely used in assembly. In high-volume production, the fastening processes are commonly automated using jigs, fixtures, and semi-automated tools. This form of automation delivers reliable assembly results at the expense of flexibility and requires component variability to be adequately controlled. On the other hand, in low- volume, high- value manufacturing, fastening processes are typically carried out manually by skilled workers. This research is aimed at addressing the aforementioned issues by developing a freeform automated threaded fastener assembly system that uses 3D visual guidance. The proof-of-concept system developed focuses on picking up fasteners from clutter, identifying a hole feature in an imprecisely positioned target component and carry out torque-controlled fastening. This approach has achieved flexibility and adaptability without the use of dedicated fixtures and robot programming. This research also investigates and evaluates different 3D imaging technology to identify the suitable technology required for fastener assembly in a non-structured industrial environment. The proposed solution utilises the commercially available technologies to enhance the precision and speed of identification of components for assembly processes, thereby improving and validating the possibility of reliably implementing this solution for industrial applications. As a part of this research, a number of novel algorithms are developed to robustly identify assembly components located in a random environment by enhancing the existing methods and technologies within the domain of the fastening processes. A bolt identification algorithm was developed to identify bolts located in a random clutter by enhancing the existing surface-based matching algorithm. A novel hole feature identification algorithm was developed to detect threaded holes and identify its size and location in 3D. The developed bolt and feature identification algorithms are robust and has sub-millimetre accuracy required to perform successful fastener assembly in industrial conditions. In addition, the processing time required for these identification algorithms - to identify and localise bolts and hole features - is less than a second, thereby increasing the speed of fastener assembly

Parallel Guiding Virtual Fixtures: Control and Stability

International audienceGuiding virtual fixtures have been proposed as a method for human-robot co-manipulation. They constrain the motion of the robot to task-relevant trajectories, which enables the human to execute the task more efficiently, accurately and/or ergonomically. When sequences of different tasks must be solved, multiple guiding virtual fixtures are required, and the appropriate guide for the current task must be detected automatically. To this end, we propose a novel control scheme for multiple guiding virtual fixtures that are active in parallel. Furthermore, we determine under which conditions using multiple fixtures is stable. Finally, we perform a pilot study for a real-world application with a humanoid robot

Virtual Reality Based Environment for Orthopedic Surgery (Veos)

The traditional way of teaching surgery involves students observing a �live� surgery and then gradually assisting experienced surgeons. The creation of a Virtual Reality environment for orthopedic surgery (VEOS) can be beneficial in improving the quality of training while decreasing the time needed for training. Developing such virtual environments for educational and training purposes can supplement existing approaches. In this research, the design and development of a virtual reality based environment for orthopedic surgery is described. The scope of the simulation environment is restricted to an orthopedic surgery process known as Less Invasive Stabilization System (LISS) surgery. The primary knowledge source for the LISS surgical process was Miguel A. Pirela-Cruz (Head of Orthopedic Surgery and Rehabilitation, Texas Tech University Health Sciences Center (TTHSC)). The VEOS was designed and developed on a PC based platform. The developed VEOS was validated through interactions with surgical residents at TTHSC. Feedback from residents and our collaborator Miguel A. Pirela-Cruz was used to make necessary modifications to the surgical environment.Industrial Engineering & Managemen

Haptic technology for micro-robotic cell injection training systems — a review

Currently, the micro-robotic cell injection procedure is performed manually by expert human bio-operators. In order to be proficient at the task, lengthy and expensive dedicated training is required. As such, effective specialized training systems for this procedure can prove highly beneficial. This paper presents a comprehensive review of haptic technology relevant to cell injection training and discusses the feasibility of developing such training systems, providing researchers with an inclusive resource enabling the application of the presented approaches, or extension and advancement of the work. A brief explanation of cell injection and the challenges associated with the procedure are first presented. Important skills, such as accuracy, trajectory, speed and applied force, which need to be mastered by the bio-operator in order to achieve successful injection, are then discussed. Then an overview of various types of haptic feedback, devices and approaches is presented. This is followed by discussion on the approaches to cell modeling. Discussion of the application of haptics to skills training across various fields and haptically-enabled virtual training systems evaluation are then presented. Finally, given the findings of the review, this paper concludes that a haptically-enabled virtual cell injection training system is feasible and recommendations are made to developers of such systems

Digitalization of the product development process at Scania engine assembly

The technology is constantly developing and companies are striving to work towards a more digital approach. Scania CV AB is a world leading Company manufacturing buses and trucks for heavy transport applications. To maintain their competitive position at the market the company has the ambition for the Product development process to become more digitalized. A goal is to implement a more simulation based and drawing free working method. This project has been carried out at the engine assembly department. The purpose with the thesis was to identify how parts of the product development process could be more digitalized. This included identifying the gap that will occur between the current working process and a more digital approach. Furthermore, it involved finding solutions for the gap and to present possible impacts of a digital working approach. The initial phase of the thesis was to find a suitable methodology for this type of study. The project proceeded with conducting a literature study to gain deeper insight of the subjects covered. A good foundation was obtained and the empirical study could commence. The data collection in the empirical study was gathered mainly within Scania through interviews, observations and archive analyses. Based on this information an analysis and result was carried out and presented. A gap was identified describing deficient areas in the current digital environment. The working method Model Based Definition (MBD) and a software called Industrial Path Solutions (IPS) are presented as solutions for the gap. Suggestions of how the working process should be modified have been set as prerequisites. Impacts including cost savings, quality improvements, shorter lead times and ergonomic benefits have been submitted.Tekniken utvecklas ständigt och företag strävar därför att arbeta mot ett mer digitalt arbetssätt. Scania CV AB är ett världsledande företag som tillverkar bussar och lastbilar för tunga transporter. För att behålla sin konkurrenskraftiga position på marknaden har företaget ambitionen att göra produktutvecklingsprocessen mer digitaliserad. Ett mål är att utveckla en mer simuleringsbaserad och ritningslös arbetsmetod. Detta projekt har genomförts på produktionsavdelningen där montering av motorer sker. Syftet med uppsatsen var att identifiera hur delar av den nuvarande produktutvecklingsprocessen skulle kunna bli mer digitaliserad. Detta innebar att identifiera det gap som kommer att uppstå mellan den nuvarande arbetsprocessen och ett mer digitaliserat tillvägagångssätt. Lösningar på gapet och effekterna av ett mer digitalt arbete skulle också presenteras. Den inledande delen av arbetet innefattade att hitta en lämplig metod för denna typ av studie. Projektet fortskred med en litteraturstudie för att få djupare inblick i de ämnen som projektet kommer att grundas i. Med en bra grundförståelse kunde en empirisk studie påbörjas. Datainsamlingen till den empiriska studien samlades huvudsakligen in på Scania genom intervjuer, observationer och arkivanalyser. Baserat på denna information genomfördes och presenterades en analys och ett resultat. Ett gap som beskriver de bristfälliga områden i den nuvarande digitala miljön identifierades. Arbetsmetoden Model Based Definition (MBD) och mjukvaran Industrial Path Solutions (IPS) presenterades som lösningar på gapet. Även förslag på hur arbetsprocessen kan ändras för att möjliggöra för ett mer digitalt tillvägagångssätt har redogjorts. Följderna av detta som inkluderar kostnadsbesparingar, kvalitetsförbättringar, kortare ledtider och ergonomifördelar har också sammanställts

Multi-Phase Multi-Modal Haptic Teleoperation

Virtual Fixtures facilitate teleoperation, for instance by guiding the human operator. Developing these Virtual Fixtures in tasks with tight tolerances remains challenging. Fixtures with a high stiffness allow for more precise guidance, whereas a lower stiffness is required to allow for corrections. We observed that many assembly operations can be split into different phases - approaching, positioning, in-contact manipulation - each with different accuracy requirements. Therefore, we propose to use multi-modal fixtures, satisfying the different requirements of these phases: i.e. a position-based Trajectory Fixture for approaching and a more accurate Visual Servoing Fixture for the positioning phase. A state estimation and arbitration component ensures smooth transitions between the fixtures to provide optimal support for the operator and to achieve global availability paired with local precision at the same time. It also allows a high stiffness to be used throughout, thus achieving good guidance for all phases. The approach is validated in an application from a space scenario, consisting of the assembly of a CubeSat subsystem. The empirical results from a pilot study on this task show that our approach is faster and requires less interaction force from the operator than the baseline method