A model-based residual approach for human-robot collaboration during manual polishing operations

A fully robotized polishing of metallic surfaces may be insufficient in case of parts with complex geometric shapes, where a manual intervention is still preferable. Within the EU SYMPLEXITY project, we are considering tasks where manual polishing operations are performed in strict physical Human-Robot Collaboration (HRC) between a robot holding the part and a human operator equipped with an abrasive tool. During the polishing task, the robot should firmly keep the workpiece in a prescribed sequence of poses, by monitoring and resisting to the external forces applied by the operator. However, the user may also wish to change the orientation of the part mounted on the robot, simply by pushing or pulling the robot body and changing thus its configuration. We propose a control algorithm that is able to distinguish the external torques acting at the robot joints in two components, one due to the polishing forces being applied at the end-effector level, the other due to the intentional physical interaction engaged by the human. The latter component is used to reconfigure the manipulator arm and, accordingly, its end-effector orientation. The workpiece position is kept instead fixed, by exploiting the intrinsic redundancy of this subtask. The controller uses a F/T sensor mounted at the robot wrist, together with our recently developed model-based technique (the residual method) that is able to estimate online the joint torques due to contact forces/torques applied at any place along the robot structure. In order to obtain a reliable residual, which is necessary to implement the control algorithm, an accurate robot dynamic model (including also friction effects at the joints and drive gains) needs to be identified first. The complete dynamic identification and the proposed control method for the human-robot collaborative polishing task are illustrated on a 6R UR10 lightweight manipulator mounting an ATI 6D sensor

Collision Detection and Contact Point Estimation Using Virtual Joint Torque Sensing Applied to a Cobot

In physical human-robot interaction (pHRI) it is essential to reliably estimate and localize contact forces between the robot and the environment. In this paper, a complete contact detection, isolation, and reaction scheme is presented and tested on a new 6-dof industrial collaborative robot. We combine two popular methods, based on monitoring energy and generalized momentum, to detect and isolate collisions on the whole robot body in a more robust way. The experimental results show the effectiveness of our implementation on the LARA5 cobot, that only relies on motor current and joint encoder measurements. For validation purposes, contact forces are also measured using an external GTE CoboSafe sensor. After a successful collision detection, the contact point location is isolated using a combination of the residual method based on the generalized momentum with a contact particle filter (CPF) scheme. We show for the first time a successful implementation of such combination on a real robot, without relying on joint torque sensor measurements

A Soft Robotic Cover with Dual Thermal Display and Sensing Capabilities

We propose a new robotic cover prototype that achieves thermal display while also being soft. We focus on the thermal cue because previous human studies have identified it as part of the touch pleasantness. The robotic cover surface can be regulated to the desired temperature by circulating water through a thermally conductive pipe embedded in the cover, of which temperature is controlled. Besides, an observer for estimating heat from human contact is implemented; it can detect human interaction while displaying the desired temperature without temperature sensing on the surface directly. We assessed the validity of the prototype in experiments of temperature control and contact detection by human hand

A Sensorized Instrument for Minimally Invasive Surgery for the Measurement of Forces during Training and Surgery: Development and Applications

The reduced access conditions present in Minimally Invasive Surgery (MIS) affect the feel of interaction forces between the instruments and the tissue being treated. This loss of haptic information compromises the safety of the procedure and must be overcome through training. Haptics in MIS is the subject of extensive research, focused on establishing force feedback mechanisms and developing appropriate sensors. This latter task is complicated by the need to place the sensors as close as possible to the instrument tip, as the measurement of forces outside of the patient\u27s body does not represent the true tool--tissue interaction. Many force sensors have been proposed, but none are yet available for surgery. The objectives of this thesis were to develop a set of instruments capable of measuring tool--tissue force information in MIS, and to evaluate the usefulness of force information during surgery and for training and skills assessment. To address these objectives, a set of laparoscopic instruments was developed that can measure instrument position and tool--tissue interaction forces in multiple degrees of freedom. Different design iterations and the work performed towards the development of a sterilizable instrument are presented. Several experiments were performed using these instruments to establish the usefulness of force information in surgery and training. The results showed that the combination of force and position information can be used in the development of realistic tissue models or haptic interfaces specifically designed for MIS. This information is also valuable in order to create tactile maps to assist in the identification of areas of different stiffness. The real-time measurement of forces allows visual force feedback to be presented to the surgeon. When applied to training scenarios, the results show that experience level correlates better with force-based metrics than those currently used in training simulators. The proposed metrics can be automatically computed, are completely objective, and measure important aspects of performance. The primary contribution of this thesis is the design and development of highly versatile instruments capable of measuring force and position during surgery. A second contribution establishes the importance and usefulness of force data during skills assessment, training and surgery

Body Wall Force Sensor for Simulated Minimally Invasive Surgery: Application to Fetal Surgery

Surgical interventions are increasingly executed minimal invasively. Surgeons insert instruments through tiny incisions in the body and pivot slender instruments to treat organs or tissue below the surface. While a blessing for patients, surgeons need to pay extra attention to overcome the fulcrum effect, reduced haptic feedback and deal with lost hand-eye coordination. The mental load makes it difficult to pay sufficient attention to the forces that are exerted on the body wall. In delicate procedures such as fetal surgery, this might be problematic as irreparable damage could cause premature delivery. As a first attempt to quantify the interaction forces applied on the patient's body wall, a novel 6 degrees of freedom force sensor was developed for an ex-vivo set up. The performance of the sensor was characterised. User experiments were conducted by 3 clinicians on a set up simulating a fetal surgical intervention. During these simulated interventions, the interaction forces were recorded and analysed when a normal instrument was employed. These results were compared with a session where a flexible instrument under haptic guidance was used. The conducted experiments resulted in interesting insights in the interaction forces and stresses that develop during such difficult surgical intervention. The results also implicated that haptic guidance schemes and the use of flexible instruments rather than rigid ones could have a significant impact on the stresses that occur at the body wall

3D locomotion biomimetic robot fish with haptic feedback

This thesis developed a biomimetic robot fish and built a novel haptic robot fish system based on the kinematic modelling and three-dimentional computational fluid dynamic (CFD) hydrodynamic analysis. The most important contribution is the successful CFD simulation of the robot fish, supporting users in understanding the hydrodynamic properties around it

Otimização muscle-in-the-loop em tempo real para reabilitação física com um exosqueleto ativo: uma mudança de paradigma

Assisting human locomotion with a wearable robotic orthosis is still quite challenging, largely due to the complexity of the neuromusculoskeletal system, the time-varying dynamics that accompany motor adaptation, and the uniqueness of every individual’s response to the assistance given by the robot. To this day, these devices have not met their well-known promise yet, mostly due to the fact that they are not perfectly suitable for the rehabilitation of neuropathologic patients. One of the main challenges hampering this goal still relies on the interface and co-dependency between the human and the machine. Nowadays, most commercial exoskeletons replay pre-defined gait patterns, whereas research exoskeletons are switching to controllers based on optimized torque profiles. In most cases, the dynamics of the human musculoskeletal system are still ignored and do not take into account the optimal conditions for inducing a positive modulation of neuromuscular activity. This is because both rehabilitation strategies are still emphasized on the macro level of the whole joint instead of focusing on the muscles’ dynamics and activity, which are the actual anatomical elements that may need to be rehabilitated. Strategies to keep the human in the loop of the exoskeleton’s control laws in real-time may help to overcome these challenges. The main purpose of the present dissertation is to make a paradigm shift in the approach on how the assistance that is given to a subject by an exoskeleton is modelled and controlled during physical rehabilitation. Therefore, in the scope of the present work, it was intended to design, concede, implement, and validate a real-time muscle-in-the-loop optimization model to find the best assistive support ratio that would induce optimal rehabilitation conditions to a specific group of impaired muscles while having a minimum impact on the other healthy muscles. The developed optimization model was implemented in the form of a plugin and was integrated on a neuromechanical model-based interface for driving a bilateral ankle exoskeleton. Experimental pilot tests evaluated the feasibility and effectiveness of the model. Results of the most significant pilots achieved EMG reductions up to 61 ± 3 % in Soleus and 41 ± 10 % in Gastrocnemius Lateralis. Moreover, results also demonstrated the efficiency of the optimization’s specific reduction on rehabilitation by looking into the muscular fatigue after each experiment. Finally, two parallel preliminary studies emerged from the pilots, which looked at muscle adaptation, after a new assistive condition had been applied, over time and at the effect of the lateral positioning of the exoskeleton’s actuators on the leg muscles.Auxiliar a locomoção humana com uma ortose robótica ainda é bastante desafiante, em grande parte devido à complexidade do sistema neuromusculoesquelético, à dinâmica variável no tempo que acompanha a adaptação motora e à singularidade da resposta de cada indivíduo à assistência dada pelo robô. Até hoje, está por cumprir a promessa inicial destes dispositivos, principalmente devido ao facto de não serem perfeitamente adequados para a reabilitação de pacientes neuropatológicos. Um dos principais desafios que dificultam esse objetivo foca-se ainda na interface e na co-dependência entre o ser humano e a máquina. Hoje em dia, a maioria dos exoesqueletos comerciais reproduz padrões de marcha predefinidos, enquanto que os exoesqueletos em investigação estão só agora a mudar para controladores com base em perfis de binário otimizados. Na maioria dos casos, a dinâmica do sistema musculoesquelético humano ainda é ignorada e não tem em consideração as condições ideais para induzir uma modulação positiva da atividade neuromuscular. Isso ocorre porque ambas as estratégias de reabilitação ainda são enfatizadas no nível macro de toda a articulação, em vez de se concentrar na dinâmica e atividade dos músculos, que são os elementos anatómicos que realmente precisam de ser reabilitados. Estratégias para manter o ser humano em loop nos comandos que controlam o exoesqueleto em tempo real podem ajudar a superar estes desafios. O principal objetivo desta dissertação é fazer uma mudança de paradigma na abordagem em como a assistência que é dada a um sujeito por um exosqueleto é modelada e controlada durante a reabilitação física. Portanto, no contexto do presente trabalho, pretendeu-se projetar, conceder, implementar e validar um modelo de otimização muscle-in-the-loop em tempo real para encontrar a melhor relação de suporte capaz de induzir as condições ideais de reabilitação para um grupo específico de músculos fragilizados, tendo um impacto mínimo nos outros músculos saudáveis. O modelo de otimização desenvolvido foi implementado na forma de um plugin e foi integrado numa interface baseada num modelo neuromecânico para o controlo de um exoesqueleto bilateral de tornozelo. Testes experimentais piloto avaliaram a viabilidade e a eficácia do modelo. Os resultados dos testes mais significativos demonstraram reduções de EMG de até 61 ± 3 % no Soleus e 41 ± 10 % no Gastrocnemius Lateral. Adicionalmente, os resultados demonstraram também a eficiência em reabilitação da redução específica no EMG devido à otimização tendo em conta a fadiga muscular após cada teste. Finalmente, dois estudos preliminares paralelos emergiram dos testes piloto, que analisaram a adaptação muscular após uma nova condição assistiva ter sido definida ao longo do tempo e o efeito do posicionamento lateral dos atuadores do exoesqueleto nos músculos da perna.Mestrado em Engenharia Biomédic