3D Visual Data-Driven Spatiotemporal Deformations for Non-Rigid Object Grasping Using Robot Hands

Sensing techniques are important for solving problems of uncertainty inherent to intelligent grasping tasks. The main goal here is to present a visual sensing system based on range imaging technology for robot manipulation of non-rigid objects. Our proposal provides a suitable visual perception system of complex grasping tasks to support a robot controller when other sensor systems, such as tactile and force, are not able to obtain useful data relevant to the grasping manipulation task. In particular, a new visual approach based on RGBD data was implemented to help a robot controller carry out intelligent manipulation tasks with flexible objects. The proposed method supervises the interaction between the grasped object and the robot hand in order to avoid poor contact between the fingertips and an object when there is neither force nor pressure data. This new approach is also used to measure changes to the shape of an object’s surfaces and so allows us to find deformations caused by inappropriate pressure being applied by the hand’s fingers. Test was carried out for grasping tasks involving several flexible household objects with a multi-fingered robot hand working in real time. Our approach generates pulses from the deformation detection method and sends an event message to the robot controller when surface deformation is detected. In comparison with other methods, the obtained results reveal that our visual pipeline does not use deformations models of objects and materials, as well as the approach works well both planar and 3D household objects in real time. In addition, our method does not depend on the pose of the robot hand because the location of the reference system is computed from a recognition process of a pattern located place at the robot forearm. The presented experiments demonstrate that the proposed method accomplishes a good monitoring of grasping task with several objects and different grasping configurations in indoor environments.The research leading to these result has received funding from the Spanish Government and European FEDER funds (DPI2015-68087R), the Valencia Regional Government (PROMETEO/2013/085) as well as the pre-doctoral grant BES-2013-062864

Design and Development of Assistive Robots for Close Interaction with People with Disabilities

People with mobility and manipulation impairments wish to live and perform tasks as independently as possible; however, for many tasks, compensatory technology does not exist, to do so. Assistive robots have the potential to address this need. This work describes various aspects of the development of three novel assistive robots: the Personal Mobility and Manipulation Appliance (PerMMA), the Robotic Assisted Transfer Device (RATD), and the Mobility Enhancement Robotic Wheelchair (MEBot). PerMMA integrates mobility with advanced bi-manual manipulation to assist people with both upper and lower extremity impairments. The RATD is a wheelchair mounted robotic arm that can lift higher payloads and its primary aim is to assist caregivers of people who cannot independently transfer from their electric powered wheelchair to other surfaces such as a shower bench or toilet. MEBot is a wheeled robot that has highly reconfigurable kinematics, which allow it to negotiate challenging terrain, such as steep ramps, gravel, or stairs. A risk analysis was performed on all three robots which included a Fault Tree Analysis (FTA) and a Failure Mode Effect Analysis (FMEA) to identify potential risks and inform strategies to mitigate them. Identified risks or PerMMA include dropping sharp or hot objects. Critical risks identified for RATD included tip over, crush hazard, and getting stranded mid-transfer, and risks for MEBot include getting stranded on obstacles and tip over. Lastly, several critical factors, such as early involvement of people with disabilities, to guide future assistive robot design are presented

Development of an open access system for remote operation of robotic manipulators

Mestrado de dupla diplomação com a UTFPR - Universidade Tecnológica Federal do ParanáExploring the realms of research, training, and learning in the field of robotic systems poses obstacles for institutions lacking the necessary infrastructure. The significant investment required to acquire physical robotic systems often limits access and hinders progress in these areas. While robotic simulation platforms provide a virtual environment for experimentation, the potential of remote robotic environments surpasses this by enabling users to interact with real robotic systems during training and research activities. This way, users, including students and researchers, can engage in a virtual experience that transcends geographical boundaries, connecting them to real-world robotic systems though the Internet. By bridging the gap between virtual and physical worlds, remote environments offer a more practical and immersive experience, and open up new horizons for collaborative research and training. Democratizing access to these technologies means empower educational institutions and research centers to engage in practical and handson learning experiences. However, the implementation of remote robotic environments comes with its own set of technical challenges: communication, security, stability and access. In light of these challenges, a ROS-based system has been developed, providing open access with promising results (low delay and run-time visualization). This system enables remote control of robotic manipulators and has been successfully validated through the remote operation of a real UR3 manipulator.Explorar as áreas de pesquisa, treinamento e aprendizado no campo de sistemas robóticos apresenta obstáculos para instituições que não possuem a infraestrutura necessária. O investimento significativo exigido para adquirir sistemas robóticos físicos muitas vezes limita o acesso e dificulta o progresso nessas áreas. Embora as plataformas de simulação robótica forneçam um ambiente virtual para experimentação, o potencial dos ambientes robóticos remotos vai além disso, permitindo que os usuários interajam com sistemas robóticos reais durante atividades de treinamento e pesquisa. Dessa forma, os usuários, incluindo estudantes e pesquisadores, podem participar de uma experiência virtual que transcende as fronteiras geográficas, conectando-os a sistemas robóticos do mundo real por meio da Internet. Ao estabelecer uma ponte entre os mundos virtual e físico, os ambientes remotos oferecem uma experiência mais prática e imersiva, abrindo novos horizontes para a pesquisa colaborativa e o treinamento. Democratizar o acesso a essas tecnologias significa capacitar instituições educacionais e centros de pesquisa a se envolverem em experiências práticas e de aprendizado prático. No entanto, a implementação de ambientes robóticos remotos traz consigo um conjunto próprio de desafios técnicos: comunicação, segurança, estabilidade e acesso. Diante desses desafios, foi desenvolvida uma plataforma baseada em ROS, oferecendo acesso aberto com resultados promissores (baixo delay e visualização em run-time). Essa plataforma possibilita o controle remoto de manipuladores robóticos e foi validada com sucesso por meio da operação remota de um manipulador UR3 real

Kinematic modeling of 5-axis horizontal milling machine emulated from vertical articulated robot

Industrijski roboti predstavljaju uspešnu alternativu 5-osnim mašinama alatkama za višeosnu obradu glodanjem delova od mekših materijala niže klase tačnosti. Iz ovih razloga se razvijaju softverska rešenja koja bi približila programiranje robota za obradu programiranju mašina alatki. U radu su predstavljeni rezultati kinematičkog modeliranja industrijskih robota vertikalne zglobne konfiguracije u cilju emuliranja horizontalnih 5-osnih mašina alatki. Na osnovu rešenja inverznog kinematičkog problema analiziran je radni prostor robota za obradu. Polazeći od jednačina direktne i inverzne kinematike razvijen je i sistem upravljanja i programiranja u G-kodu baziran na EMC2 softverskom sistemu. Razvijeni sistem upravljanja i programiranja omogućava upotrebu postojećih CAD/CAM sistema za programiranje 5-osnih mašina alatki, odnosno direktno je primenljiv od strane programera i operatera koji imaju iskustva u CNC tehnologijama i programiranju u G-kodu. Verifikacija kinematičkog modeliranja i razvijenog sistema upravljanja i programiranja je ostvarena eksperimentima obrade na rasploživom 6-oosnom robotu domaće proizvodnje LOLA 50.Industrial robots are a proper alternative to 5-axis machine tools for multi-axis milling of parts from softer materials and lower tolerances. For these reasons, researchers develop software solutions to bring programming of machining robots closer to programming of machine tools. The paper presents the results of kinematic modeling of vertical articulated industrial robots to emulate 5-axis horizontal machine tools. The solution to the inverse kinematics problem was the basis for the analysis of the machining robot workspace. Starting from equations of direct and inverse kinematics the control and programming system in G-code was developed, based on EMC2 software system. Developed control and programming system enables use of the available CAD/CAM systems for programming 5-axis machine tools, i.e. it is directly applicable by programmers and operators experienced in CNC technologies and G-code programming. Verification of kinematic modeling and developed control and programming system was carried out by machining experiments on an available domestic 6-axis robot LOLA 50

EDAN - An EMG-controlled Daily Assistant To Help People With Physical Disabilities

Injuries, accidents, strokes, and other diseases can significantly degrade the capabilities to perform even the most simple activities in daily life. A large share of these cases involves neuromuscular diseases, which lead to severely reduced muscle function. However, even though affected people are no longer able to move their limbs, residual muscle function can still be existent. Previous work has shown that this residual muscular activity can suffice to apply an EMG-based user interface. In this paper, we introduce DLR's robotic wheelchair EDAN (EMG-controlled Daily Assistant), which is equipped with a torque-controlled, eight degree-of-freedom light-weight arm and a dexterous, five-fingered robotic hand. Using electromyography, muscular activity of the user is measured,processed and utilized to control both the wheelchair and the robotic manipulator. This EMG-based interface is enhanced with shared control functionality to allow for efficient and safe physical interaction with the environment

Aspects of an open architecture robot controller and its integration with a stereo vision sensor.

The work presented in this thesis attempts to improve the performance of industrial robot systems in a flexible manufacturing environment by addressing a number of issues related to external sensory feedback and sensor integration, robot kinematic positioning accuracy, and robot dynamic control performance. To provide a powerful control algorithm environment and the support for external sensor integration, a transputer based open architecture robot controller is developed. It features high computational power, user accessibility at various robot control levels and external sensor integration capability. Additionally, an on-line trajectory adaptation scheme is devised and implemented in the open architecture robot controller, enabling a real-time trajectory alteration of robot motion to be achieved in response to external sensory feedback. An in depth discussion is presented on integrating a stereo vision sensor with the robot controller to perform external sensor guided robot operations. Key issues for such a vision based robot system are precise synchronisation between the vision system and the robot controller, and correct target position prediction to counteract the inherent time delay in image processing. These were successfully addressed in a demonstrator system based on a Puma robot. Efforts have also been made to improve the Puma robot kinematic and dynamic performance. A simple, effective, on-line algorithm is developed for solving the inverse kinematics problem of a calibrated industrial robot to improve robot positioning accuracy. On the dynamic control aspect, a robust adaptive robot tracking control algorithm is derived that has an improved performance compared to a conventional PID controller as well as exhibiting relatively modest computational complexity. Experiments have been carried out to validate the open architecture robot controller and demonstrate the performance of the inverse kinematics algorithm, the adaptive servo control algorithm, and the on-line trajectory generation. By integrating the open architecture robot controller with a stereo vision sensor system, robot visual guidance has been achieved with experimental results showing that the integrated system is capable of detecting, tracking and intercepting random objects moving in 3D trajectory at a velocity up to 40mm/s

Study and Development of Mechatronic Devices and Machine Learning Schemes for Industrial Applications

Obiettivo del presente progetto di dottorato è lo studio e sviluppo di sistemi meccatronici e di modelli machine learning per macchine operatrici e celle robotizzate al fine di incrementarne le prestazioni operative e gestionali. Le pressanti esigenze del mercato hanno imposto lavorazioni con livelli di accuratezza sempre più elevati, tempi di risposta e di produzione ridotti e a costi contenuti. In questo contesto nasce il progetto di dottorato, focalizzato su applicazioni di lavorazioni meccaniche (e.g. fresatura), che includono sistemi complessi quali, ad esempio, macchine a 5 assi e, tipicamente, robot industriali, il cui utilizzo varia a seconda dell’impiego. Oltre alle specifiche problematiche delle lavorazioni, si deve anche considerare l’interazione macchina-robot per permettere un’efficiente capacità e gestione dell’intero impianto. La complessità di questo scenario può evidenziare sia specifiche problematiche inerenti alle lavorazioni (e.g. vibrazioni) sia inefficienze più generali che riguardano l’impianto produttivo (e.g. asservimento delle macchine con robot, consumo energetico). Vista la vastità della tematica, il progetto si è suddiviso in due parti, lo studio e sviluppo di due specifici dispositivi meccatronici, basati sull’impiego di attuatori piezoelettrici, che puntano principalmente alla compensazione di vibrazioni indotte dal processo di lavorazione, e l’integrazione di robot per l’asservimento di macchine utensili in celle robotizzate, impiegando modelli di machine learning per definire le traiettorie ed i punti di raggiungibilità del robot, al fine di migliorarne l’accuratezza del posizionamento del pezzo in diverse condizioni. In conclusione, la presente tesi vuole proporre soluzioni meccatroniche e di machine learning per incrementare le prestazioni di macchine e sistemi robotizzati convenzionali. I sistemi studiati possono essere integrati in celle robotizzate, focalizzandosi sia su problematiche specifiche delle lavorazioni in macchine operatrici sia su problematiche a livello di impianto robot-macchina. Le ricerche hanno riguardato un’approfondita valutazione dello stato dell’arte, la definizione dei modelli teorici, la progettazione funzionale e l’identificazione delle criticità del design dei prototipi, la realizzazione delle simulazioni e delle prove sperimentali e l’analisi dei risultati.The aim of this Ph.D. project is the study and development of mechatronic systems and machine learning models for machine tools and robotic applications to improve their performances. The industrial demands have imposed an ever-increasing accuracy and efficiency requirement whilst constraining the cost. In this context, this project focuses on machining processes (e.g. milling) that include complex systems such as 5-axes machine tool and industrial robots, employed for various applications. Beside the issues related to the machining process itself, the interaction between the machining centre and the robot must be considered for the complete industrial plant’s improvement. This scenario´s complexity depicts both specific machining problematics (e.g. vibrations) and more general issues related to the complete plant, such as machine tending with an industrial robot and energy consumption. Regarding the immensity of this area, this project is divided in two parts, the study and development of two mechatronic devices, based on piezoelectric stack actuators, for the active vibration control during the machining process, and the robot machine tending within the robotic cell, employing machine learning schemes for the trajectory definition and robot reachability to improve the corresponding positioning accuracy. In conclusion, this thesis aims to provide a set of solutions, based on mechatronic devices and machine learning schemes, to improve the conventional machining centre and the robotic systems performances. The studied systems can be integrated within a robotic cell, focusing on issues related to the specific machining process and to the interaction between robot-machining centre. This research required a thorough study of the state-of-the-art, the formulation of theoretical models, the functional design development, the identification of the critical aspects in the prototype designs, the simulation and experimental campaigns, and the analysis of the obtained results

Techno-economic assessment of an automated lithium-ion battery module disassembly process: investigating the optimal level of automated disassembly.

The use of lithium-ion batteries (LIBs) has rapidly increased recent years, mainly due to the global adoption of electric vehicles (EVs). Continued growth is expected, which will inevitably lead to a large amount of battery waste. Proper recycling is then required to reinsert valuable raw materials to the value chain. Recycling of LIBs are normally initiated by disassembly, followed by various mechanical and metallurgical treatments. Disassembly is one of the most labour intensive steps when recycling LIBs. Considering the expected growth, a fully automated disassembly process will be required. However, this represents a non-negligible investment. Determining the optimal disassembly level prior to recycling is a crucial step that must be considered before investing in automated disassembly. Currently, most EV LIBs are only disassembled to module level prior to recycling. Instead, disassembly to cell level could produce greater purity material streams and less material to handle downstream. The question is whether or not it is economic viable to carry out deeper robotic disassembly down to cell level instead of stopping at module level. This study presents a techno-economic assessment of a robotic module disassembly line, furnishing guidelines on the necessary degree of automation in EV LIB disassembly. Different case study scenarios are proposed, demonstrating that investments in a robotic module disassembly line could be profitable