Capturing Hands in Action using Discriminative Salient Points and Physics Simulation

Hand motion capture is a popular research field, recently gaining more attention due to the ubiquity of RGB-D sensors. However, even most recent approaches focus on the case of a single isolated hand. In this work, we focus on hands that interact with other hands or objects and present a framework that successfully captures motion in such interaction scenarios for both rigid and articulated objects. Our framework combines a generative model with discriminatively trained salient points to achieve a low tracking error and with collision detection and physics simulation to achieve physically plausible estimates even in case of occlusions and missing visual data. Since all components are unified in a single objective function which is almost everywhere differentiable, it can be optimized with standard optimization techniques. Our approach works for monocular RGB-D sequences as well as setups with multiple synchronized RGB cameras. For a qualitative and quantitative evaluation, we captured 29 sequences with a large variety of interactions and up to 150 degrees of freedom.Comment: Accepted for publication by the International Journal of Computer Vision (IJCV) on 16.02.2016 (submitted on 17.10.14). A combination into a single framework of an ECCV'12 multicamera-RGB and a monocular-RGBD GCPR'14 hand tracking paper with several extensions, additional experiments and detail

Inter-finger Small Object Manipulation with DenseTact Optical Tactile Sensor

The ability to grasp and manipulate small objects in cluttered environments remains a significant challenge. This paper introduces a novel approach that utilizes a tactile sensor-equipped gripper with eight degrees of freedom to overcome these limitations. We employ DenseTact 2.0 for the gripper, enabling precise control and improved grasp success rates, particularly for small objects ranging from 5mm to 25mm. Our integrated strategy incorporates the robot arm, gripper, and sensor to manipulate and orient small objects for subsequent classification effectively. We contribute a specialized dataset designed for classifying these objects based on tactile sensor output and a new control algorithm for in-hand orientation tasks. Our system demonstrates 88% of successful grasp and successfully classified small objects in cluttered scenarios

ANN Control Based on Patterns Recognition for A Robotic Hand Under Different Load Conditions

في هذا البحث, الشبكة العصبية الاصطناعية (ANN) قد تم تدريبها على انماط نسب المركبات العمودية الى الافقية لقوى التماس عند وقت حدوث الانزلاق, لتكون قادرة على تمييز الانزلاق تحت انواع مختلفة من الأحمال (الحمل الستاتيكي والحمل الديناميكي), ومن ثم توليد اشارة راجعة تستخدم كمشغل لمحرك اليد الصناعية. هذه العملية اجريت بدون الحاجة لأي معلومات حول خواص الجسم الممسوك, مثل الوزن, تركيب السطح, الشكل, معامل الاحتكاك و نوع الحمل المؤثر على الجسم الممسوك. لتحقيق ذلك , تم اقتراح تصميم جديد لرأس الاصبع من اجل كشف الانزلاق في اتجاهات متعددة بين الجسم الممسوك ورؤس الاصابع الاصطناعية. هذا التصميم يتألف من اصبعين مع نظام تشغيل يتضمن اجزاء مرنة (نوابض انضغاطية). هذه النوابض تعمل كمعوض لقوة المسك عند وقت حدوث الانزلاق حتى في وضعية التوقف لمحرك اليد. نسب مركبات قوى التماس يمكن حسابها بواسطة حساسات قوى تقليدية (FlexiForce sensor) بعد معالجة بيانات القوى باستخدام برنامج Matlab/Simulink ومن خلال علاقات رياضية معينة التي تم اشتقاقها لوصف الآلية الميكانيكية للإصبع الاصطناعي.In this paper, the Artificial Neural Network (ANN) is trained on the patterns of the normal component to tangential component ratios at the time of slippage occurrence, so that it can be able to distinguish the slippage occurrence under different type of load (quasi-static and dynamic loads), and then generates a feedback signal used as an input signal to run the actuator. This process is executed without the need for any information about the characteristics of the grasped object, such as weight, surface texture, shape, coefficient of the friction and the type of the load exerted on the grasped object. For fulfillment this approach, a new fingertip design has been proposed in order to detect the slippage in multi-direction between the grasped object and the artificial fingertips. This design is composed of two under-actuated fingers with an actuation system which includes flexible parts (compressive springs). These springs operate as a compensator for the grasping force at the time of slippage occurrence in spite of the actuator is in stopped situation. The contact force component ratios can be calculated via a conventional sensor (Flexiforce sensor) after processed the force data using Matlab/Simulink program through a specific mathematical model which is derived according to the mechanism of the artificial finger

Design, implementation, and evaluation of a variable stiffness transradial hand prosthesis

We present the design, implementation, and experimental evaluation of a low-cost, customizable, easy-to-use transradial hand prosthesis capable of adapting its compliance. Variable stiffness actuation (VSA) of the prosthesis is based on antagonistically arranged tendons coupled to nonlinear springs driven through a Bowden cable based power transmission. Bowden cable based antagonistic VSA can, not only regulate the stiffness and the position of the prosthetic hand but also enables a light-weight and low-cost design, by the opportunistic placement of motors, batteries, and controllers on any convenient location on the human body, while nonlinear springs are conveniently integrated inside the forearm. The transradial hand prosthesis also features tendon driven underactuated compliant fingers that allow natural adaption of the hand shape to wrap around a wide variety of object geometries, while the modulation of the stiffness of their drive tendons enables the prosthesis to perform various tasks with high dexterity. The compliant fingers of the prosthesis add inherent robustness and flexibility, even under impacts. The control of the variable stiffness transradial hand prosthesis is achieved by an sEMG based natural human-machine interface

A shape memory alloy-based biomimetic robotic hand : design, modelling and experimental evaluation

Every year more the 400,000 people are subject to an upper limb amputation. Projections foresee that this number may double by the 2050. Infections, trauma, cancer, or complications that arise in blood vessels represent the main causes for amputations. The access to prosthetic care is worldwide extremely limited. This is mainly due to the high cost both of commercially available prostheses and of the rehabilitation procedure which every prostheses user has to endure. Aside from high costs, commercially available hand prostheses have faced high rejection rates, mainly due to the their heavy weight, noisy operation and also to the unnatural feel of the fingers. To overcome these limitations, new materials, such as Shape Memory Alloys (SMAs), have been considered as potential candidate actuators for these kind of devices. In order to provide a contribution in the development of performant and easily affordable hand prostheses, the development of a novel and cost-effective five-fingered hand prototype actuated by Shape Memory Alloy (SMA) wires is presented in this work. The dissertation starts with the description of a first generation of a SMA actuated finger. Structure assemblage and performances in term of force, motion and reactiveness are investigated to highlight advantages and disadvantages of the prototype. In order to improve the achievable performances, a second generation of SMA actuated finger having soft features is introduced. Its structure, a five-fingered hand prosthesis having intrinsically elastic fingers, capable to grasp several types of objects with a considerable force, and an entirely 3D printed structure is then presented. Comparing this prototype with the most important prostheses developed so far, relevant advantages especially in term of noiseless actuation, cost, weight, responsiveness and force can be highlighted. A finite element based framework is then developed, to enable additional structure optimization and further improve the SMA finger performances. On the same time, a concentrated parameters physics-based model is formulated to allow, in the future, an easier control of the device, characterized by strong nonlinearities mainly due to the Shape Memory alloy hysteretic behavior.Jedes Jahr werden weltweit bei mehr als 400.000 Menschen Amputationen der oberen Gliedmaßen durchgeführt. Prognosen gehen davon aus, dass sich diese Zahl bis zum Jahr 2050 verdoppeln wird. Hauptursachen der Amputationen sind Infektionen, Unfälle, Krebs oder Durchblutungsstörungen. Der Zugang zu prothetischer Versorgung ist besonders in den Entwicklungsländern stark eingeschränkt. Dies liegt vor allem an den hohen Kosten sowohl der im Handel erhältlichen Prothesen als auch des Rehabilitationsprozesses, den jeder Prothesenträger durchlaufen muss. Neben den hohen Kosten haben kommerziell erhältliche Handprothesen aufgrund ihres hohen Gewichts, des lauten Betriebes und auch des unnatürlichen Gefühls hohe Ablehnungsraten. Um diese Einschränkungen zu überwinden, wurden neue Materialien, wie z.B. Formgedächtnislegierungen (SMAs), als potenzielle Materialien für den Antrieb von Prothesen untersucht . Um einen Beitrag zur Entwicklung von leistungsfähigen und erschwinglichen Handprothesen zu leisten, wird in dieser Arbeit die Entwicklung eines neuartigen und kostengünstigen Fünf-Finger-Handprototyps vorgestellt, der durch Drähte aus Formgedächtnislegierungen aktiviert wird. Die Doktorarbeit beginnt mit der Beschreibung der ersten Generation eines SMA-aktivierten Fingers. Zuerst wird der Aufbau und das Wirkungsprinzip des SMA Fingers erläutert und die Leistungs- und Bewegungsfähigkeit des Systems untersucht sowie Vor- und Nachteile des Prototyps dargestellt. Anschließend, um die erreichbare Leistungsfähigkeit zu verbessern, wird eine zweite Generation von SMA-gesteuerten Fingern vorgestellt, die eine vollständig in 3D gedruckte Struktur aufweisen. Diese Fünffinger-Handprothese mit inhärent elastischen Fingern ermöglicht nicht nur das Greifen unterschiedlich geformter Objekte sondern auch das Heben und Halten schwerer Gegenstände. Dieser neuartige Prototyp wird mit den wichtigsten bisher entwickelten Prothesen verglichen und die relevanten Vorteile insbesondere in Bezug auf geräuschlose Ansteuerung, Kosten, Gewicht, Reaktionszeit und Kraft hervorgehoben. Abschließend wird ein Finite-Elemente-Modell entwickelt, mit Hilfe dessen die Fingerstruktur weiter optimiert und die Leistungsfähigkeit des SMA-Fingers noch verbessert werden kann. Zusätzlich wird ein Konzentriertes-Parameter-Modell formuliert, um, in der Zukunft, eine leichtere Regelung des Systems zu ermöglichen. Dieses ist notwendig, da der SMA-Finger starke Nichtlinearitäten aufweist, die auf das hysteretische Verhalten der Formgedächtnislegierung zurückzuführen sind