Learning Object Manipulation Skills from Video via Approximate Differentiable Physics

We aim to teach robots to perform simple object manipulation tasks by watching a single video demonstration. Towards this goal, we propose an optimization approach that outputs a coarse and temporally evolving 3D scene to mimic the action demonstrated in the input video. Similar to previous work, a differentiable renderer ensures perceptual fidelity between the 3D scene and the 2D video. Our key novelty lies in the inclusion of a differentiable approach to solve a set of Ordinary Differential Equations (ODEs) that allows us to approximately model laws of physics such as gravity, friction, and hand-object or object-object interactions. This not only enables us to dramatically improve the quality of estimated hand and object states, but also produces physically admissible trajectories that can be directly translated to a robot without the need for costly reinforcement learning. We evaluate our approach on a 3D reconstruction task that consists of 54 video demonstrations sourced from 9 actions such as pull something from right to left or put something in front of something. Our approach improves over previous state-of-the-art by almost 30%, demonstrating superior quality on especially challenging actions involving physical interactions of two objects such as put something onto something. Finally, we showcase the learned skills on a Franka Emika Panda robot.Comment: Accepted for IROS2022, code at https://github.com/petrikvladimir/video_skills_learning_with_approx_physics, project page at https://data.ciirc.cvut.cz/public/projects/2022Real2SimPhysics

Differentiable Collision Detection: a Randomized Smoothing Approach

Collision detection appears as a canonical operation in a large range of robotics applications from robot control to simulation, including motion planning and estimation. While the seminal works on the topic date back to the 80s, it is only recently that the question of properly differentiating collision detection has emerged as a central issue, thanks notably to the ongoing and various efforts made by the scientific community around the topic of differentiable physics. Yet, very few solutions have been suggested so far, and only with a strong assumption on the nature of the shapes involved. In this work, we introduce a generic and efficient approach to compute the derivatives of collision detection for any pair of convex shapes, by notably leveraging randomized smoothing techniques which have shown to be particularly adapted to capture the derivatives of non-smooth problems. This approach is implemented in the HPP-FCL and Pinocchio ecosystems, and evaluated on classic datasets and problems of the robotics literature, demonstrating few micro-second timings to compute informative derivatives directly exploitable by many real robotic applications including differentiable simulation.Comment: 7 pages, 6 figures, 2 table

Spontaneous Spinal Epidural Haematoma Secondary to Autoimmune Acquired Haemophilia

Spontaneous spinal epidural haematoma is a rare entity associated with high morbidity. Although there are previous reports of spinal haematoma secondary to X-linked genetic haemophilia, there are no such cases secondary to acquired autoimmune haemophilia. We report the case of a 71-year-old patient who presented with sudden quadriplegia secondary to cervical (C2 to T1) epidural haematoma as a result of undiagnosed autoimmune acquired haemophilia A. She underwent emergency cervical laminectomy and evacuation of spinal haematoma with significant recovery in upper limb function. This case highlights the importance of haematological investigations in patients with spontaneous spinal haematoma

Flavonoid diosmetin increases ATP levels in kidney cells and relieves ATP depleting effect of ochratoxin A

Diosmetin (DIOS) is a flavone aglycone commonly occurring in citrus species and olive leaves, in addition it is one of the active ingredients of some medications. Based on both in vitro and in vivo studies several beneficial effects are attributed to DIOS but the biochemical background of its action seems to be complex and it has not been completely explored yet. Previous investigations suggest that most of the flavonoid aglycones have negative effect on ATP synthesis in a dose dependent manner. In our study 17 flavonoids were tested and interestingly DIOS caused a significant elevation of intracellular ATP levels after 6- and 12-h incubation in MDCK kidney cells. In order to understand the mechanism of action, intracellular ATP and protein levels, ATP/ADP ratio, cell viability and ROS levels were determined after DIOS treatment. In addition, impacts of different enzyme inhibitors and effect of DIOS on isolated rat liver mitochondria were also tested. Finally, the influence of DIOS on the ATP depleting effect of the mycotoxin, ochratoxin A was also investigated. Our major conclusions are the followings: DIOS increases intracellular ATP levels both in kidney and in liver cells. Inhibition of glycolysis or citric acid cycle does not decrease the observed effect. DIOS-induced elevation of ATP levels is completely abolished by the inhibition of ATP synthase. DIOS is able to completely reverse the ATP-depleting effect of the mycotoxin, ochratoxin A. Most probably the DIOS-induced impact on ATP system does not originate from the antioxidant property of DIOS. Based on our findings DIOS may be promising agent to positively influence ATP depletion caused by some metabolic poisons

Early and Non-invasive Diagnosis of Aspergillosis Revealed by Infection Kinetics Monitored in a Rat Model

Background:Aspergillus fumigatus is a ubiquitous saprophytic airborne fungus responsible for more than one million deaths every year. The siderophores of A. fumigatus represent important virulence factors that contribute to the microbiome-metabolome dialog in a host. From a diagnostic point of view, the monitoring of Aspergillus secondary metabolites in urine of a host is promising due to the non-invasiveness, rapidity, sensitivity, and potential for standardization.Methods: Using a model of experimental aspergillosis in immunocompromised Lewis rats, the fungal siderophores ferricrocin (FC) and triacetylfusarinine C (TAFC) were monitored in rat urine before and after lung inoculation with A. fumigatus conidia. Molecular biomarkers in high-dose (HD) and low-dose (LD) infection models were separated using high performance liquid chromatography (HPLC) and were detected by mass spectrometry (MS). In the current work, we corroborated the in vivo MS infection kinetics data with micro-positron emission tomography/computed tomography (μPET/CT) kinetics utilizing 68Ga-labeled TAFC.Results: In the HD model, the initial FC signal reflecting aspergillosis appeared as early as 4 h post-infection. The results from seven biological replicates showed exponentially increasing metabolite profiles over time. In A. fumigatus, TAFC was found to be a less produced biomarker that exhibited a kinetic profile identical to that of FC. The amount of siderophores contributed by the inoculating conidia was negligible and undetectable in the HD and LD models, respectively. In the μPET/CT scans, the first detectable signal in HD model was recorded 48 h post-infection. Regarding the MS assay, among nine biological replicates in the LD model, three animals did not develop any infection, while one animal experienced an exponential increase of metabolites and died on day 6 post-infection. All remaining animals had constant or random FC levels and exhibited few or no symptoms to the experiment termination. In the LD model, the TAFC concentration was not statistically significant, while the μPET/CT scan was positive as early as 6 days post-infection.Conclusion: Siderophore detection in rat urine by MS represents an early and non-invasive tool for diagnosing aspergillosis caused by A. fumigatus. μPET/CT imaging further determines the infection location in vivo and allows the visualization of the infection progression over time

Feedback-based Fabric Strip Folding

Accurate manipulation of a deformable body such as a piece of fabric is difficult because of its many degrees of freedom and unobservable properties affecting its dynamics. To alleviate these challenges, we propose the application of feedback-based control to robotic fabric strip folding. The feedback is computed from the low dimensional state extracted from a camera image. We trained the controller using reinforcement learning in simulation which was calibrated to cover the real fabric strip behaviors. The proposed feedback-based folding was experimentally compared to two state-of-the-art folding methods and our method outperformed both of them in terms of accuracy.Peer reviewe

Influence of Recent History to System Controllability

Workshop: Reinforcement Learning under Partial ObservabilityPeer reviewe

Influence of Recent History to System Controllability

Workshop: Reinforcement Learning under Partial ObservabilityPeer reviewe