38 research outputs found
Meta-Learning via Classifier(-free) Guidance
State-of-the-art meta-learning techniques do not optimize for zero-shot adaptation to unseen tasks, a setting in which humans excel. On the contrary, meta-learning algorithms learn hyperparameters and weight initializations that explicitly optimize for few-shot learning performance. In this work, we take inspiration from recent advances in generative modeling and language-conditioned image synthesis to propose meta-learning techniques that use natural language guidance to achieve higher zero-shot performance compared to the state-of-the-art. We do so by recasting the meta-learning problem as a multi-modal generative modeling problem: given a task, we consider its adapted neural network weights and its natural language description as equivalent multi-modal task representations. We first train an unconditional generative hypernetwork model to produce neural network weights; then we train a second "guidance" model that, given a natural language task description, traverses the hypernetwork latent space to find high-performance task-adapted weights in a zero-shot manner. We explore two alternative approaches for latent space guidance: "HyperCLIP"-based classifier guidance and a conditional Hypernetwork Latent Diffusion Model ("HyperLDM"), which we show to benefit from the classifier-free guidance technique common in image generation. Finally, we demonstrate that our approaches outperform existing meta-learning methods with zero-shot learning experiments on our Meta-VQA dataset, which we specifically constructed to reflect the multi-modal meta-learning setting
Vandermonde Neural Operators
Fourier Neural Operators (FNOs) have emerged as very popular machine learning
architectures for learning operators, particularly those arising in PDEs.
However, as FNOs rely on the fast Fourier transform for computational
efficiency, the architecture can be limited to input data on equispaced
Cartesian grids. Here, we generalize FNOs to handle input data on
non-equispaced point distributions. Our proposed model, termed as Vandermonde
Neural Operator (VNO), utilizes Vandermonde-structured matrices to efficiently
compute forward and inverse Fourier transforms, even on arbitrarily distributed
points. We present numerical experiments to demonstrate that VNOs can be
significantly faster than FNOs, while retaining comparable accuracy, and
improve upon accuracy of comparable non-equispaced methods such as the Geo-FNO.Comment: 21 pages, 10 figure
Perforated red blood cells enable compressible and injectable hydrogels as therapeutic vehicles
Hydrogels engineered for medical use within the human body need to be
delivered in a minimally invasive fashion without altering their biochemical
and mechanical properties to maximize their therapeutic outcomes. In this
regard, key strategies applied for creating such medical hydrogels include
formulating precursor solutions that can be crosslinked in situ with physical
or chemical cues following their delivery or forming macroporous hydrogels at
sub-zero temperatures via cryogelation prior to their delivery. Here, we
present a new class of injectable composite materials with shape recovery
ability. The shape recovery is derived from the physical properties of red
blood cells (RBCs) that are first modified via hypotonic swelling and then
integrated into the hydrogel scaffolds before polymerization. The RBCs'
hypotonic swelling induces the formation of nanometer-sized pores on their cell
membranes, which enable fast liquid release under compression. The resulting
biocomposite hydrogel scaffolds display high deformability and shape-recovery
ability. The scaffolds can repeatedly compress up to ~87% of their original
volumes during injection and subsequent retraction through syringe needles of
different sizes; this cycle of injection and retraction can be repeated up to
ten times without causing any substantial mechanical damage to the scaffolds.
Our biocomposite material system and fabrication approach for injectable
materials will be foundational for the minimally invasive delivery of
drug-loaded scaffolds, tissue-engineered constructs, and personalized medical
platforms that could be administered to the human body with conventional
needle-syringe systems
ViSE: Vision-Based 3D Online Shape Estimation of Continuously Deformable Robots
The precise control of soft and continuum robots requires knowledge of their
shape. The shape of these robots has, in contrast to classical rigid robots,
infinite degrees of freedom. To partially reconstruct the shape, proprioceptive
techniques use built-in sensors resulting in inaccurate results and increased
fabrication complexity. Exteroceptive methods so far rely on placing reflective
markers on all tracked components and triangulating their position using
multiple motion-tracking cameras. Tracking systems are expensive and infeasible
for deformable robots interacting with the environment due to marker occlusion
and damage. Here, we present a regression approach for 3D shape estimation
using a convolutional neural network. The proposed approach takes advantage of
data-driven supervised learning and is capable of real-time marker-less shape
estimation during inference. Two images of a robotic system are taken
simultaneously at 25 Hz from two different perspectives, and are fed to the
network, which returns for each pair the parameterized shape. The proposed
approach outperforms marker-less state-of-the-art methods by a maximum of 4.4%
in estimation accuracy while at the same time being more robust and requiring
no prior knowledge of the shape. The approach can be easily implemented due to
only requiring two color cameras without depth and not needing an explicit
calibration of the extrinsic parameters. Evaluations on two types of soft
robotic arms and a soft robotic fish demonstrate our method's accuracy and
versatility on highly deformable systems in real-time. The robust performance
of the approach against different scene modifications (camera alignment and
brightness) suggests its generalizability to a wider range of experimental
setups, which will benefit downstream tasks such as robotic grasping and
manipulation
Getting the Ball Rolling: Learning a Dexterous Policy for a Biomimetic Tendon-Driven Hand with Rolling Contact Joints
Biomimetic, dexterous robotic hands have the potential to replicate much of
the tasks that a human can do, and to achieve status as a general manipulation
platform. Recent advances in reinforcement learning (RL) frameworks have
achieved remarkable performance in quadrupedal locomotion and dexterous
manipulation tasks. Combined with GPU-based highly parallelized simulations
capable of simulating thousands of robots in parallel, RL-based controllers
have become more scalable and approachable. However, in order to bring
RL-trained policies to the real world, we require training frameworks that
output policies that can work with physical actuators and sensors as well as a
hardware platform that can be manufactured with accessible materials yet is
robust enough to run interactive policies. This work introduces the biomimetic
tendon-driven Faive Hand and its system architecture, which uses tendon-driven
rolling contact joints to achieve a 3D printable, robust high-DoF hand design.
We model each element of the hand and integrate it into a GPU simulation
environment to train a policy with RL, and achieve zero-shot transfer of a
dexterous in-hand sphere rotation skill to the physical robot hand.Comment: for project website, see https://srl-ethz.github.io/get-ball-rolling/
. for video, see https://youtu.be/YahsMhqNU8o . Submitted to the 2023
IEEE-RAS International Conference on Humanoid Robot
Low Voltage Electrohydraulic Actuators for Untethered Robotics
Rigid robots can be precise in repetitive tasks but struggle in unstructured
environments. Nature's versatility in such environments inspires researchers to
develop biomimetic robots that incorporate compliant and contracting artificial
muscles. Among the recently proposed artificial muscle technologies,
electrohydraulic actuators are promising since they offer comparable
performance to mammalian muscles in terms of speed and power density. However,
they require high driving voltages and have safety concerns due to exposed
electrodes. These high voltages lead to either bulky or inefficient driving
electronics that make untethered, high-degree-of-freedom bio-inspired robots
difficult to realize. Here, we present low voltage electrohydraulic actuators
(LEAs) that match mammalian skeletal muscles in average power density (50.5
W/kg) and peak strain rate (971 percent/s) at a driving voltage of just 1100 V.
This driving voltage is approx. 5 - 7 times lower compared to other
electrohydraulic actuators using paraelectric dielectrics. Furthermore, LEAs
are safe to touch, waterproof, and self-clearing, which makes them easy to
implement in wearables and robotics. We characterize, model, and physically
validate key performance metrics of the actuator and compare its performance to
state-of-the-art electrohydraulic designs. Finally, we demonstrate the utility
of our actuators on two muscle-based electrohydraulic robots: an untethered
soft robotic swimmer and a robotic gripper. We foresee that LEAs can become a
key building block for future highly-biomimetic untethered robots and wearables
with many independent artificial muscles such as biomimetic hands, faces, or
exoskeletons.Comment: Stephan-Daniel Gravert and Elia Varini contributed equally to this
wor