Causal Confusion in Imitation Learning

Behavioral cloning reduces policy learning to supervised learning by training a discriminative model to predict expert actions given observations. Such discriminative models are non-causal: the training procedure is unaware of the causal structure of the interaction between the expert and the environment. We point out that ignoring causality is particularly damaging because of the distributional shift in imitation learning. In particular, it leads to a counter-intuitive "causal misidentification" phenomenon: access to more information can yield worse performance. We investigate how this problem arises, and propose a solution to combat it through targeted interventions---either environment interaction or expert queries---to determine the correct causal model. We show that causal misidentification occurs in several benchmark control domains as well as realistic driving settings, and validate our solution against DAgger and other baselines and ablations.Comment: Published at NeurIPS 2019 9 pages, plus references and appendice

Microfluidic Digestive Systems for Drug Analysis

As most medicines are given orally, it is important to study what happens to new drugs upon ingestion. A better, more representative and more efficient model of the human gastrointestinal (GI) tract is required for these studies than used today. This thesis describes novel methods to test the behavior of drugs in the human GI tract based on continuously flowing microsystems. In order to reach the desired site of action, drug molecules must first survive the harsh conditions in the GI tract. Samples containing drugs or foods were exposed to artificial versions of digestive juices in micromixers to study the effects of digestion on the samples. The digested samples then reached a gut-on-a-chip – a miniaturized model of the human intestinal barrier – to study the absorption of drugs across a layer of living intestinal cells into the body compartment. This uptake process was studied using state-of-the-art analytical-chemical techniques such as mass spectrometry. Two different model drugs were studied in this system, with one drug clearly broken down by the preceding digestion. The second part of this thesis describes different novel methods and equipment for use in such gut-on-a-chip systems, including a control system to deliver liquids to organs-on-chips for longer experiments (days), new microfabrication strategies to produce very thin, porous membranes to serve as support for the gut-on-a-chip barriers, and 3D-printed components for use in microfluidic digestive systems. It is envisioned that the work described in this thesis may contribute to faster and more efficient studies of novel drugs and foods

EDGI: Equivariant Diffusion for Planning with Embodied Agents

Embodied agents operate in a structured world, often solving tasks with spatial, temporal, and permutation symmetries. Most algorithms for planning and model-based reinforcement learning (MBRL) do not take this rich geometric structure into account, leading to sample inefficiency and poor generalization. We introduce the Equivariant Diffuser for Generating Interactions (EDGI), an algorithm for MBRL and planning that is equivariant with respect to the product of the spatial symmetry group SE(3), the discrete-time translation group Z, and the object permutation group Sn. EDGI follows the Diffuser framework (Janner et al., 2022) in treating both learning a world model and planning in it as a conditional generative modeling problem, training a diffusion model on an offline trajectory dataset. We introduce a new SE(3)xZxSn-equivariant diffusion model that supports multiple representations. We integrate this model in a planning loop, where conditioning and classifier guidance let us softly break the symmetry for specific tasks as needed. On object manipulation and navigation tasks, EDGI is substantially more sample efficient and generalizes better across the symmetry group than non-equivariant models.Comment: Accepted at NeurIPS 2023. v2: matches camera-ready versio

Geometric Algebra Transformers

Problems involving geometric data arise in a variety of fields, including computer vision, robotics, chemistry, and physics. Such data can take numerous forms, such as points, direction vectors, planes, or transformations, but to date there is no single architecture that can be applied to such a wide variety of geometric types while respecting their symmetries. In this paper we introduce the Geometric Algebra Transformer (GATr), a general-purpose architecture for geometric data. GATr represents inputs, outputs, and hidden states in the projective geometric algebra, which offers an efficient 16-dimensional vector space representation of common geometric objects as well as operators acting on them. GATr is equivariant with respect to E(3), the symmetry group of 3D Euclidean space. As a transformer, GATr is scalable, expressive, and versatile. In experiments with n-body modeling and robotic planning, GATr shows strong improvements over non-geometric baselines

Mesh Neural Networks for SE(3)-Equivariant Hemodynamics Estimation on the Artery Wall

Computational fluid dynamics (CFD) is a valuable asset for patient-specific cardiovascular-disease diagnosis and prognosis, but its high computational demands hamper its adoption in practice. Machine-learning methods that estimate blood flow in individual patients could accelerate or replace CFD simulation to overcome these limitations. In this work, we consider the estimation of vector-valued quantities on the wall of three-dimensional geometric artery models. We employ group-equivariant graph convolution in an end-to-end SE(3)-equivariant neural network that operates directly on triangular surface meshes and makes efficient use of training data. We run experiments on a large dataset of synthetic coronary arteries and find that our method estimates directional wall shear stress (WSS) with an approximation error of 7.6% and normalised mean absolute error (NMAE) of 0.4% while up to two orders of magnitude faster than CFD. Furthermore, we show that our method is powerful enough to accurately predict transient, vector-valued WSS over the cardiac cycle while conditioned on a range of different inflow boundary conditions. These results demonstrate the potential of our proposed method as a plugin replacement for CFD in the personalised prediction of hemodynamic vector and scalar fields.Comment: Preprint. Under Revie

Facile fabrication of microperforated membranes with re-useable SU-8 molds for organs-on-chips

Microperforated membranes are essential components of various organ-on-a-chip (OOC) barrier models devel- oped to study transport of molecular compounds and cells across cell layers in e.g. the intestine and blood-brain barrier. These OOC membranes have two functions: 1) to support growth of cells on one or both sides, and 2) to act as a filter-like barrier to separate adjacent compartments. Thin, microperforated poly(dimethylsiloxane) (PDMS) membranes can be fabricated by micromolding from silicon molds comprising arrays of micropillars for the formation of micropores. However, these molds are made by deep reactive ion etching (DRIE) and are expensive to fabricate. We describe the micromolding of thin PDMS membranes with easier-to-make, SU-8 epoxy photoresist molds. With a multilayer, SU-8, pillar microarray mold, massively parallel arrays of micropores can be formed in a thin layer of PDMS, resulting in a flexible barrier membrane that can be easily incorporated and sealed between other layers making up the OOC device. The membranes we describe here have a 30-μm thickness, with 12-μm-diameter circular pores arranged at a 100-μm pitch in a square array. We show application of these membranes in gut-on-a-chip devices, and expect that the reported fabrication strategy will also be suitable for other membrane dimension