130 research outputs found

    How worms move in 3D

    Get PDF
    Animals that live in the sky, underwater or underground display unique three dimensional behaviours made possible by their ability to generate movement in all directions. As animals explore their environment, they constantly adapt their locomotion strategies to balance factors such as distance travelled, speed, and energy expenditure. While exploration strategies have been widely studied across a variety of species, how animals explore 3D space remains an open problem. The nematode Caenorhabditis elegans presents an ideal candidate for the study of 3D exploration as it is naturally found in complex fluid and granular environments and is well sized (~1mm long) for the simultaneous capture of individual postures and long term trajectories using a fixed imaging setup. However, until recently C. elegans has been studied almost exclusively in planar environments and in 3D neither its modes of locomotion nor its exploration strategies are known. Here we present methods for reconstructing microscopic postures and tracking macroscopic trajectories from a large corpus of triaxial recordings of worms freely exploring complex gelatinous fluids. To account for the constantly changing optical properties of these gels we develop a novel differentiable renderer to construct images from 3D postures for direct comparison with the recorded images. The method is robust to interference such as air bubbles and dirt trapped in the gel, stays consistent through complex sequences of postures and recovers reliable estimates from low-resolution, blurry images. Using this approach we generate a large dataset of 3D exploratory trajectories (over 6 hours) and midline postures (over 4 hours). We find that C. elegans explore 3D space through the composition of quasi-planar regions separated by turns and variable-length runs. To achieve this, C. elegans use locomotion gaits and complex manoeuvres that differ from those previously observed on an agar surface. We show that the associated costs of locomotion increase with non-planarity and we develop a mathematical model to probe the implications of this connection. We find that quasi-planar strategies (such as we find in the data) yield the largest volumes explored as they provide a balance between 3D coverage and trajectory distance. Taken together, our results link locomotion primitives with exploration strategies in the context of short term volumetric foraging to provide a first integrated study into how worms move in 3D

    NR-SLAM: Non-Rigid Monocular SLAM

    Full text link
    In this paper we present NR-SLAM, a novel non-rigid monocular SLAM system founded on the combination of a Dynamic Deformation Graph with a Visco-Elastic deformation model. The former enables our system to represent the dynamics of the deforming environment as the camera explores, while the later allows us to model general deformations in a simple way. The presented system is able to automatically initialize and extend a map modeled by a sparse point cloud in deforming environments, that is refined with a sliding-window Deformable Bundle Adjustment. This map serves as base for the estimation of the camera motion and deformation and enables us to represent arbitrary surface topologies, overcoming the limitations of previous methods. To assess the performance of our system in challenging deforming scenarios, we evaluate it in several representative medical datasets. In our experiments, NR-SLAM outperforms previous deformable SLAM systems, achieving millimeter reconstruction accuracy and bringing automated medical intervention closer. For the benefit of the community, we make the source code public.Comment: 12 pages, 7 figures, submited to the IEEE Transactions on Robotics (T-RO

    Deep Structured Layers for Instance-Level Optimization in 2D and 3D Vision

    Get PDF
    The approach we present in this thesis is that of integrating optimization problems as layers in deep neural networks. Optimization-based modeling provides an additional set of tools enabling the design of powerful neural networks for a wide battery of computer vision tasks. This thesis shows formulations and experiments for vision tasks ranging from image reconstruction to 3D reconstruction. We first propose an unrolled optimization method with implicit regularization properties for reconstructing images from noisy camera readings. The method resembles an unrolled majorization minimization framework with convolutional neural networks acting as regularizers. We report state-of-the-art performance in image reconstruction on both noisy and noise-free evaluation setups across many datasets. We further focus on the task of monocular 3D reconstruction of articulated objects using video self-supervision. The proposed method uses a structured layer for accurate object deformation that controls a 3D surface by displacing a small number of learnable handles. While relying on a small set of training data per category for self-supervision, the method obtains state-of-the-art reconstruction accuracy with diverse shapes and viewpoints for multiple articulated objects. We finally address the shortcomings of the previous method that revolve around regressing the camera pose using multiple hypotheses. We propose a method that recovers a 3D shape from a 2D image by relying solely on 3D-2D correspondences regressed from a convolutional neural network. These correspondences are used in conjunction with an optimization problem to estimate per sample the camera pose and deformation. We quantitatively show the effectiveness of the proposed method on self-supervised 3D reconstruction on multiple categories without the need for multiple hypotheses

    Planning Framework for Robotic Pizza Dough Stretching with a Rolling Pin

    Get PDF
    Stretching a pizza dough with a rolling pin is a nonprehensile manipulation. Since the object is deformable, force closure cannot be established, and the manipulation is carried out in a nonprehensile way. The framework of this pizza dough stretching application that is explained in this chapter consists of four sub-procedures: (i) recognition of the pizza dough on a plate, (ii) planning the necessary steps to shape the pizza dough to the desired form, (iii) path generation for a rolling pin to execute the output of the pizza dough planner, and (iv) inverse kinematics for the bi-manual robot to grasp and control the rolling pin properly. Using the deformable object model described in Chap. 3, each sub-procedure of the proposed framework is explained sequentially

    Einstein vs. Bergson

    Get PDF
    On 6 April 1922, Einstein met Bergson to debate the nature of time: is the time the physicist calculates the same time the philosopher reflects on? Einstein claimed that only scientific time is real, while Bergson argued that scientific time always presupposes a living and perceiving subject. On that day, nearly 100 years ago, conflict was inevitable. Is it still inevitable today? How many kinds of time are there

    Deep Shape-from-Template: Single-image quasi-isometric deformable registration and reconstruction

    Get PDF
    Shape-from-Template (SfT) solves 3D vision from a single image and a deformable 3D object model, called a template. Concretely, SfT computes registration (the correspondence between the template and the image) and reconstruction (the depth in camera frame). It constrains the object deformation to quasi-isometry. Real-time and automatic SfT represents an open problem for complex objects and imaging conditions. We present four contributions to address core unmet challenges to realise SfT with a Deep Neural Network (DNN). First, we propose a novel DNN called DeepSfT, which encodes the template in its weights and hence copes with highly complex templates. Second, we propose a semi-supervised training procedure to exploit real data. This is a practical solution to overcome the render gap that occurs when training only with simulated data. Third, we propose a geometry adaptation module to deal with different cameras at training and inference. Fourth, we combine statistical learning with physics-based reasoning. DeepSfT runs automatically and in real-time and we show with numerous experiments and an ablation study that it consistently achieves a lower 3D error than previous work. It outperforms in generalisation and achieves great performance in terms of reconstruction and registration error with wide-baseline, occlusions, illumination changes, weak texture and blur.Agencia Estatal de InvestigaciónMinisterio de Educació

    State of the Art in Dense Monocular Non-Rigid 3D Reconstruction

    Get PDF
    3D reconstruction of deformable (or non-rigid) scenes from a set of monocular2D image observations is a long-standing and actively researched area ofcomputer vision and graphics. It is an ill-posed inverse problem,since--without additional prior assumptions--it permits infinitely manysolutions leading to accurate projection to the input 2D images. Non-rigidreconstruction is a foundational building block for downstream applicationslike robotics, AR/VR, or visual content creation. The key advantage of usingmonocular cameras is their omnipresence and availability to the end users aswell as their ease of use compared to more sophisticated camera set-ups such asstereo or multi-view systems. This survey focuses on state-of-the-art methodsfor dense non-rigid 3D reconstruction of various deformable objects andcomposite scenes from monocular videos or sets of monocular views. It reviewsthe fundamentals of 3D reconstruction and deformation modeling from 2D imageobservations. We then start from general methods--that handle arbitrary scenesand make only a few prior assumptions--and proceed towards techniques makingstronger assumptions about the observed objects and types of deformations (e.g.human faces, bodies, hands, and animals). A significant part of this STAR isalso devoted to classification and a high-level comparison of the methods, aswell as an overview of the datasets for training and evaluation of thediscussed techniques. We conclude by discussing open challenges in the fieldand the social aspects associated with the usage of the reviewed methods.<br

    Vehicle and Traffic Safety

    Get PDF
    The book is devoted to contemporary issues regarding the safety of motor vehicles and road traffic. It presents the achievements of scientists, specialists, and industry representatives in the following selected areas of road transport safety and automotive engineering: active and passive vehicle safety, vehicle dynamics and stability, testing of vehicles (and their assemblies), including electric cars as well as autonomous vehicles. Selected issues from the area of accident analysis and reconstruction are discussed. The impact on road safety of aspects such as traffic control systems, road infrastructure, and human factors is also considered

    State of the Art in Dense Monocular Non-Rigid 3D Reconstruction

    Full text link
    3D reconstruction of deformable (or non-rigid) scenes from a set of monocular 2D image observations is a long-standing and actively researched area of computer vision and graphics. It is an ill-posed inverse problem, since--without additional prior assumptions--it permits infinitely many solutions leading to accurate projection to the input 2D images. Non-rigid reconstruction is a foundational building block for downstream applications like robotics, AR/VR, or visual content creation. The key advantage of using monocular cameras is their omnipresence and availability to the end users as well as their ease of use compared to more sophisticated camera set-ups such as stereo or multi-view systems. This survey focuses on state-of-the-art methods for dense non-rigid 3D reconstruction of various deformable objects and composite scenes from monocular videos or sets of monocular views. It reviews the fundamentals of 3D reconstruction and deformation modeling from 2D image observations. We then start from general methods--that handle arbitrary scenes and make only a few prior assumptions--and proceed towards techniques making stronger assumptions about the observed objects and types of deformations (e.g. human faces, bodies, hands, and animals). A significant part of this STAR is also devoted to classification and a high-level comparison of the methods, as well as an overview of the datasets for training and evaluation of the discussed techniques. We conclude by discussing open challenges in the field and the social aspects associated with the usage of the reviewed methods.Comment: 25 page
    corecore