Real-time 3D reconstruction of non-rigid shapes with a single moving camera

© . This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/This paper describes a real-time sequential method to simultaneously recover the camera motion and the 3D shape of deformable objects from a calibrated monocular video. For this purpose, we consider the Navier-Cauchy equations used in 3D linear elasticity and solved by finite elements, to model the time-varying shape per frame. These equations are embedded in an extended Kalman filter, resulting in sequential Bayesian estimation approach. We represent the shape, with unknown material properties, as a combination of elastic elements whose nodal points correspond to salient points in the image. The global rigidity of the shape is encoded by a stiffness matrix, computed after assembling each of these elements. With this piecewise model, we can linearly relate the 3D displacements with the 3D acting forces that cause the object deformation, assumed to be normally distributed. While standard finite-element-method techniques require imposing boundary conditions to solve the resulting linear system, in this work we eliminate this requirement by modeling the compliance matrix with a generalized pseudoinverse that enforces a pre-fixed rank. Our framework also ensures surface continuity without the need for a post-processing step to stitch all the piecewise reconstructions into a global smooth shape. We present experimental results using both synthetic and real videos for different scenarios ranging from isometric to elastic deformations. We also show the consistency of the estimation with respect to 3D ground truth data, include several experiments assessing robustness against artifacts and finally, provide an experimental validation of our performance in real time at frame rate for small mapsPeer ReviewedPostprint (author's final draft

Deep Detection of People and their Mobility Aids for a Hospital Robot

Robots operating in populated environments encounter many different types of people, some of whom might have an advanced need for cautious interaction, because of physical impairments or their advanced age. Robots therefore need to recognize such advanced demands to provide appropriate assistance, guidance or other forms of support. In this paper, we propose a depth-based perception pipeline that estimates the position and velocity of people in the environment and categorizes them according to the mobility aids they use: pedestrian, person in wheelchair, person in a wheelchair with a person pushing them, person with crutches and person using a walker. We present a fast region proposal method that feeds a Region-based Convolutional Network (Fast R-CNN). With this, we speed up the object detection process by a factor of seven compared to a dense sliding window approach. We furthermore propose a probabilistic position, velocity and class estimator to smooth the CNN's detections and account for occlusions and misclassifications. In addition, we introduce a new hospital dataset with over 17,000 annotated RGB-D images. Extensive experiments confirm that our pipeline successfully keeps track of people and their mobility aids, even in challenging situations with multiple people from different categories and frequent occlusions. Videos of our experiments and the dataset are available at http://www2.informatik.uni-freiburg.de/~kollmitz/MobilityAidsComment: 7 pages, ECMR 2017, dataset and videos: http://www2.informatik.uni-freiburg.de/~kollmitz/MobilityAids

Search for a Metallic Dangling-Bond Wire on nn-doped H-passivated Semiconductor Surfaces

We have theoretically investigated the electronic properties of neutral and $n$-doped dangling bond (DB) quasi-one-dimensional structures (lines) in the Si(001):H and Ge(001):H substrates with the aim of identifying atomic-scale interconnects exhibiting metallic conduction for use in on-surface circuitry. Whether neutral or doped, DB lines are prone to suffer geometrical distortions or have magnetic ground-states that render them semiconducting. However, from our study we have identified one exception -- a dimer row fully stripped of hydrogen passivation. Such a DB-dimer line shows an electronic band structure which is remarkably insensitive to the doping level and, thus, it is possible to manipulate the position of the Fermi level, moving it away from the gap. Transport calculations demonstrate that the metallic conduction in the DB-dimer line can survive thermally induced disorder, but is more sensitive to imperfect patterning. In conclusion, the DB-dimer line shows remarkable stability to doping and could serve as a one-dimensional metallic conductor on $n$-doped samples.Comment: 8 pages, 5 figure

Use of graphics in decision aids for telerobotic control: (Parts 5-8 of an 8-part MIT progress report)

Four separate projects recently completed or in progress at the MIT Man-Machine Systems Laboratory are summarized. They are: a decision aid for retrieving a tumbling satellite in space; kinematic control and graphic display of redundant teleoperators; real time terrain/object generation: a quad-tree approach; and two dimensional control for three dimensional obstacle avoidance

Revealing hidden scenes by photon-efficient occlusion-based opportunistic active imaging

The ability to see around corners, i.e., recover details of a hidden scene from its reflections in the surrounding environment, is of considerable interest in a wide range of applications. However, the diffuse nature of light reflected from typical surfaces leads to mixing of spatial information in the collected light, precluding useful scene reconstruction. Here, we employ a computational imaging technique that opportunistically exploits the presence of occluding objects, which obstruct probe-light propagation in the hidden scene, to undo the mixing and greatly improve scene recovery. Importantly, our technique obviates the need for the ultrafast time-of-flight measurements employed by most previous approaches to hidden-scene imaging. Moreover, it does so in a photon-efficient manner based on an accurate forward model and a computational algorithm that, together, respect the physics of three-bounce light propagation and single-photon detection. Using our methodology, we demonstrate reconstruction of hidden-surface reflectivity patterns in a meter-scale environment from non-time-resolved measurements. Ultimately, our technique represents an instance of a rich and promising new imaging modality with important potential implications for imaging science.Comment: Related theory in arXiv:1711.0629

Birefringence induced by pp-wave modes in an electromagnetically active dynamic aether

In the framework of the Einstein-Maxwell-aether theory we study the birefringence effect, which can occur in the pp-wave symmetric dynamic aether. The dynamic aether is considered to be latently birefringent quasi-medium, which displays this hidden property if and only if the aether motion is non-uniform, i.e., when the aether flow is characterized by the non-vanishing expansion, shear, vorticity or acceleration. In accordance with the dynamo-optical scheme of description of the interaction between electromagnetic waves and the dynamic aether, we shall model the susceptibility tensors by the terms linear in the covariant derivative of the aether velocity four-vector. When the pp-wave modes appear in the dynamic aether, we deal with a gravitationally induced degeneracy removal with respect to hidden susceptibility parameters. As a consequence, the phase velocities of electromagnetic waves possessing orthogonal polarizations do not coincide, thus displaying the birefringence effect. Two electromagnetic field configurations are studied in detail: longitudinal and transversal with respect to the aether pp-wave front. For both cases the solutions are found, which reveal anomalies in the electromagnetic response on the action of the pp-wave aether mode.Comment: 13 pages, 0 figures, list of references is modified, misprints eliminate