The Alignment Between 3-D Data and Articulated Shapes with Bending Surfaces

International audienceIn this paper we address the problem of aligning 3-D data with articulated shapes. This problem resides at the core of many motion tracking methods with applications in human motion capture, action recognition, medical-image analysis, etc. We describe an articulated and bending surface representation well suited for this task as well as a method which aligns (or registers) such a surface to 3-D data. Articulated objects, e.g., humans and animals, are covered with clothes and skin which may be seen as textured surfaces. These surfaces are both articulated and deformable and one realistic way to model them is to assume that they bend in the neighborhood of the shape's joints. We will introduce a surface-bending model as a function of the articulated-motion parameters. This combined articulated-motion and surface-bending model better predicts the observed phenomena in the data and therefore is well suited for surface registration. Given a set of sparse 3-D data (gathered with a stereo camera pair) and a textured, articulated, and bending surface, we describe a register-and-fit method that proceeds as follows. First, the data-to-surface registration problem is formalized as a classifier and is carried out using an EM algorithm. Second, the data-to-surface fitting problem is carried out by minimizing the distance from the registered data points to the surface over the joint variables. In order to illustrate the method we applied it to the problem of hand tracking. A hand model with 27 degrees of freedom is successfully registered and fitted to a sequence of 3-D data points gathered with a stereo camera pair

Motion Capture of Hands in Action Using Discriminative Salient Points

Abstract. Capturing the motion of two hands interacting with an object is a very challenging task due to the large number of degrees of freedom, self-occlusions, and similarity between the fingers, even in the case of multiple cameras observing the scene. In this paper we propose to use discriminatively learned salient points on the fingers and to estimate the finger-salient point associations simultaneously with the estimation of the hand pose. We introduce a differentiable objective function that also takes edges, optical flow and collisions into account. Our qualitative and quantitative evaluations show that the proposed approach achieves very accurate results for several challenging sequences containing hands and objects in action.

Visualization and suppression of interfacial recombination for high-efficiency large-area pin perovskite solar cells

The performance of perovskite solar cells is predominantly limited by non-radiative recombination, either through trap-assisted recombination in the absorber layer or via minority carrier recombination at the perovskite/transport layer interfaces. Here, we use transient and absolute photoluminescence imaging to visualize all non-radiative recombination pathways in planar pin-type perovskite solar cells with undoped organic charge transport layers. We find significant quasi-Fermi-level splitting losses (135 meV) in the perovskite bulk, whereas interfacial recombination results in an additional free energy loss of 80 meV at each individual interface, which limits the open-circuit voltage (V) of the complete cell to ~1.12 V. Inserting ultrathin interlayers between the perovskite and transport layers leads to a substantial reduction of these interfacial losses at both the p and n contacts. Using this knowledge and approach, we demonstrate reproducible dopant-free 1 cm perovskite solar cells surpassing 20% efficiency (19.83% certified) with stabilized power output, a high V (1.17 V) and record fill factor (>81%)

Tracking with the kinematics of extremal contours

This paper addresses the problem of articulated motion tracking from image sequences. We describe a method that relies on an explicit parameterization of the extremal contours in terms of the joint parameters of an associated kinematic model. The latter allows us to predict the extremal contours from the body-part primitives of an articulated model and to compare them with observed image contours. The error function that measures the discrepancy between observed contours and predicted contours is minimized using an analytical expression of the Jacobian that maps joint velocities onto contour velocities. In practice we model people both by their geometry (truncated elliptical cones) and with their articulated structure – a kinematic model with 40 rotational degrees of freedom. We observe image data gathered with several synchronized cameras. The tracker has been successfully applied to image sequences gathered at 30 frames/second

RNA decoys

A method is proposed to track the full hand motion from 3D points on the surface of the hand that were reconstructed and tracked using a stereoscopic set of cameras. This approach combines the advantages of previous methods that use 2D motion (e.g. optical flow), and those that use a 3D reconstruction at each time frame to capture the hand motion. Matching either contours or a 3D reconstruction against a 3D hand model is usually very difficult due to self-occlusions and the locally-cylindrical structure of each phalanx in the model, but our use of 3D point trajectories constrains the motion and overcomes these problems. Our trackin