Semantically Informed Multiview Surface Refinement

We present a method to jointly refine the geometry and semantic segmentation of 3D surface meshes. Our method alternates between updating the shape and the semantic labels. In the geometry refinement step, the mesh is deformed with variational energy minimization, such that it simultaneously maximizes photo-consistency and the compatibility of the semantic segmentations across a set of calibrated images. Label-specific shape priors account for interactions between the geometry and the semantic labels in 3D. In the semantic segmentation step, the labels on the mesh are updated with MRF inference, such that they are compatible with the semantic segmentations in the input images. Also, this step includes prior assumptions about the surface shape of different semantic classes. The priors induce a tight coupling, where semantic information influences the shape update and vice versa. Specifically, we introduce priors that favor (i) adaptive smoothing, depending on the class label; (ii) straightness of class boundaries; and (iii) semantic labels that are consistent with the surface orientation. The novel mesh-based reconstruction is evaluated in a series of experiments with real and synthetic data. We compare both to state-of-the-art, voxel-based semantic 3D reconstruction, and to purely geometric mesh refinement, and demonstrate that the proposed scheme yields improved 3D geometry as well as an improved semantic segmentation

Gas Kinematics and Excitation in the Filamentary IRDC G035.39-00.33

Some theories of dense molecular cloud formation involve dynamical environments driven by converging atomic flows or collisions between preexisting molecular clouds. The determination of the dynamics and physical conditions of the gas in clouds at the early stages of their evolution is essential to establish the dynamical imprints of such collisions, and to infer the processes involved in their formation. We present multi-transition 13CO and C18O maps toward the IRDC G035.39-00.33, believed to be at the earliest stages of evolution. The 13CO and C18O gas is distributed in three filaments (Filaments 1, 2 and 3), where the most massive cores are preferentially found at the intersecting regions between them. The filaments have a similar kinematic structure with smooth velocity gradients of ~0.4-0.8 km s-1 pc-1. Several scenarios are proposed to explain these gradients, including cloud rotation, gas accretion along the filaments, global gravitational collapse, and unresolved sub-filament structures. These results are complemented by HCO+, HNC, H13CO+ and HN13C single-pointing data to search for gas infall signatures. The 13CO and C18O gas motions are supersonic across G035.39-00.33, with the emission showing broader linewidths toward the edges of the IRDC. This could be due to energy dissipation at the densest regions in the cloud. The average H2 densities are ~5000-7000 cm-3, with Filaments 2 and 3 being denser and more massive than Filament 1. The C18O data unveils three regions with high CO depletion factors (f_D~5-12), similar to those found in massive starless cores.Comment: 20 pages, 14 figures, 6 tables, accepted for publication in MNRA

ALMA data suggest the presence of a spiral structure in the inner wind of CW Leo

(abbreviated) We aim to study the inner wind of the well-known AGB star CW Leo. Different diagnostics probing different geometrical scales have pointed toward a non-homogeneous mass-loss process: dust clumps are observed at milli-arcsec scale, a bipolar structure is seen at arcsecond-scale and multi-concentric shells are detected beyond 1". We present the first ALMA Cycle 0 band 9 data around 650 GHz. The full-resolution data have a spatial resolution of 0".42x0".24, allowing us to study the morpho-kinematical structure within ~6". Results: We have detected 25 molecular lines. The emission of all but one line is spatially resolved. The dust and molecular lines are centered around the continuum peak position. The dust emission has an asymmetric distribution with a central peak flux density of ~2 Jy. The molecular emission lines trace different regions in the wind acceleration region and suggest that the wind velocity increases rapidly from about 5 R* almost reaching the terminal velocity at ~11 R*. The channel maps for the brighter lines show a complex structure; specifically for the 13CO J=6-5 line different arcs are detected within the first few arcseconds. The curved structure present in the PV map of the 13CO J=6-5 line can be explained by a spiral structure in the inner wind, probably induced by a binary companion. From modeling the ALMA data, we deduce that the potential orbital axis for the binary system lies at a position angle of ~10-20 deg to the North-East and that the spiral structure is seen almost edge-on. We infer an orbital period of 55 yr and a binary separation of 25 au (or ~8.2 R*). We tentatively estimate that the companion is an unevolved low-mass main-sequence star. The ALMA data hence provide us for the first time with the crucial kinematical link between the dust clumps seen at milli-arcsecond scale and the almost concentric arcs seen at arcsecond scale.Comment: 22 pages, 18 Figures, Astronomy & Astrophysic

Texture Mixer: A Network for Controllable Synthesis and Interpolation of Texture

This paper addresses the problem of interpolating visual textures. We formulate this problem by requiring (1) by-example controllability and (2) realistic and smooth interpolation among an arbitrary number of texture samples. To solve it we propose a neural network trained simultaneously on a reconstruction task and a generation task, which can project texture examples onto a latent space where they can be linearly interpolated and projected back onto the image domain, thus ensuring both intuitive control and realistic results. We show our method outperforms a number of baselines according to a comprehensive suite of metrics as well as a user study. We further show several applications based on our technique, which include texture brush, texture dissolve, and animal hybridization.Comment: Accepted to CVPR'1

Evaluation of CNN-based Single-Image Depth Estimation Methods

While an increasing interest in deep models for single-image depth estimation methods can be observed, established schemes for their evaluation are still limited. We propose a set of novel quality criteria, allowing for a more detailed analysis by focusing on specific characteristics of depth maps. In particular, we address the preservation of edges and planar regions, depth consistency, and absolute distance accuracy. In order to employ these metrics to evaluate and compare state-of-the-art single-image depth estimation approaches, we provide a new high-quality RGB-D dataset. We used a DSLR camera together with a laser scanner to acquire high-resolution images and highly accurate depth maps. Experimental results show the validity of our proposed evaluation protocol

A Replica Inference Approach to Unsupervised Multi-Scale Image Segmentation

We apply a replica inference based Potts model method to unsupervised image segmentation on multiple scales. This approach was inspired by the statistical mechanics problem of "community detection" and its phase diagram. Specifically, the problem is cast as identifying tightly bound clusters ("communities" or "solutes") against a background or "solvent". Within our multiresolution approach, we compute information theory based correlations among multiple solutions ("replicas") of the same graph over a range of resolutions. Significant multiresolution structures are identified by replica correlations as manifest in information theory overlaps. With the aid of these correlations as well as thermodynamic measures, the phase diagram of the corresponding Potts model is analyzed both at zero and finite temperatures. Optimal parameters corresponding to a sensible unsupervised segmentation correspond to the "easy phase" of the Potts model. Our algorithm is fast and shown to be at least as accurate as the best algorithms to date and to be especially suited to the detection of camouflaged images.Comment: 26 pages, 22 figure

Unsupervised Video Understanding by Reconciliation of Posture Similarities

Understanding human activity and being able to explain it in detail surpasses mere action classification by far in both complexity and value. The challenge is thus to describe an activity on the basis of its most fundamental constituents, the individual postures and their distinctive transitions. Supervised learning of such a fine-grained representation based on elementary poses is very tedious and does not scale. Therefore, we propose a completely unsupervised deep learning procedure based solely on video sequences, which starts from scratch without requiring pre-trained networks, predefined body models, or keypoints. A combinatorial sequence matching algorithm proposes relations between frames from subsets of the training data, while a CNN is reconciling the transitivity conflicts of the different subsets to learn a single concerted pose embedding despite changes in appearance across sequences. Without any manual annotation, the model learns a structured representation of postures and their temporal development. The model not only enables retrieval of similar postures but also temporal super-resolution. Additionally, based on a recurrent formulation, next frames can be synthesized.Comment: Accepted by ICCV 201