Guided Depth Super-Resolution by Deep Anisotropic Diffusion

Performing super-resolution of a depth image using the guidance from an RGB image is a problem that concerns several fields, such as robotics, medical imaging, and remote sensing. While deep learning methods have achieved good results in this problem, recent work highlighted the value of combining modern methods with more formal frameworks. In this work, we propose a novel approach which combines guided anisotropic diffusion with a deep convolutional network and advances the state of the art for guided depth super-resolution. The edge transferring/enhancing properties of the diffusion are boosted by the contextual reasoning capabilities of modern networks, and a strict adjustment step guarantees perfect adherence to the source image. We achieve unprecedented results in three commonly used benchmarks for guided depth super-resolution. The performance gain compared to other methods is the largest at larger scales, such as x32 scaling. Code for the proposed method will be made available to promote reproducibility of our results

Feedback first: the surprisingly weak effects of magnetic fields, viscosity, conduction, and metal diffusion on galaxy formation

Using high-resolution simulations with explicit treatment of stellar feedback physics based on the FIRE (Feedback in Realistic Environments) project, we study how galaxy formation and the interstellar medium (ISM) are affected by magnetic fields, anisotropic Spitzer-Braginskii conduction and viscosity, and sub-grid metal diffusion from unresolved turbulence. We consider controlled simulations of isolated (non-cosmological) galaxies but also a limited set of cosmological "zoom-in" simulations. Although simulations have shown significant effects from these physics with weak or absent stellar feedback, the effects are much weaker than those of stellar feedback when the latter is modeled explicitly. The additional physics have no systematic effect on galactic star formation rates (SFRs) . In contrast, removing stellar feedback leads to SFRs being over-predicted by factors of $\sim 10 -100$. Without feedback, neither galactic winds nor volume filling hot-phase gas exist, and discs tend to runaway collapse to ultra-thin scale-heights with unphysically dense clumps congregating at the galactic center. With stellar feedback, a multi-phase, turbulent medium with galactic fountains and winds is established. At currently achievable resolutions and for the investigated halo mass range $10^{10}-10^{13} M_{\odot}$, the additional physics investigated here (MHD, conduction, viscosity, metal diffusion) have only weak ($\sim10\%$-level) effects on regulating SFR and altering the balance of phases, outflows, or the energy in ISM turbulence, consistent with simple equipartition arguments. We conclude that galactic star formation and the ISM are primarily governed by a combination of turbulence, gravitational instabilities, and feedback. We add the caveat that AGN feedback is not included in the present work

Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network

Depth estimation from a single image is a fundamental problem in computer vision. In this paper, we propose a simple yet effective convolutional spatial propagation network (CSPN) to learn the affinity matrix for depth prediction. Specifically, we adopt an efficient linear propagation model, where the propagation is performed with a manner of recurrent convolutional operation, and the affinity among neighboring pixels is learned through a deep convolutional neural network (CNN). We apply the designed CSPN to two depth estimation tasks given a single image: (1) To refine the depth output from state-of-the-art (SOTA) existing methods; and (2) to convert sparse depth samples to a dense depth map by embedding the depth samples within the propagation procedure. The second task is inspired by the availability of LIDARs that provides sparse but accurate depth measurements. We experimented the proposed CSPN over two popular benchmarks for depth estimation, i.e. NYU v2 and KITTI, where we show that our proposed approach improves in not only quality (e.g., 30% more reduction in depth error), but also speed (e.g., 2 to 5 times faster) than prior SOTA methods.Comment: 14 pages, 8 figures, ECCV 201