Depth Super-Resolution Meets Uncalibrated Photometric Stereo

A novel depth super-resolution approach for RGB-D sensors is presented. It disambiguates depth super-resolution through high-resolution photometric clues and, symmetrically, it disambiguates uncalibrated photometric stereo through low-resolution depth cues. To this end, an RGB-D sequence is acquired from the same viewing angle, while illuminating the scene from various uncalibrated directions. This sequence is handled by a variational framework which fits high-resolution shape and reflectance, as well as lighting, to both the low-resolution depth measurements and the high-resolution RGB ones. The key novelty consists in a new PDE-based photometric stereo regularizer which implicitly ensures surface regularity. This allows to carry out depth super-resolution in a purely data-driven manner, without the need for any ad-hoc prior or material calibration. Real-world experiments are carried out using an out-of-the-box RGB-D sensor and a hand-held LED light source.Comment: International Conference on Computer Vision (ICCV) Workshop, 201

Photometric Depth Super-Resolution

This study explores the use of photometric techniques (shape-from-shading and uncalibrated photometric stereo) for upsampling the low-resolution depth map from an RGB-D sensor to the higher resolution of the companion RGB image. A single-shot variational approach is first put forward, which is effective as long as the target's reflectance is piecewise-constant. It is then shown that this dependency upon a specific reflectance model can be relaxed by focusing on a specific class of objects (e.g., faces), and delegate reflectance estimation to a deep neural network. A multi-shot strategy based on randomly varying lighting conditions is eventually discussed. It requires no training or prior on the reflectance, yet this comes at the price of a dedicated acquisition setup. Both quantitative and qualitative evaluations illustrate the effectiveness of the proposed methods on synthetic and real-world scenarios.Comment: IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2019. First three authors contribute equall

Dense semantic labeling of sub-decimeter resolution images with convolutional neural networks

Semantic labeling (or pixel-level land-cover classification) in ultra-high resolution imagery (< 10cm) requires statistical models able to learn high level concepts from spatial data, with large appearance variations. Convolutional Neural Networks (CNNs) achieve this goal by learning discriminatively a hierarchy of representations of increasing abstraction. In this paper we present a CNN-based system relying on an downsample-then-upsample architecture. Specifically, it first learns a rough spatial map of high-level representations by means of convolutions and then learns to upsample them back to the original resolution by deconvolutions. By doing so, the CNN learns to densely label every pixel at the original resolution of the image. This results in many advantages, including i) state-of-the-art numerical accuracy, ii) improved geometric accuracy of predictions and iii) high efficiency at inference time. We test the proposed system on the Vaihingen and Potsdam sub-decimeter resolution datasets, involving semantic labeling of aerial images of 9cm and 5cm resolution, respectively. These datasets are composed by many large and fully annotated tiles allowing an unbiased evaluation of models making use of spatial information. We do so by comparing two standard CNN architectures to the proposed one: standard patch classification, prediction of local label patches by employing only convolutions and full patch labeling by employing deconvolutions. All the systems compare favorably or outperform a state-of-the-art baseline relying on superpixels and powerful appearance descriptors. The proposed full patch labeling CNN outperforms these models by a large margin, also showing a very appealing inference time.Comment: Accepted in IEEE Transactions on Geoscience and Remote Sensing, 201

Measuring Black Hole Spin using X-ray Reflection Spectroscopy

I review the current status of X-ray reflection (a.k.a. broad iron line) based black hole spin measurements. This is a powerful technique that allows us to measure robust black hole spins across the mass range, from the stellar-mass black holes in X-ray binaries to the supermassive black holes in active galactic nuclei. After describing the basic assumptions of this approach, I lay out the detailed methodology focusing on "best practices" that have been found necessary to obtain robust results. Reflecting my own biases, this review is slanted towards a discussion of supermassive black hole (SMBH) spin in active galactic nuclei (AGN). Pulling together all of the available XMM-Newton and Suzaku results from the literature that satisfy objective quality control criteria, it is clear that a large fraction of SMBHs are rapidly-spinning, although there are tentative hints of a more slowly spinning population at high (M>5*10^7Msun) and low (M<2*10^6Msun) mass. I also engage in a brief review of the spins of stellar-mass black holes in X-ray binaries. In general, reflection-based and continuum-fitting based spin measures are in agreement, although there remain two objects (GROJ1655-40 and 4U1543-475) for which that is not true. I end this review by discussing the exciting frontier of relativistic reverberation, particularly the discovery of broad iron line reverberation in XMM-Newton data for the Seyfert galaxies NGC4151, NGC7314 and MCG-5-23-16. As well as confirming the basic paradigm of relativistic disk reflection, this detection of reverberation demonstrates that future large-area X-ray observatories such as LOFT will make tremendous progress in studies of strong gravity using relativistic reverberation in AGN.Comment: 19 pages. To appear in proceedings of the ISSI-Bern workshop on "The Physics of Accretion onto Black Holes" (8-12 Oct 2012). Revised version adds a missing source to Table 1 and Fig.6 (IRAS13224-3809) and corrects the referencing of the discovery of soft lags in 1H0707-495 (which were in fact first reported in Fabian et al. 2009

A Stellar Dynamical Measurement of the Black Hole Mass in the Maser Galaxy NGC 4258

We determine the mass of the black hole at the center of the spiral galaxy NGC 4258 by constructing axisymmetric dynamical models of the galaxy. These models are constrained by high spatial resolution imaging and long-slit spectroscopy of the nuclear region obtained with the {\em Hubble Space Telescope}, complemented by ground-based observations extending to larger radii. Our best mass estimate is \MBH = (3.3 \pm 0.2) \times 10^7 \MSun for a distance of 7.28 Mpc (statistical errors only). This is within 15% of (3.82\pm 0.01) \times 10^7 \MSun, the mass determined from the kinematics of water masers (rescaled to the same distance) assuming they are in Keplerian rotation in a warped disk. The construction of accurate dynamical models of NGC 4258 is somewhat compromised by an unresolved active nucleus and color gradients, the latter caused by variations in the stellar population and/or obscuring dust. These problems are not present in the $\sim 30$ other black hole mass determinations from stellar dynamics that have been published by us and other groups; thus, the relatively close agreement between the stellar dynamical mass and the maser mass in NGC 4258 enhances our confidence in the black hole masses determined in other galaxies from stellar dynamics using similar methods and data of comparable quality.Comment: 58 pages, submitted to ApJ. Some figures excluded due to size. The entire paper is at http://www.noao.edu/noao/staff/lauer/nuker_papers.htm