Probabilistic Human Mesh Recovery in 3D Scenes from Egocentric Views

Automatic perception of human behaviors during social interactions is crucial for AR/VR applications, and an essential component is estimation of plausible 3D human pose and shape of our social partners from the egocentric view. One of the biggest challenges of this task is severe body truncation due to close social distances in egocentric scenarios, which brings large pose ambiguities for unseen body parts. To tackle this challenge, we propose a novel scene-conditioned diffusion method to model the body pose distribution. Conditioned on the 3D scene geometry, the diffusion model generates bodies in plausible human-scene interactions, with the sampling guided by a physics-based collision score to further resolve human-scene inter-penetrations. The classifier-free training enables flexible sampling with different conditions and enhanced diversity. A visibility-aware graph convolution model guided by per-joint visibility serves as the diffusion denoiser to incorporate inter-joint dependencies and per-body-part control. Extensive evaluations show that our method generates bodies in plausible interactions with 3D scenes, achieving both superior accuracy for visible joints and diversity for invisible body parts. The code is available at https://sanweiliti.github.io/egohmr/egohmr.html.Comment: Camera ready version for ICCV 2023, appendix include

Hybrid Scene Compression for Visual Localization

Localizing an image wrt. a 3D scene model represents a core task for many computer vision applications. An increasing number of real-world applications of visual localization on mobile devices, e.g., Augmented Reality or autonomous robots such as drones or self-driving cars, demand localization approaches to minimize storage and bandwidth requirements. Compressing the 3D models used for localization thus becomes a practical necessity. In this work, we introduce a new hybrid compression algorithm that uses a given memory limit in a more effective way. Rather than treating all 3D points equally, it represents a small set of points with full appearance information and an additional, larger set of points with compressed information. This enables our approach to obtain a more complete scene representation without increasing the memory requirements, leading to a superior performance compared to previous compression schemes. As part of our contribution, we show how to handle ambiguous matches arising from point compression during RANSAC. Besides outperforming previous compression techniques in terms of pose accuracy under the same memory constraints, our compression scheme itself is also more efficient. Furthermore, the localization rates and accuracy obtained with our approach are comparable to state-of-the-art feature-based methods, while using a small fraction of the memory.Comment: Published at CVPR 201

Starlight Demonstration of the Dragonfly Instrument: an Integrated Photonic Pupil Remapping Interferometer for High Contrast Imaging

In the two decades since the first extra-solar planet was discovered, the detection and characterization of extra-solar planets has become one of the key endeavors in all of modern science. Recently direct detection techniques such as interferometry or coronography have received growing attention because they reveal the population of exoplanets inaccessible to Doppler or transit techniques, and moreover they allow the faint signal from the planet itself to be investigated. Next-generation stellar interferometers are increasingly incorporating photonic technologies due to the increase in fidelity of the data generated. Here, we report the design, construction and commissioning of a new high contrast imager; the integrated pupil-remapping interferometer; an instrument we expect will find application in the detection of young faint companions in the nearest star-forming regions. The laboratory characterisation of the instrument demonstrated high visibility fringes on all interferometer baselines in addition to stable closure phase signals. We also report the first successful on-sky experiments with the prototype instrument at the 3.9-m Anglo-Australian Telescope. Performance metrics recovered were consistent with ideal device behaviour after accounting for expected levels of decoherence and signal loss from the uncompensated seeing. The prospect of complete Fourier-coverage coupled with the current performance metrics means that this photonically-enhanced instrument is well positioned to contribute to the science of high contrast companions.Comment: 10 pages, 7 figures, accepted to Mon. Not. of Roy. Ast. Soc., 201

Sparsity Invariant CNNs

In this paper, we consider convolutional neural networks operating on sparse inputs with an application to depth upsampling from sparse laser scan data. First, we show that traditional convolutional networks perform poorly when applied to sparse data even when the location of missing data is provided to the network. To overcome this problem, we propose a simple yet effective sparse convolution layer which explicitly considers the location of missing data during the convolution operation. We demonstrate the benefits of the proposed network architecture in synthetic and real experiments with respect to various baseline approaches. Compared to dense baselines, the proposed sparse convolution network generalizes well to novel datasets and is invariant to the level of sparsity in the data. For our evaluation, we derive a novel dataset from the KITTI benchmark, comprising 93k depth annotated RGB images. Our dataset allows for training and evaluating depth upsampling and depth prediction techniques in challenging real-world settings and will be made available upon publication

Integrated AlGaAs source of highly indistinguishable and energy-time entangled photons

The generation of nonclassical states of light in miniature chips is a crucial step towards practical implementations of future quantum technologies. Semiconductor materials are ideal to achieve extremely compact and massively parallel systems and several platforms are currently under development. In this context, spontaneous parametric down conversion in AlGaAs devices combines the advantages of room temperature operation, possibility of electrical injection and emission in the telecom band. Here we report on a chip-based AlGaAs source, producing indistinguishable and energy-time entangled photons with a brightness of $7.2\times10^6$ pairs/s and a signal-to-noise ratio of $141\pm12$. Indistinguishability between the photons is demonstrated via a Hong-Ou-Mandel experiment with a visibility of $89\pm3\%$, while energy-time entanglement is tested via a Franson interferometer leading to a value for the Bell parameter $S=2.70\pm0.10$