Dual-Resolution Correspondence Networks

We tackle the problem of establishing dense pixel-wise correspondences between a pair of images. In this work, we introduce Dual-Resolution Correspondence Networks (DRC-Net), to obtain pixel-wise correspondences in a coarse-to-fine manner. DRC-Net extracts both coarse- and fine- resolution feature maps. The coarse maps are used to produce a full but coarse 4D correlation tensor, which is then refined by a learnable neighbourhood consensus module. The fine-resolution feature maps are used to obtain the final dense correspondences guided by the refined coarse 4D correlation tensor. The selected coarse-resolution matching scores allow the fine-resolution features to focus only on a limited number of possible matches with high confidence. In this way, DRC-Net dramatically increases matching reliability and localisation accuracy, while avoiding to apply the expensive 4D convolution kernels on fine-resolution feature maps. We comprehensively evaluate our method on large-scale public benchmarks including HPatches, InLoc, and Aachen Day-Night. It achieves the state-of-the-art results on all of them

HDRFusion:HDR SLAM using a low-cost auto-exposure RGB-D sensor

We describe a new method for comparing frame appearance in a frame-to-model 3-D mapping and tracking system using an low dynamic range (LDR) RGB-D camera which is robust to brightness changes caused by auto exposure. It is based on a normalised radiance measure which is invariant to exposure changes and not only robustifies the tracking under changing lighting conditions, but also enables the following exposure compensation perform accurately to allow online building of high dynamic range (HDR) maps. The latter facilitates the frame-to-model tracking to minimise drift as well as better capturing light variation within the scene. Results from experiments with synthetic and real data demonstrate that the method provides both improved tracking and maps with far greater dynamic range of luminosity.Comment: 14 page

Correspondence Networks with Adaptive Neighbourhood Consensus

In this paper, we tackle the task of establishing dense visual correspondences between images containing objects of the same category. This is a challenging task due to large intra-class variations and a lack of dense pixel level annotations. We propose a convolutional neural network architecture, called adaptive neighbourhood consensus network (ANC-Net), that can be trained end-to-end with sparse key-point annotations, to handle this challenge. At the core of ANC-Net is our proposed non-isotropic 4D convolution kernel, which forms the building block for the adaptive neighbourhood consensus module for robust matching. We also introduce a simple and efficient multi-scale self-similarity module in ANC-Net to make the learned feature robust to intra-class variations. Furthermore, we propose a novel orthogonal loss that can enforce the one-to-one matching constraint. We thoroughly evaluate the effectiveness of our method on various benchmarks, where it substantially outperforms state-of-the-art methods.Comment: CVPR 2020. Project page: https://ancnet.avlcode.org

FlowNet3D++: Geometric Losses For Deep Scene Flow Estimation

We present FlowNet3D++, a deep scene flow estimation network. Inspired by classical methods, FlowNet3D++ incorporates geometric constraints in the form of point-to-plane distance and angular alignment between individual vectors in the flow field, into FlowNet3D. We demonstrate that the addition of these geometric loss terms improves the previous state-of-art FlowNet3D accuracy from 57.85% to 63.43%. To further demonstrate the effectiveness of our geometric constraints, we propose a benchmark for flow estimation on the task of dynamic 3D reconstruction, thus providing a more holistic and practical measure of performance than the breakdown of individual metrics previously used to evaluate scene flow. This is made possible through the contribution of a novel pipeline to integrate point-based scene flow predictions into a global dense volume. FlowNet3D++ achieves up to a 15.0% reduction in reconstruction error over FlowNet3D, and up to a 35.2% improvement over KillingFusion alone. We will release our scene flow estimation code later.Comment: Accepted in WACV 202

Merkel cell polyomavirus large T antigen disrupts lysosome clustering by translocating human Vam6p from the cytoplasm to the nucleus

Merkel cell polyomavirus (MCV) has been recently described as the cause for most human Merkel cell carcinomas. MCV is similar to simian virus 40 (SV40) and encodes a nuclear large T (LT) oncoprotein that is usually mutated to eliminate viral replication among tumor-derived MCV. We identified the hVam6p cytoplasmic protein involved in lysosomal processing as a novel interactor with MCV LT but not SV40 LT. hVam6p binds through its clathrin heavy chain homology domain to a unique region of MCV LT adjacent to the retinoblastoma binding site. MCV LT translocates hVam6p to the nucleus, sequestering it from involvement in lysosomal trafficking. A naturally occurring, tumor-derived mutant LT (MCV350) lacking a nuclear localization signal binds hVam6p but fails to inhibit hVam6p-induced lysosomal clustering. MCV has evolved a novel mechanism to target hVam6p that may contribute to viral uncoating or egress through lysosomal processing during virus replication

HDRFusion:HDR SLAM using a low-cost auto-exposure RGB-D sensor

We describe a new method for comparing frame appearance in a frame-to-model 3-D mapping and tracking system using an low dynamic range (LDR) RGB-D camera which is robust to brightness changes caused by auto exposure. It is based on a normalised radiance measure which is invariant to exposure changes and not only robustifies the tracking under changing lighting conditions, but also enables the following exposure compensation perform accurately to allow online building of high dynamic range (HDR) maps. The latter facilitates the frame-to-model tracking to minimise drift as well as better capturing light variation within the scene. Results from experiments with synthetic and real data demonstrate that the method provides both improved tracking and maps with far greater dynamic range of luminosity.Comment: 14 page

Conversion of Sox2-dependent Merkel cell carcinoma to a differentiated neuron-like phenotype by T antigen inhibition

Viral cancers show oncogene addiction to viral oncoproteins, which are required for survival and proliferation of the dedifferentiated cancer cell. Human Merkel cell carcinomas (MCCs) that harbor a clonally integrated Merkel cell polyomavirus (MCV) genome have low mutation burden and require viral T antigen expression for tumor growth. Here, we showed that MCV+ MCC cells cocultured with keratinocytes undergo neuron-like differentiation with neurite outgrowth, secretory vesicle accumulation, and the generation of sodium-dependent action potentials, hallmarks of a neuronal cell lineage. Cocultured keratinocytes are essential for induction of the neuronal phenotype. Keratinocyte-conditioned medium was insufficient to induce this phenotype. Single-cell RNA sequencing revealed that T antigen knockdown inhibited cell cycle gene expression and reduced expression of key Merkel cell lineage/MCC marker genes, including HES6, SOX2, ATOH1, and KRT20. Of these, T antigen knockdown directly inhibited Sox2 and Atoh1 expression. MCV large T up-regulated Sox2 through its retinoblastoma protein-inhibition domain, which in turn activated Atoh1 expression. The knockdown of Sox2 in MCV+ MCCs mimicked T antigen knockdown by inducing MCC cell growth arrest and neuron-like differentiation. These results show Sox2-dependent conversion of an undifferentiated, aggressive cancer cell to a differentiated neuron-like phenotype and suggest that the ontology of MCC arises from a neuronal cell precursor