Automated structure discovery in atomic force microscopy

Atomic force microscopy (AFM) with molecule-functionalized tips has emerged as the primary experimental technique for probing the atomic structure of organic molecules on surfaces. Most experiments have been limited to nearly planar aromatic molecules due to difficulties with interpretation of highly distorted AFM images originating from nonplanar molecules. Here, we develop a deep learning infrastructure that matches a set of AFM images with a unique descriptor characterizing the molecular configuration, allowing us to predict the molecular structure directly. We apply this methodology to resolve several distinct adsorption configurations of 1S-camphor on Cu(111) based on low-temperature AFM measurements. This approach will open the door to applying high-resolution AFM to a large variety of systems, for which routine atomic and chemical structural resolution on the level of individual objects/molecules would be a major breakthrough

Infrastructure-based Multi-Camera Calibration using Radial Projections

Multi-camera systems are an important sensor platform for intelligent systems such as self-driving cars. Pattern-based calibration techniques can be used to calibrate the intrinsics of the cameras individually. However, extrinsic calibration of systems with little to no visual overlap between the cameras is a challenge. Given the camera intrinsics, infrastucture-based calibration techniques are able to estimate the extrinsics using 3D maps pre-built via SLAM or Structure-from-Motion. In this paper, we propose to fully calibrate a multi-camera system from scratch using an infrastructure-based approach. Assuming that the distortion is mainly radial, we introduce a two-stage approach. We first estimate the camera-rig extrinsics up to a single unknown translation component per camera. Next, we solve for both the intrinsic parameters and the missing translation components. Extensive experiments on multiple indoor and outdoor scenes with multiple multi-camera systems show that our calibration method achieves high accuracy and robustness. In particular, our approach is more robust than the naive approach of first estimating intrinsic parameters and pose per camera before refining the extrinsic parameters of the system. The implementation is available at https://github.com/youkely/InfrasCal.Comment: ECCV 202

Models and methods for geometric computer vision

Abstract Automatic three-dimensional scene reconstruction from multiple images is a central problem in geometric computer vision. This thesis considers topics that are related to this problem area. New models and methods are presented for various tasks in such specific domains as camera calibration, image-based modeling and image matching. In particular, the main themes of the thesis are geometric camera calibration and quasi-dense image matching. In addition, a topic related to the estimation of two-view geometric relations is studied, namely, the computation of a planar homography from corresponding conics. Further, as an example of a reconstruction system, a structure-from-motion approach is presented for modeling sewer pipes from video sequences. In geometric camera calibration, the thesis concentrates on central cameras. A generic camera model and a plane-based camera calibration method are presented. The experiments with various real cameras show that the proposed calibration approach is applicable for conventional perspective cameras as well as for many omnidirectional cameras, such as fish-eye lens cameras. In addition, a method is presented for the self-calibration of radially symmetric central cameras from two-view point correspondences. In image matching, the thesis proposes a method for obtaining quasi-dense pixel matches between two wide baseline images. The method extends the match propagation algorithm to the wide baseline setting by using an affine model for the local geometric transformations between the images. Further, two adaptive propagation strategies are presented, where local texture properties are used for adjusting the local transformation estimates during the propagation. These extensions make the quasi-dense approach applicable for both rigid and non-rigid wide baseline matching. In this thesis, quasi-dense matching is additionally applied for piecewise image registration problems which are encountered in specific object recognition and motion segmentation. The proposed object recognition approach is based on grouping the quasi-dense matches between the model and test images into geometrically consistent groups, which are supposed to represent individual objects, whereafter the number and quality of grouped matches are used as recognition criteria. Finally, the proposed approach for dense two-view motion segmentation is built on a layer-based segmentation framework which utilizes grouped quasi-dense matches for initializing the motion layers, and is applicable under wide baseline conditions

Robust and practical depth map fusion for time-of-flight cameras

Abstract Fusion of overlapping depth maps is an important part in many 3D reconstruction pipelines. Ideally fusion produces an accurate and nonredundant point cloud robustly even from noisy and partially poorly registered depth maps. In this paper, we improve an existing fusion algorithm towards a more ideal solution. Our method builds a nonredundant point cloud from a sequence of depth maps so that the new measurements are either added to the existing point cloud if they are in an area which is not yet covered or used to refine the existing points. The method is robust to outliers and erroneous depth measurements as well as small depth map registration errors due to inaccurate camera poses. The results show that the method overcomes its predecessor both in accuracy and robustness

Deep learning for magnification independent breast cancer histopathology image classification

Abstract Microscopic analysis of breast tissues is necessary for a definitive diagnosis of breast cancer which is the most common cancer among women. Pathology examination requires time consuming scanning through tissue images under different magnification levels to find clinical assessment clues to produce correct diagnoses. Advances in digital imaging techniques offers assessment of pathology images using computer vision and machine learning methods which could automate some of the tasks in the diagnostic pathology workflow. Such automation could be beneficial to obtain fast and precise quantification, reduce observer variability, and increase objectivity. In this work, we propose to classify breast cancer histopathology images independent of their magnifications using convolutional neural networks (CNNs). We propose two different architectures; single task CNN is used to predict malignancy and multi-task CNN is used to predict both malignancy and image magnification level simultaneously. Evaluations and comparisons with previous results are carried out on BreaKHis dataset. Experimental results show that our magnification independent CNN approach improved the performance of magnification specific model. Our results in this limited set of training data are comparable with previous state-of-the-art results obtained by hand-crafted features. However, unlike previous methods, our approach has potential to directly benefit from additional training data, and such additional data could be captured with same or different magnification levels than previous data