Hybrid Focal Stereo Networks for Pattern Analysis in Homogeneous Scenes

In this paper we address the problem of multiple camera calibration in the presence of a homogeneous scene, and without the possibility of employing calibration object based methods. The proposed solution exploits salient features present in a larger field of view, but instead of employing active vision we replace the cameras with stereo rigs featuring a long focal analysis camera, as well as a short focal registration camera. Thus, we are able to propose an accurate solution which does not require intrinsic variation models as in the case of zooming cameras. Moreover, the availability of the two views simultaneously in each rig allows for pose re-estimation between rigs as often as necessary. The algorithm has been successfully validated in an indoor setting, as well as on a difficult scene featuring a highly dense pilgrim crowd in Makkah.Comment: 13 pages, 6 figures, submitted to Machine Vision and Application

Non-parametric Models of Distortion in Imaging Systems.

Traditional radial lens distortion models are based on the physical construction of lenses. However, manufacturing defects and physical shock often cause the actual observed distortion to be different from what can be modeled by the physically motivated models. In this work, we initially propose a Gaussian process radial distortion model as an alternative to the physically motivated models. The non-parametric nature of this model helps implicitly select the right model complexity, whereas for traditional distortion models one must perform explicit model selection to decide the right parametric complexity. Next, we forego the radial distortion assumption and present a completely non-parametric, mathematically motivated distortion model based on locally-weighted homographies. The separation from an underlying physical model allows this model to capture arbitrary sources of distortion. We then apply this fully non-parametric distortion model to a zoom lens, where the distortion complexity can vary across zoom levels and the lens exhibits noticeable non-radial distortion. Through our experiments and evaluation, we show that the proposed models are as accurate as the traditional parametric models at characterizing radial distortion while flexibly capturing non-radial distortion if present in the imaging system.PhDComputer Science and EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/120690/1/rpradeep_1.pd

TennisSense: a platform for extracting semantic information from multi-camera tennis data

In this paper, we introduce TennisSense, a technology platform for the digital capture, analysis and retrieval of tennis training and matches. Our algorithms for extracting useful metadata from the overhead court camera are described and evaluated. We track the tennis ball using motion images for ball candidate detection and then link ball candidates into locally linear tracks. From these tracks we can infer when serves and rallies take place. Using background subtraction and hysteresis-type blob tracking, we track the tennis players positions. The performance of both modules is evaluated using ground-truthed data. The extracted metadata provides valuable information for indexing and efficient browsing of hours of multi-camera tennis footage and we briefly illustrative how this data is used by our tennis-coach playback interface

Multi-scale data fusion for surface metrology

The major trends in manufacturing are miniaturization, convergence of the traditional research fields and creation of interdisciplinary research areas. These trends have resulted in the development of multi-scale models and multi-scale surfaces to optimize the performance. Multi-scale surfaces that exhibit specific properties at different scales for a specific purpose require multi-scale measurement and characterization. Researchers and instrument developers have developed instruments that are able to perform measurements at multiple scales but lack the much required multi- scale characterization capability. The primary focus of this research was to explore possible multi-scale data fusion strategies and options for surface metrology domain and to develop enabling software tools in order to obtain effective multi-scale surface characterization, maximizing fidelity while minimizing measurement cost and time. This research effort explored the fusion strategies for surface metrology domain and narrowed the focus on Discrete Wavelet Frame (DWF) based multi-scale decomposition. An optimized multi-scale data fusion strategy ‘FWR method’ was developed and was successfully demonstrated on both high aspect ratio surfaces and non-planar surfaces. It was demonstrated that the datum features can be effectively characterized at a lower resolution using one system (Vision CMM) and the actual features of interest could be characterized at a higher resolution using another system (Coherence Scanning Interferometer) with higher capability while minimizing the measurement time

Neural Lens Modeling

Recent methods for 3D reconstruction and rendering increasingly benefit from end-to-end optimization of the entire image formation process. However, this approach is currently limited: effects of the optical hardware stack and in particular lenses are hard to model in a unified way. This limits the quality that can be achieved for camera calibration and the fidelity of the results of 3D reconstruction. In this paper, we propose NeuroLens, a neural lens model for distortion and vignetting that can be used for point projection and ray casting and can be optimized through both operations. This means that it can (optionally) be used to perform pre-capture calibration using classical calibration targets, and can later be used to perform calibration or refinement during 3D reconstruction, e.g., while optimizing a radiance field. To evaluate the performance of our proposed model, we create a comprehensive dataset assembled from the Lensfun database with a multitude of lenses. Using this and other real-world datasets, we show that the quality of our proposed lens model outperforms standard packages as well as recent approaches while being much easier to use and extend. The model generalizes across many lens types and is trivial to integrate into existing 3D reconstruction and rendering systems.Comment: To be presented at CVPR 2023, Project webpage: https://neural-lens.github.i

UAV-입체사진측량의 최적 촬영을 위한 카메라 세팅 및 비행방법

학위논문(석사) -- 서울대학교대학원 : 공과대학 에너지시스템공학부, 2023. 2. 송재준.Over the last decade, structure-from-motion (SfM) and multi-view-stereo (MVS) techniques have proven effective in generating high-resolution and high-accuracy 3D point clouds with the possibility to integrate with unmanned aerial vehicles (UAVs). However, the SfM-MVS techniques still had the limitation that the error of point clouds could not be predetermined before point cloud generation. In this work, a theoretical error prediction model is formulated based on the propagation of 2D image errors to 3D point cloud errors, and the disruption effect of blur and noise on 2D image errors is analyzed according to camera settings, UAV flight method, camera specification, and illumination. By comparing the error predictions with those observed in experimental data, an error prediction performance of R^2=0.83 is confirmed. Based on the high performance of error prediction, this work presents a method to determine the optimum photographing settings, including camera settings and the UAV flight method, by which the point cloud errors are minimized under illumination and time constraints. The importance of the optimum photographing settings is verified by comparing the error levels of the optimum photographing setting with those of arbitrary settings. For site validation and comparison with the light detecting and ranging (LiDAR) method, the SfM-MVS method utilizing the optimum photographing settings was applied along with LiDAR to a tunnel face located in Yeoju-si, Korea, where the light and surveying time is limited. As a result, the SfM-MVS method could achieve a point cloud with 3 times better accuracy and 20 times higher resolution at a cost of 1/9 than the LiDAR method.지난 10년간 structure from motion (SfM)과 multi view stereo (MVS) 기술이 측량 분야에서 고해상도와 고정밀도의 3차원 점군자료를 경제적으로 생성할 수 있고 UAV와의 결합 능력을 보여주었음에도 불구하고 아직까지 오차수준을 생성 전에 결정할 수 없다는 문제가 있다. 본 연구에선 블러와 잡음에 의한 이미지 오차의 3차원 점군자료로의 전파를 기반으로 이론적 오차예측 모델을 구성하였고, 카메라 세팅, UAV 비행방법, 카메라 사양 그리고 조도에 따른 블러와 잡음의 크기를 분석하였다. 실험을 통해 예측한 오차와 관측한 오차를 비교하였고 R^2=0.83 라는 우수한 예측 성능을 확인하였다. 본 연구는 높은 오차 예측 성능을 기반으로 주어진 조도 및 시간 제약 조건에서 점군자료 오차를 최소화하는 카메라 세팅 및 UAV 비행 방법을 포함한 최적 촬영조건을 도출하는 방법을 제시하였다. 최적 촬영조건과 임의 촬영조건 간 오차 수준을 비교하여 카메라 세팅과 UAV 비행방법을 조정하는 것만으로도 월등히 높은 품질의 점군자료를 획득할 수 있는 것을 확인함으로써 최적 촬영조건의 중요성을 검증하였다. 본 연구에서 광량과 시간이 제한적인 여주시에 위치한 지하 터널 막장면을 대상으로 최적 촬영조건을 이용하는 SfM-MVS 기술과 light detecting and ranging (LiDAR) 기술을 비교하였다. 그 결과 SfM-MVS 기술을 이용하면 LiDAR 기술보다 3배 정확하고 20배 고해상도의 점군자료를 9배 더 경제적으로 획득할 수 있음을 확인했다.Chapter 1. Introduction 1 Chapter 2. Theory and methodology 6 2.1. Theoretical model for error prediction 6 2.1.1. Error propagation from 2D image to 3D point cloud 7 2.1.2. Image quality factors 11 2.1.3. Effects of parameters on image disruption 14 2.1.4. Methodology for theoretical model validation 26 2.2. Derivation of optimum photographing settings 32 2.2.1. Constraints: illumination, time constraints 32 2.2.2. Optimum photographing settings derivation 36 2.2.3. Field application of the derivation procedure 38 Chapter 3. Validation and comparison 41 3.1. Validation of the theoretical model 41 3.1.1. M value calibration 41 3.1.2. Q value calibration 43 3.1.3. Validation result 45 3.2. Optimum photographing settings application 48 3.2.1. Importance of optimum photographing settings 48 3.2.2. Comparison with the LiDAR 49 Chapter 4. Discussion 55 4.1. Implication of the error prediction model 55 4.2. Feasibility of the derivation 60 Chapter 5. Conclusion 64 References 65 Abstract in Korean 72석

Model-Based Environmental Visual Perception for Humanoid Robots

The visual perception of a robot should answer two fundamental questions: What? and Where? In order to properly and efficiently reply to these questions, it is essential to establish a bidirectional coupling between the external stimuli and the internal representations. This coupling links the physical world with the inner abstraction models by sensor transformation, recognition, matching and optimization algorithms. The objective of this PhD is to establish this sensor-model coupling

Particle Swarm Optimisation in Practice: Multiple Applications in a Digital Microscope System

We demonstrate that particle swarm optimisation (PSO) can be used to solve a variety of problems arising during operation of a digital inspection microscope. This is a use case for the feasibility of heuristics in a real-world product. We show solutions to four measurement problems, all based on PSO. This allows for a compact software implementation solving different problems. We have found that PSO can solve a variety of problems with small software footprints and good results in a real-world embedded system. Notably, in the microscope application, this eliminates the need to return the device to the factory for calibration