Web-based Campus Virtual Tour Application using ORB Image Stitching

Information disclosure in the digital age has demanded the public to obtain information easily and meaningful. In this paper, we propose the development of web-based campus virtual tour 360-degree information system application at the State University of Malang, Indonesia which aims to introduce the assets of the institution in an interesting view to public. This application receives a stitched or panoramic image generated through the ORB image stitching algorithm as an input and displays it in virtual tour manner. This paper realizes the image stitching algorithm to present the visualization of the 360-degree dynamic building and campus environment, so it looks real as if it were in the actual location. Virtual tour approach can produce a more immersive and attractive appearance than regular photos

Learning Lens Blur Fields

Optical blur is an inherent property of any lens system and is challenging to model in modern cameras because of their complex optical elements. To tackle this challenge, we introduce a high-dimensional neural representation of blur$-$$\textit{the lens blur field}$$-$and a practical method for acquiring it. The lens blur field is a multilayer perceptron (MLP) designed to (1) accurately capture variations of the lens 2D point spread function over image plane location, focus setting and, optionally, depth and (2) represent these variations parametrically as a single, sensor-specific function. The representation models the combined effects of defocus, diffraction, aberration, and accounts for sensor features such as pixel color filters and pixel-specific micro-lenses. To learn the real-world blur field of a given device, we formulate a generalized non-blind deconvolution problem that directly optimizes the MLP weights using a small set of focal stacks as the only input. We also provide a first-of-its-kind dataset of 5D blur fields$-$for smartphone cameras, camera bodies equipped with a variety of lenses, etc. Lastly, we show that acquired 5D blur fields are expressive and accurate enough to reveal, for the first time, differences in optical behavior of smartphone devices of the same make and model

듀얼 픽셀 이미지 기반 제로샷 디포커스 디블러링

학위논문(석사) -- 서울대학교대학원 : 공과대학 협동과정 인공지능전공, 2022. 8. 한보형.Defocus deblurring in dual-pixel (DP) images is a challenging problem due to diverse camera optics and scene structures. Most of the existing algorithms rely on supervised learning approaches trained on the Canon DSLR dataset but often suffer from weak generalizability to out-of-distribution images including the ones captured by smartphones. We propose a novel zero-shot defocus deblurring algorithm, which only requires a pair of DP images without any training data and a pre-calibrated ground-truth blur kernel. Specifically, our approach first initializes a sharp latent map using a parametric blur kernel with a symmetry constraint. It then uses a convolutional neural network (CNN) to estimate the defocus map that best describes the observed DP image. Finally, it employs a generative model to learn scene-specific non-uniform blur kernels to compute the final enhanced images. We demonstrate that the proposed unsupervised technique outperforms the counterparts based on supervised learning when training and testing run in different datasets. We also present that our model achieves competitive accuracy when tested on in-distribution data.듀얼 픽셀(DP) 이미지 센서를 사용하는 스마트폰에서의 Defocus Blur 현상은 다양한 카메라 광학 구조와 물체의 깊이 마다 다른 흐릿함 정도로 인해 원 영상 복원이 쉽지 않습니다. 기존 알고리즘들은 모두 Canon DSLR 데이터에서 훈련된 지도 학습 접근 방식에 의존하여 스마트폰으로 촬영된 사진에서는 잘 일반화가 되지 않습니다. 본 논문에서는 훈련 데이터와 사전 보정된 실제 Blur 커널 없이도, 한 쌍의 DP 사진만으로도 학습이 가능한 Zero-shot Defocus Deblurring 알고리즘을 제안합니다. 특히, 본 논문에서는 대칭적으로 모델링 된 Blur Kernel을 사용하여 초기 영상을 복원하며, 이후 CNN(Convolutional Neural Network)을 사용하여 관찰된 DP 이미지를 가장 잘 설명하는 Defocus Map을 추정합니다. 마지막으로 CNN을 사용하여 장면 별 Non-uniform한 Blur Kernel을 학습하여 최종 복원 영상의 성능을 개선합니다. 학습과 추론이 다른 데이터 세트에서 실행될 때, 제안된 방법은 비지도 기술 임에도 불구하고 최근에 발표된 지도 학습을 기반의 방법들보다 우수한 성능을 보여줍니다. 또한 학습 된 것과 같은 분포 내 데이터에서 추론할 때도 지도 학습 기반의 방법들과 정량적 또는 정성적으로 비슷한 성능을 보이는 것을 확인할 수 있었습니다.1. Introduction 6 1.1. Background 6 1.2. Overview 9 1.3. Contribution 11 2. Related Works 12 2.1.Defocus Deblurring 12 2.2.Defocus Map 13 2.3.Multiplane Image Representation 14 2.4.DP Blur Kernel 14 3. Proposed Methods 16 3.1. Latent Map Initialization 17 3.2. Defocus Map Estimation 20 3.3. Learning Blur Kernel s 22 3.4. Implementation Details 25 4. Experiments 28 4.1. Dataset 28 4.2. Quantitative Results 29 4.3. Qualitative Results 31 5. Conclusions 37 5.1.Summary 37 5.2. Discussion 38석

Architectural Digital Photogrammetry

This study is to exploit texturing techniques of a common modelling software in the way of creating virtual models of an exist architectures using oriented panoramas. In this research, The panoramic image-based interactive modelling is introduced as assembly point of photography, topography, photogrammetry and modelling techniques. It is an interactive system for generating photorealistic, textured 3D models of architectural structures and urban scenes. The technique is suitable for the architectural survey because it is not a «point by point» survey, and it exploit the geometrical constraints in the architecture to simplify modelling. Many factors are presented to be critical features that affect the modelling quality and accuracy, such as the way and the position in shooting the photos, stitching the multi-image panorama photos, the orientation, texturing techniques and so on. During the last few years, many Image-based modelling programmes have been released. Whereas, in this research, the photo modelling programs was not in use, it meant to face the fundamentals of the photogrammetry and to go beyond the limitations of such software by avoiding the automatism. In addition, it meant to exploit the potent commands of a program as 3DsMax to obtain the final representation of the Architecture. Such representation can be used in different fields (from detailed architectural survey to an architectural representation in cinema and video games), considering the accuracy and the quality which they are vary too. After the theoretical studies of this technique, it was applied in four applications to different types of close range surveys. This practice allowed to comprehend the practical problems in the whole process (from photographing all the way to modelling) and to propose the methods in the ways to improve it and to avoid any complications. It was compared with the laser scanning to study the accuracy of this technique. Thus, it is realized that not only the accuracy of this technique is linked to the size of the surveyed object, but also the size changes the way in which the survey to be approached. Since the 3D modelling program is not dedicated to be used for the image-based modelling, texturing problems was faced. It was analyzed in: how the program can behave with the Bitmap, how to project it, how it could be an interactive projection, and what are the limitations

Computational Imaging Approach to Recovery of Target Coordinates Using Orbital Sensor Data

This dissertation addresses the components necessary for simulation of an image-based recovery of the position of a target using orbital image sensors. Each component is considered in detail, focusing on the effect that design choices and system parameters have on the accuracy of the position estimate. Changes in sensor resolution, varying amounts of blur, differences in image noise level, selection of algorithms used for each component, and lag introduced by excessive processing time all contribute to the accuracy of the result regarding recovery of target coordinates using orbital sensor data. Using physical targets and sensors in this scenario would be cost-prohibitive in the exploratory setting posed, therefore a simulated target path is generated using Bezier curves which approximate representative paths followed by the targets of interest. Orbital trajectories for the sensors are designed on an elliptical model representative of the motion of physical orbital sensors. Images from each sensor are simulated based on the position and orientation of the sensor, the position of the target, and the imaging parameters selected for the experiment (resolution, noise level, blur level, etc.). Post-processing of the simulated imagery seeks to reduce noise and blur and increase resolution. The only information available for calculating the target position by a fully implemented system are the sensor position and orientation vectors and the images from each sensor. From these data we develop a reliable method of recovering the target position and analyze the impact on near-realtime processing. We also discuss the influence of adjustments to system components on overall capabilities and address the potential system size, weight, and power requirements from realistic implementation approaches

Wide-Field InfrarRed Survey Telescope-Astrophysics Focused Telescope Assets WFIRST-AFTA 2015 Report

This report describes the 2014 study by the Science Definition Team (SDT) of the Wide-Field Infrared Survey Telescope (WFIRST) mission. It is a space observatory that will address the most compelling scientific problems in dark energy, exoplanets and general astrophysics using a 2.4-m telescope with a wide-field infrared instrument and an optical coronagraph. The Astro2010 Decadal Survey recommended a Wide Field Infrared Survey Telescope as its top priority for a new large space mission. As conceived by the decadal survey, WFIRST would carry out a dark energy science program, a microlensing program to determine the demographics of exoplanets, and a general observing program utilizing its ultra wide field. In October 2012, NASA chartered a Science Definition Team (SDT) to produce, in collaboration with the WFIRST Study Office at GSFC and the Program Office at JPL, a Design Reference Mission (DRM) for an implementation of WFIRST using one of the 2.4-m, Hubble-quality telescope assemblies recently made available to NASA. This DRM builds on the work of the earlier WFIRST SDT, reported by Green et al. (2012) and the previous WFIRST-2.4 DRM, reported by Spergel et. (2013). The 2.4-m primary mirror enables a mission with greater sensitivity and higher angular resolution than the 1.3-m and 1.1-m designs considered previously, increasing both the science return of the primary surveys and the capabilities of WFIRST as a Guest Observer facility. The addition of an on-axis coronagraphic instrument to the baseline design enables imaging and spectroscopic studies of planets around nearby stars.Comment: This report describes the 2014 study by the Science Definition Team of the Wide-Field Infrared Survey Telescope mission. 319 pages; corrected a misspelled name in the authors list and a typo in the abstrac

Bagadus: next generation sport analysis and multimedia platform using camera array and sensor networks

Today, a large number of (elite) sports clubs spend a large amount of resources to analyze their game performance, either manually or using one of the many existing analytics tools. In the area of soccer, there exist several systems where trainers and coaches can analyze the game play in order to improve the performance. However, most of these systems are cumbersome and relies on manual work from many people and/or heavy video processing. In this thesis, we present Bagadus, a prototype of a soccer analysis application which integrates a sensor system, soccer analytics annotations and video processing of a video camera array. The prototype is currently installed at Alfheim Stadium in Norway, and we demonstrate how the system can follow and zoom in on particular player(s), and search for and playout events from the games using the stitched panorama video and/or the camera switching mode

Real-Time Computational Gigapixel Multi-Camera Systems

The standard cameras are designed to truthfully mimic the human eye and the visual system. In recent years, commercially available cameras are becoming more complex, and offer higher image resolutions than ever before. However, the quality of conventional imaging methods is limited by several parameters, such as the pixel size, lens system, the diffraction limit, etc. The rapid technological advancements, increase in the available computing power, and introduction of Graphics Processing Units (GPU) and Field-Programmable-Gate-Arrays (FPGA) open new possibilities in the computer vision and computer graphics communities. The researchers are now focusing on utilizing the immense computational power offered on the modern processing platforms, to create imaging systems with novel or significantly enhanced capabilities compared to the standard ones. One popular type of the computational imaging systems offering new possibilities is a multi-camera system. This thesis will focus on FPGA-based multi-camera systems that operate in real-time. The aim of themulti-camera systems presented in this thesis is to offer a wide field-of-view (FOV) video coverage at high frame rates. The wide FOV is achieved by constructing a panoramic image from the images acquired by the multi-camera system. Two new real-time computational imaging systems that provide new functionalities and better performance compared to conventional cameras are presented in this thesis. Each camera system design and implementation are analyzed in detail, built and tested in real-time conditions. Panoptic is a miniaturized low-cost multi-camera system that reconstructs a 360 degrees view in real-time. Since it is an easily portable system, it provides means to capture the complete surrounding light field in dynamic environment, such as when mounted on a vehicle or a flying drone. The second presented system, GigaEye II , is a modular high-resolution imaging system that introduces the concept of distributed image processing in the real-time camera systems. This thesis explains in detail howsuch concept can be efficiently used in real-time computational imaging systems. The purpose of computational imaging systems in the form of multi-camera systems does not end with real-time panoramas. The application scope of these cameras is vast. They can be used in 3D cinematography, for broadcasting live events, or for immersive telepresence experience. The final chapter of this thesis presents three potential applications of these systems: object detection and tracking, high dynamic range (HDR) imaging, and observation of multiple regions of interest. Object detection and tracking, and observation of multiple regions of interest are extremely useful and desired capabilities of surveillance systems, in security and defense industry, or in the fast-growing industry of autonomous vehicles. On the other hand, high dynamic range imaging is becoming a common option in the consumer market cameras, and the presented method allows instantaneous capture of HDR videos. Finally, this thesis concludes with the discussion of the real-time multi-camera systems, their advantages, their limitations, and the future predictions