직접 볼륨 렌더링에서 점진적 렌즈 샘플링을 사용한 피사계 심도 렌더링

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·컴퓨터공학부, 2021. 2. 신영길.Direct volume rendering is a widely used technique for extracting information from 3D scalar fields acquired by measurement or numerical simulation. To visualize the structure inside the volume, the voxels scalar value is often represented by a translucent color. This translucency of direct volume rendering makes it difficult to perceive the depth between the nested structures. Various volume rendering techniques to improve depth perception are mainly based on illustrative rendering techniques, and physically based rendering techniques such as depth of field effects are difficult to apply due to long computation time. With the development of immersive systems such as virtual and augmented reality and the growing interest in perceptually motivated medical visualization, it is necessary to implement depth of field in direct volume rendering. This study proposes a novel method for applying depth of field effects to volume ray casting to improve the depth perception. By performing ray casting using multiple rays per pixel, objects at a distance in focus are sharply rendered and objects at an out-of-focus distance are blurred. To achieve these effects, a thin lens camera model is used to simulate rays passing through different parts of the lens. And an effective lens sampling method is used to generate an aliasing-free image with a minimum number of lens samples that directly affect performance. The proposed method is implemented without preprocessing based on the GPU-based volume ray casting pipeline. Therefore, all acceleration techniques of volume ray casting can be applied without restrictions. We also propose multi-pass rendering using progressive lens sampling as an acceleration technique. More lens samples are progressively used for ray generation over multiple render passes. Each pixel has a different final render pass depending on the predicted maximum blurring size based on the circle of confusion. This technique makes it possible to apply a different number of lens samples for each pixel, depending on the degree of blurring of the depth of field effects over distance. This acceleration method reduces unnecessary lens sampling and increases the cache hit rate of the GPU, allowing us to generate the depth of field effects at interactive frame rates in direct volume rendering. In the experiments using various data, the proposed method generated realistic depth of field effects in real time. These results demonstrate that our method produces depth of field effects with similar quality to the offline image synthesis method and is up to 12 times faster than the existing depth of field method in direct volume rendering.직접 볼륨 렌더링(direct volume rendering, DVR)은 측정 또는 수치 시뮬레이션으로 얻은 3차원 공간의 스칼라 필드(3D scalar fields) 데이터에서 정보를 추출하는데 널리 사용되는 기술이다. 볼륨 내부의 구조를 가시화하기 위해 복셀(voxel)의 스칼라 값은 종종 반투명의 색상으로 표현된다. 이러한 직접 볼륨 렌더링의 반투명성은 중첩된 구조 간 깊이 인식을 어렵게 한다. 깊이 인식을 향상시키기 위한 다양한 볼륨 렌더링 기법들은 주로 삽화풍 렌더링(illustrative rendering)을 기반으로 하며, 피사계 심도(depth of field, DoF) 효과와 같은 물리 기반 렌더링(physically based rendering) 기법들은 계산 시간이 오래 걸리기 때문에 적용이 어렵다. 가상 및 증강 현실과 같은 몰입형 시스템의 발전과 인간의 지각에 기반한 의료영상 시각화에 대한 관심이 증가함에 따라 직접 볼륨 렌더링에서 피사계 심도를 구현할 필요가 있다. 본 논문에서는 직접 볼륨 렌더링의 깊이 인식을 향상시키기 위해 볼륨 광선투사법에 피사계 심도 효과를 적용하는 새로운 방법을 제안한다. 픽셀 당 여러 개의 광선을 사용한 광선투사법(ray casting)을 수행하여 초점이 맞는 거리에 있는 물체는 선명하게 표현되고 초점이 맞지 않는 거리에 있는 물체는 흐리게 표현된다. 이러한 효과를 얻기 위하여 렌즈의 서로 다른 부분을 통과하는 광선들을 시뮬레이션 하는 얇은 렌즈 카메라 모델(thin lens camera model)이 사용되었다. 그리고 성능에 직접적으로 영향을 끼치는 렌즈 샘플은 최적의 렌즈 샘플링 방법을 사용하여 최소한의 개수를 가지고 앨리어싱(aliasing)이 없는 이미지를 생성하였다. 제안한 방법은 기존의 GPU 기반 볼륨 광선투사법 파이프라인 내에서 전처리 없이 구현된다. 따라서 볼륨 광선투사법의 모든 가속화 기법을 제한없이 적용할 수 있다. 또한 가속 기술로 누진 렌즈 샘플링(progressive lens sampling)을 사용하는 다중 패스 렌더링(multi-pass rendering)을 제안한다. 더 많은 렌즈 샘플들이 여러 렌더 패스들을 거치면서 점진적으로 사용된다. 각 픽셀은 착란원(circle of confusion)을 기반으로 예측된 최대 흐림 정도에 따라 다른 최종 렌더링 패스를 갖는다. 이 기법은 거리에 따른 피사계 심도 효과의 흐림 정도에 따라 각 픽셀에 다른 개수의 렌즈 샘플을 적용할 수 있게 한다. 이러한 가속화 방법은 불필요한 렌즈 샘플링을 줄이고 GPU의 캐시(cache) 적중률을 높여 직접 볼륨 렌더링에서 상호작용이 가능한 프레임 속도로 피사계 심도 효과를 렌더링 할 수 있게 한다. 다양한 데이터를 사용한 실험에서 제안한 방법은 실시간으로 사실적인 피사계 심도 효과를 생성했다. 이러한 결과는 우리의 방법이 오프라인 이미지 합성 방법과 유사한 품질의 피사계 심도 효과를 생성하면서 직접 볼륨 렌더링의 기존 피사계 심도 렌더링 방법보다 최대 12배까지 빠르다는 것을 보여준다.CHAPTER 1 INTRODUCTION 1 1.1 Motivation 1 1.2 Dissertation Goals 5 1.3 Main Contributions 6 1.4 Organization of Dissertation 8 CHAPTER 2 RELATED WORK 9 2.1 Depth of Field on Surface Rendering 10 2.1.1 Object-Space Approaches 11 2.1.2 Image-Space Approaches 15 2.2 Depth of Field on Volume Rendering 26 2.2.1 Blur Filtering on Slice-Based Volume Rendering 28 2.2.2 Stochastic Sampling on Volume Ray Casting 30 CHAPTER 3 DEPTH OF FIELD VOLUME RAY CASTING 33 3.1 Fundamentals 33 3.1.1 Depth of Field 34 3.1.2 Camera Models 36 3.1.3 Direct Volume Rendering 42 3.2 Geometry Setup 48 3.3 Lens Sampling Strategy 53 3.3.1 Sampling Techniques 53 3.3.2 Disk Mapping 57 3.4 CoC-Based Multi-Pass Rendering 60 3.4.1 Progressive Lens Sample Sequence 60 3.4.2 Final Render Pass Determination 62 CHAPTER 4 GPU IMPLEMENTATION 66 4.1 Overview 66 4.2 Rendering Pipeline 67 4.3 Focal Plane Transformation 74 4.4 Lens Sample Transformation 76 CHAPTER 5 EXPERIMENTAL RESULTS 78 5.1 Number of Lens Samples 79 5.2 Number of Render Passes 82 5.3 Render Pass Parameter 84 5.4 Comparison with Previous Methods 87 CHAPTER 6 CONCLUSION 97 Bibliography 101 Appendix 111Docto

Depth of field guided visualisation on light field displays

Light field displays are capable of realistic visualization of arbitrary 3D content. However, due to the finite number of light rays reproduced by the display, its bandwidth is limited in terms of angular and spatial resolution. Consequently, 3D content that falls outside of that bandwidth will cause aliasing during visualization. Therefore, a light field to be visualized must be properly preprocessed. In this thesis, we propose three methods that properly filter the parts in the input light field that would cause aliasing. First method is based on a 2D FIR circular filter that is applied over the 4D light field. Second method utilizes the structured nature of the epipolar plane images representing the light field. Third method adopts real-time multi-layer depth-of-field rendering using tiled splatting. We also establish a connection between lens parameters in the proposed depth-of-field rendering and the display’s bandwidth in order to determine the optimal blurring amount. As we prepare light field for light field displays, a stage is added to the proposed real-time rendering pipeline that simultaneously renders adjacent views. The rendering performance of the proposed methods is demonstrated on Holografika’s Holovizio 722RC projection-based light field display

Doctor of Philosophy in Computing

dissertationThe aim of direct volume rendering is to facilitate exploration and understanding of three-dimensional scalar fields referred to as volume datasets. Improving understanding is done by improving depth perception, whereas facilitating exploration is done by speeding up volume rendering. In this dissertation, improving both depth perception and rendering speed is considered. The impact of depth of field (DoF) on depth perception in direct volume rendering is evaluated by conducting a user study in which the test subjects had to choose which of two features, located at different depths, appeared to be in front in a volume-rendered image. Whereas DoF was expected to improve perception in all cases, the user study revealed that if used on the back feature, DoF reduced depth perception, whereas it produced a marked improvement when used on the front feature. We then worked on improving the speed of volume rendering on distributed memory machines. Distributed volume rendering has three stages: loading, rendering, and compositing. In this dissertation, the focus is on image compositing, more specifically, trying to optimize communication in image compositing algorithms. For that, we have developed the Task Overlapped Direct Send Tree image compositing algorithm, which works on both CPU- and GPU-accelerated supercomputers, which focuses on communication avoidance and overlapping communication with computation; the Dynamically Scheduled Region-Based image compositing algorithm that uses spatial and temporal awareness to efficiently schedule communication among compositing nodes, and a rendering and compositing pipeline that allows both image compositing and rendering to be done on GPUs of GPU-accelerated supercomputers. We tested these on CPU- and GPU-accelerated supercomputers and explain how these improvements allow us to obtain better performance than image compositing algorithms that focus on load-balancing and algorithms that have no spatial and temporal awareness of the rendering and compositing stages

Space-variant picture coding

PhDSpace-variant picture coding techniques exploit the strong spatial non-uniformity of the human visual system in order to increase coding efficiency in terms of perceived quality per bit. This thesis extends space-variant coding research in two directions. The first of these directions is in foveated coding. Past foveated coding research has been dominated by the single-viewer, gaze-contingent scenario. However, for research into the multi-viewer and probability-based scenarios, this thesis presents a missing piece: an algorithm for computing an additive multi-viewer sensitivity function based on an established eye resolution model, and, from this, a blur map that is optimal in the sense of discarding frequencies in least-noticeable- rst order. Furthermore, for the application of a blur map, a novel algorithm is presented for the efficient computation of high-accuracy smoothly space-variant Gaussian blurring, using a specialised filter bank which approximates perfect space-variant Gaussian blurring to arbitrarily high accuracy and at greatly reduced cost compared to the brute force approach of employing a separate low-pass filter at each image location. The second direction is that of artifi cially increasing the depth-of- field of an image, an idea borrowed from photography with the advantage of allowing an image to be reduced in bitrate while retaining or increasing overall aesthetic quality. Two synthetic depth of field algorithms are presented herein, with the desirable properties of aiming to mimic occlusion eff ects as occur in natural blurring, and of handling any number of blurring and occlusion levels with the same level of computational complexity. The merits of this coding approach have been investigated by subjective experiments to compare it with single-viewer foveated image coding. The results found the depth-based preblurring to generally be significantly preferable to the same level of foveation blurring

Digital Alchemy: Matter and Metamorphosis in Contemporary Digital Animation and Interface Design

The recent proliferation of special effects in Hollywood film has ushered in an era of digital transformation. Among scholars, digital technology is hailed as a revolutionary moment in the history of communication and representation. Nevertheless, media scholars and cultural historians have difficulty finding a language adequate to theorizing digital artifacts because they are not just texts to be deciphered. Rather, digital media artifacts also invite critiques about the status of reality because they resurrect ancient problems of embodiment and transcendence.In contrast to scholarly approaches to digital technology, computer engineers, interface designers, and special effects producers have invented a robust set of terms and phrases to describe the practice of digital animation. In order to address this disconnect between producers of new media and scholars of new media, I argue that the process of digital animation borrows extensively from a set of preexisting terms describing materiality that were prominent for centuries prior to the scientific revolution. Specifically, digital animators and interface designers make use of the ancient science, art, and technological craft of alchemy. Both alchemy and digital animation share several fundamental elements: both boast the power of being able to transform one material, substance, or thing into a different material, substance, or thing. Both seek to transcend the body and materiality but in the process, find that this elusive goal (realism and gold) is forever receding onto the horizon.The introduction begins with a literature review of the field of digital media studies. It identifies a gap in the field concerning disparate arguments about new media technology. On the one hand, scholars argue that new technologies like cyberspace and digital technology enable radical new forms of engagement with media on individual, social, and economic levels. At the same time that media scholars assert that our current epoch is marked by a historical rupture, many other researchers claim that new media are increasingly characterized by ancient metaphysical problems like embodiment and transcendence. In subsequent chapters I investigate this disparity

Validating Stereoscopic Volume Rendering

The evaluation of stereoscopic displays for surface-based renderings is well established in terms of accurate depth perception and tasks that require an understanding of the spatial layout of the scene. In comparison direct volume rendering (DVR) that typically produces images with a high number of low opacity, overlapping features is only beginning to be critically studied on stereoscopic displays. The properties of the specific images and the choice of parameters for DVR algorithms make assessing the effectiveness of stereoscopic displays for DVR particularly challenging and as a result existing literature is sparse with inconclusive results. In this thesis stereoscopic volume rendering is analysed for tasks that require depth perception including: stereo-acuity tasks, spatial search tasks and observer preference ratings. The evaluations focus on aspects of the DVR rendering pipeline and assess how the parameters of volume resolution, reconstruction filter and transfer function may alter task performance and the perceived quality of the produced images. The results of the evaluations suggest that the transfer function and choice of recon- struction filter can have an effect on the performance on tasks with stereoscopic displays when all other parameters are kept consistent. Further, these were found to affect the sensitivity and bias response of the participants. The studies also show that properties of the reconstruction filters such as post-aliasing and smoothing do not correlate well with either task performance or quality ratings. Included in the contributions are guidelines and recommendations on the choice of pa- rameters for increased task performance and quality scores as well as image based methods of analysing stereoscopic DVR images

Fast Perception-Based Depth of Field Rendering

Current algorithms to create depth of field (DOF) effects are either too costly to be applied in VR systems, or they produce inaccurate results. In this paper, we present a new algorithm to create DOF effects. The algorithm is based on two techniques: one of high accuracy and one of high speed but less accurate. The latter is used to create DOF effects in the peripheral viewing area where accurate results are not necessary. The first is applied to the viewing volume focussed at by the viewer. Both techniques make extensive use of rendering hardware, for texturing as well as image processing. The algorithm presented in this paper is an improvement over other (fast) DOF algorithms in that it is faster and provides better quality DOF effects where it matters most.

Fast perception-based depth of field rendering

Categories and Subject �� � �¤ � �� � �����������©�������������������������������§¡¤£§¦�����������¦���¦�¨�¦�£����§����¨�

Abstract Fast Perception-Based Depth of Field Rendering

Current algorithms to create depth of field (DOF) effects are either too costly to be applied in VR systems, or they produce inaccurate results. In this paper, we present a new algorithm to create DOF effects. The algorithm is based on two techniques: one of high accuracy and one of high speed but less accurate. The latter is used to create DOF effects in the peripheral viewing area where accurate results are not necessary. The first is applied to the viewing volume focussed at by the viewer. Both techniques make extensive use of rendering hardware, for texturing as well as image processing. The algorithm presented in this paper is an improvement over other (fast) DOF algorithms in that it is faster and provides better quality DOF effects where it matters most.