3DTV: Capture, transmission, and display of 3D video

[No abstract available

Doppler Encoded Excitation Patterning (DEEP) Microscopy

Traditional optical imaging systems rely on lenses and spatially-resolved detection to probe distinct locations on the object. We develop a novel computational approach to 2D and 3D imaging that instead measures the object\u27s spatial Fourier transform using a single-element detector and without requiring precision optics. This wide-field technique can be used to image biological and synthetic structures in fluoresced or scattered light using coherent or broadband illumination. It employs dynamic structured illumination, acousto-optics, RF electronics, and tomographic algorithms to circumvent several trade-offs in conventional imaging, such as the dependence of the optical transfer function on the imaging lenses and the coupling of resolution and depth of field. We use Fourier optics concepts to derive the dynamic optical transfer function, evaluate different Fourier sampling strategies, and investigate and compare tomographic algorithms for 2D and 3D image synthesis. We also develop conceptual and analytical models to describe imaging of fluorescent as well as amplitude and phase scattering objects, the effects of broadband and spatially-incoherent illumination, and nonlinear wide-field super-resolution imaging. We consider sources of noise, analyze and simulate SNR behavior for several types of noise and Fourier sampling strategies, and compare the sensitivity of the technique to conventional imaging. We describe several experimental proof-of-concept systems and present two-dimensional high-resolution tomographic image reconstructions in both scattered and fluoresced light demonstrating a thousandfold improvement in the depth of field compared to conventional lens-based microscopy. Finally, we explore approaches for high-speed Fourier sampling and propose several related sensing techniques, including wide-field fluorescence imaging in scattering media

Optimization strategies for respiratory motion management in stereotactic body radiation therapy

Various challenges arise during the treatment of lung tumors with stereotactic body radiation therapy (SBRT), which is a form of hypofractionated high precision conformal radiation therapy delivered to small targets. The dose is applied in only a few fractions and respiratory organ and tumor motion is a source of uncertainty additional to interfractional set-up errors. Respiratory organ and tumor motion is highly patient-specific and it affects the whole radiotherapy treatment chain. In this thesis, motion management techniques for SBRT are evaluated and improved in a clinical setting. A clinical need for improvement has been present at the LMU university hospital for each issue addressed in this thesis: Initially, the usage of respiratory correlated computed tomography (4DCT), which is vital for SBRT treatment, was seen as impractical and prone to uncertainties in the data reconstruction in its current form. Therefore, the 4DCT reconstruction workflow has been improved to minimize these potential error sources. Secondly, treatment planning for tumors affected by respiratory motion was evaluated and subsequently improved. Finally, the treatment technique of respiratory gating was implemented at the clinic, which led to the need of evaluating the respiratory gating characteristics of the novel system configuration. At first, the 4DCT reconstruction workflow used in clinical practice was investigated, as in the presence of respiratory motion the knowledge of tumor position over time is essential in SBRT treatments. Using 4DCT, the full motion range of the individual tumor can be determined. However, certain 4DCT reconstruction methods can under- or overestimate tumor motion due to limitations in the data acquisition scheme and due to the incorrect sorting of certain X-ray computed tomography (CT) image slices into different respiratory phases. As the regular clinical workflow of cycle-based sorting (CBS) without maximum inspiration detection (and therefore no clear starting point for the individual breathing cycles) seemed to be affected by these potential errors, the usage of CBS with correct maximum detection and another sorting algorithm of the respiration states, so-called local amplitude-based sorting (LAS), both have been implemented for a reduction of image artifacts and improved 4DCT quality. The three phase binning algorithms have been investigated in a phantom study (using 10 different breathing waveforms) and in a patient study (with 10 different patients). The mis-representation of the tumor volume was reduced in both implemented sorting algorithms compared to the previously used CBS approach (without correct maximum detection) in the phantom and the patient study. The clinical recommendation was the use of CBS with improved maximum detection, as too many manual interventions would be needed for the LAS workflow. Secondly, a combination of the actual patient breathing trace during treatment, the log files generated by the linear accelerator (LINAC), and Monte Carlo (MC) four-dimensional (4D) dose calculations for each individual fraction was implemented as a 4D dose evaluation tool. This workflow was tested in a clinical environment for SBRT treatment planning on multiple CT datasets featuring: a native free-breathing 3DCT, an average intensity projection (AIP) as well as a maximum intensity projection (MIP), both obtained from the patient's 4DCT, and density overrides (DOs) in a 3DCT. This study has been carried out for 5 SBRT patients for three-dimensional conformal radiation therapy (3D-CRT) and volumetric modulated arc therapy (VMAT) treatment plans. The dose has been recalculated on each 4DCT breathing phase according the the patient's breathing waveform and accumulated to the gross tumor volume (GTV) at the end-of-exhale (EOE) breathing phase using deformable image registration. Even though the least differences in planned and recalculated dose were found for AIP and MIP treatment planning, the results indicate a strong dependency on individual tumor motion due to the variability of breathing motion in general, and on tumor size. The combination of the patient's individual breathing trace during each SBRT fraction with 4D MC dose calculation based on the LINAC log file information leads to a good approximation of actual dose delivery. Finally, in order to ensure precise and accurate treatment for respiratory gating techniques, the technical characteristics of the LINAC in combination with a breathing motion monitoring system as s surrogate for tumor motion have to be identified. The dose delivery accuracy and the latency of a surface imaging system in connection with a modern medical LINAC were investigated using a dynamic breathing motion phantom. The dosimetric evaluation has been carried out using a static 2D-diode array. The measurement of the dose difference between gated and ungated radiation delivery was found to be below 1% (for clinical relevant gating levels of about 30%). The beam-on latency, or time delay, determined using radiographic films was found to be up to 851 ms±100 ms. With these known parameters, an adjustment of the pre-selected gating level or the internal target volume (ITV) margins could be made. With the highly patient-specific character of respiratory motion, lung SBRT faces many additional challenges besides the specific issues addressed in this thesis. However, the findings of this thesis have improved clinical workflows at the Department of Radiation Oncology of the LMU University hospital. In a future perspective, a workflow using evaluation of the actual 4D dose in combination with accurate 4DCT image acquisition and specialized treatment delivery (such as respiratory gating) has the potential for a safe further reduction of treatment margins and increased sparing of organs-at-risk (OARs) in SBRT without compromising tumor dose targeting accuracy

Pulse Oxigraphy: And other new in-depth perspectives through the near infrared window

The aim of this thesis was to investigate the feasability of contactless imaging pulse oximetry (proposed term: pulse oxigraphy). The patent disclosed in chapter 2 claims that such pulse oxigraphy can be achieved with camera-derived photoplethysmographic pulse waves at three wavelengths, preferably being 660, 810 and 940nm. From the absorption curves of hemoglobin and oxyhemoglobin it can be easily derived that two of these wavelengths (660 and 940nm) contain oxygenation-related information, and they have proven to be useful for conventional pulse oximetry (in transmission- mode as well as in reflectance-mode). The additional third wavelength (810nm) lies at a so-called isobestic point where the absorption curves of hemoglobin and oxyhemoglobin intersect. Thus, images and/or plethysmographic pulse waves recorded at 810nm do not contain oxygenation-related information, which is useful for reference purposes when dealing with shadows, reflections, movement artifacts and variations in geometry. With regard to pulse oxigraphy the following results were obtained: In chapter 3 we proved that it is possible to derive photoplethysmographic pulse waves containing the heart rythm of a living person at all three required wavelengths from camera recordings collected at a distance of 72 cm. To investigate and validate the capabilities for pulse oxigraphy with this set up, direct measurements on volunteers were sub optimal, because of: Signal-to-noise issues, sequentially recorded heartbeats for oxygen saturation calculations, and lack of a method to induce prolonged stable and adjustable oxygen saturation levels

Inverse problem theory in shape and action modeling

In this thesis we consider shape and action modeling problems under the perspective of inverse problem theory. Inverse problem theory proposes a mathematical framework for solving model parameter estimation problems. Inverse problems are typically ill-posed, which makes their solution challenging. Regularization theory and Bayesian statistical methods, which are proposed in the context of inverse problem theory, provide suitable methods for dealing with ill-posed problems. Regarding the application of inverse problem theory in shape and action modeling, we first discuss the problem of saliency prediction, considering a model proposed by the coherence theory of attention. According to coherence theory, salience regions emerge via proto-objects which we model using harmonic functions (thin-membranes). We also discuss the modeling of the 3D scene, as it is fundamental for extracting suitable scene features, which guide the generation of proto-objects. The next application we consider is the problem of image fusion. In this context, we propose a variational image fusion framework, based on confidence driven total variation regularization, and we consider its application to the problem of depth image fusion, which is an important step in the dense 3D scene reconstruction pipeline. The third problem we encounter regards action modeling, and in particular the recognition of human actions based on 3D data. Here, we employ a Bayesian nonparametric model to capture the idiosyncratic motions of the different body parts. Recognition is achieved by comparing the motion behaviors of the subject to a dictionary of behaviors for each action, learned by examples collected from other subjects. Next, we consider the 3D modeling of articulated objects from images taken from the web, with application to the 3D modeling of animals. By decomposing the full object in rigid components and by considering different aspects of these components, we model the object up this hierarchy, in order to obtain a 3D model of the entire object. Single view 3D modeling as well as model registration is performed, based on regularization methods. The last problem we consider, is the modeling of 3D specular (non-Lambertian) surfaces from a single image. To solve this challenging problem we propose a Bayesian non-parametric model for estimating the normal field of the surface from its appearance, by identifying the material of the surface. After computing an initial model of the surface, we apply regularization of its normal field considering also a photo-consistency constraint, in order to estimate the final shape of the surface. Finally, we conclude this thesis by summarizing the most significant results and by suggesting future directions regarding the application of inverse problem theory to challenging computer vision problems, as the ones encountered in this work

On Improving Generalization of CNN-Based Image Classification with Delineation Maps Using the CORF Push-Pull Inhibition Operator

Deployed image classification pipelines are typically dependent on the images captured in real-world environments. This means that images might be affected by different sources of perturbations (e.g. sensor noise in low-light environments). The main challenge arises by the fact that image quality directly impacts the reliability and consistency of classification tasks. This challenge has, hence, attracted wide interest within the computer vision communities. We propose a transformation step that attempts to enhance the generalization ability of CNN models in the presence of unseen noise in the test set. Concretely, the delineation maps of given images are determined using the CORF push-pull inhibition operator. Such an operation transforms an input image into a space that is more robust to noise before being processed by a CNN. We evaluated our approach on the Fashion MNIST data set with an AlexNet model. It turned out that the proposed CORF-augmented pipeline achieved comparable results on noise-free images to those of a conventional AlexNet classification model without CORF delineation maps, but it consistently achieved significantly superior performance on test images perturbed with different levels of Gaussian and uniform noise

Fehlerkaschierte Bildbasierte Darstellungsverfahren

Creating photo-realistic images has been one of the major goals in computer graphics since its early days. Instead of modeling the complexity of nature with standard modeling tools, image-based approaches aim at exploiting real-world footage directly,as they are photo-realistic by definition. A drawback of these approaches has always been that the composition or combination of different sources is a non-trivial task, often resulting in annoying visible artifacts. In this thesis we focus on different techniques to diminish visible artifacts when combining multiple images in a common image domain. The results are either novel images, when dealing with the composition task of multiple images, or novel video sequences rendered in real-time, when dealing with video footage from multiple cameras.Fotorealismus ist seit jeher eines der großen Ziele in der Computergrafik. Anstatt die Komplexität der Natur mit standardisierten Modellierungswerkzeugen nachzubauen, gehen bildbasierte Ansätze den umgekehrten Weg und verwenden reale Bildaufnahmen zur Modellierung, da diese bereits per Definition fotorealistisch sind. Ein Nachteil dieser Variante ist jedoch, dass die Komposition oder Kombination mehrerer Quellbilder eine nichttriviale Aufgabe darstellt und häufig unangenehm auffallende Artefakte im erzeugten Bild nach sich zieht. In dieser Dissertation werden verschiedene Ansätze verfolgt, um Artefakte zu verhindern oder abzuschwächen, welche durch die Komposition oder Kombination mehrerer Bilder in einer gemeinsamen Bilddomäne entstehen. Im Ergebnis liefern die vorgestellten Verfahren neue Bilder oder neue Ansichten einer Bildsammlung oder Videosequenz, je nachdem, ob die jeweilige Aufgabe die Komposition mehrerer Bilder ist oder die Kombination mehrerer Videos verschiedener Kameras darstellt