Rectification and intermediate view synthesis

In this project c++ code supporting intermediate view synthesis was developed. The idea was to create classes and functions which can be later easily used to create intermediate views. Main part of the code is responsible for rectification. Images from two cameras may be rectified and then further operations with the images can be done. In this case the next operation on the rectified images is intermediate view synthesis. Special function computes from two rectified images the virtual view. The virtual image can be computed for any place set between two cameras taking the real image

INTERMEDIATE VIEW RECONSTRUCTION FOR MULTISCOPIC 3D DISPLAY

This thesis focuses on Intermediate View Reconstruction (IVR) which generates additional images from the available stereo images. The main application of IVR is to generate the content of multiscopic 3D displays, and it can be applied to generate different viewpoints to Free-viewpoint TV (FTV). Although IVR is considered a good approach to generate additional images, there are some problems with the reconstruction process, such as detecting and handling the occlusion areas, preserving the discontinuity at edges, and reducing image artifices through formation of the texture of the intermediate image. The occlusion area is defined as the visibility of such an area in one image and its disappearance in the other one. Solving IVR problems is considered a significant challenge for researchers. In this thesis, several novel algorithms have been specifically designed to solve IVR challenges by employing them in a highly robust intermediate view reconstruction algorithm. Computer simulation and experimental results confirm the importance of occluded areas in IVR. Therefore, we propose a novel occlusion detection algorithm and another novel algorithm to Inpaint those areas. Then, these proposed algorithms are employed in a novel occlusion-aware intermediate view reconstruction that finds an intermediate image with a given disparity between two input images. This novelty is addressed by adding occlusion awareness to the reconstruction algorithm and proposing three quality improvement techniques to reduce image artifices: filling the re-sampling holes, removing ghost contours, and handling the disocclusion area. We compared the proposed algorithms to the previously well-known algorithms on each field qualitatively and quantitatively. The obtained results show that our algorithms are superior to the previous well-known algorithms. The performance of the proposed reconstruction algorithm is tested under 13 real images and 13 synthetic images. Moreover, analysis of a human-trial experiment conducted with 21 participants confirmed that the reconstructed images from our proposed algorithm have very high quality compared with the reconstructed images from the other existing algorithms

a Berlin case study

Durch den Prozess der Urbanisierung verändert die Menschheit die Erdoberfläche in großem Ausmaß und auf unwiederbringliche Weise. Die optische Fernerkundung ist eine Art der Erdbeobachtung, die das Verständnis dieses dynamischen Prozesses und seiner Auswirkungen erweitern kann. Die vorliegende Arbeit untersucht, inwiefern hyperspektrale Daten Informationen über Versiegelung liefern können, die der integrierten Analyse urbaner Mensch-Umwelt-Beziehungen dienen. Hierzu wird die Verarbeitungskette von Vorverarbeitung der Rohdaten bis zur Erstellung referenzierter Karten zu Landbedeckung und Versiegelung am Beispiel von Hyperspectral Mapper Daten von Berlin ganzheitlich untersucht. Die traditionelle Verarbeitungskette wird mehrmals erweitert bzw. abgewandelt. So wird die radiometrische Vorverarbeitung um die Normalisierung von Helligkeitsgradienten erweitert, welche durch die direktionellen Reflexionseigenschaften urbaner Oberflächen entstehen. Die Klassifikation in fünf spektral komplexe Landnutzungsklassen wird mit Support Vector Maschinen ohne zusätzliche Merkmalsextraktion oder Differenzierung von Subklassen durchgeführt...thesi

Compact Environment Modelling from Unconstrained Camera Platforms

Mobile robotic systems need to perceive their surroundings in order to act independently. In this work a perception framework is developed which interprets the data of a binocular camera in order to transform it into a compact, expressive model of the environment. This model enables a mobile system to move in a targeted way and interact with its surroundings. It is shown how the developed methods also provide a solid basis for technical assistive aids for visually impaired people

Edge-based motion segmentation

Motion segmentation is the process of dividing video frames into regions which have different motions, providing a cut-out of the moving objects. Such a segmentation is a necessary first stage in many video analysis applications, but providing an accurate, efficient motion segmentation still presents a challenge. This dissertation proposes a novel approach to motion segmentation, using the image edges in a frame. Using edges, a motion can be calculated for each object. Edges provide good motion information, and it is shown that a set of edges, labelled according to the object motion that they obey, is sufficient to completely determine the labelling of the whole frame, up to unresolvable ambiguities. The areas of the frame between edges are divided into regions, grouping together pixels of similar colour, and these regions can each be assigned to different motion layers by reference to the edges. The depth ordering of these layers can also be deduced. A Bayesian framework is presented, which determines the most likely region labelling and depth ordering, given edges labelled with their probability of obeying each of the object motions. An efficient implementation of this framework is presented, initially for segmenting two motions (foreground and background) using two frames. The ExpectationMaximisation algorithm is used to determine the two motions and calculate the label probability for each edge. The frame is then segmented into regions. The best motion labelling for these regions is determined using simulated annealing. Extensions of this simple implementation are then presented. It is demonstrated how, by tracking the edges into further frames, the statistics may be accumulated to provide an even more accurate and robust segmentation. This also allows a complete sequence to be segmented. It is then demonstrated that the framework can be extended to a larger number of motions. A new hierarchical method of initialising the Expectation-Maximisation algorithm is described, which also determines the best number of motions. These techniques have been extensively tested on thirty-four real sequences, covering a wide range of genres. The results demonstrate that the proposed edge-based approach is an accurate and efficient method of obtaining a motion segmentation

Intermediate View Synthesis Considering Occluded and Ambiguously Referenced Image Regions

In this paper, we present an algorithm for synthesizing intermediate views from a single stereopair. The key contributions of this algorithm are the incorporation of scene assumptions and a disparity estimation confidence measure that lead to the accurate synthesis of occluded and ambiguously referenced regions. The synthesized views have been displayed on a multi-view binocular imaging system, with subjectively effective motion parallax and diminished eye strain. 1. INTRODUCTION For three-dimensional television (3D-TV) to become feasible and acceptable on a wide scale, the added realism must outweigh any required increases in processing and system complexity, and the stereoscopic information must be comfortable to view. Both of these goals can be achieved if intermediate views of the scene are available. An intermediate view is defined as the image that would be obtained from a camera located between and on a straight-line connecting the given stereopair's cameras. While binocular ..

How sketches work: a cognitive theory for improved system design

Evidence is presented that in the early stages of design or composition the mental processes used by artists for visual invention require a different type of support from those used for visualising a nearly complete object. Most research into machine visualisation has as its goal the production of realistic images which simulate the light pattern presented to the retina by real objects. In contrast sketch attributes preserve the results of cognitive processing which can be used interactively to amplify visual thought. The traditional attributes of sketches include many types of indeterminacy which may reflect the artist's need to be "vague". Drawing on contemporary theories of visual cognition and neuroscience this study discusses in detail the evidence for the following functions which are better served by rough sketches than by the very realistic imagery favoured in machine visualising systems. 1. Sketches are intermediate representational types which facilitate the mental translation between descriptive and depictive modes of representing visual thought. 2. Sketch attributes exploit automatic processes of perceptual retrieval and object recognition to improve the availability of tacit knowledge for visual invention. 3. Sketches are percept-image hybrids. The incomplete physical attributes of sketches elicit and stabilise a stream of super-imposed mental images which amplify inventive thought. 4. By segregating and isolating meaningful components of visual experience, sketches may assist the user to attend selectively to a limited part of a visual task, freeing otherwise over-loaded cognitive resources for visual thought. 5. Sequences of sketches and sketching acts support the short term episodic memory for cognitive actions. This assists creativity, providing voluntary control over highly practised mental processes which can otherwise become stereotyped. An attempt is made to unite the five hypothetical functions. Drawing on the Baddeley and Hitch model of working memory, it is speculated that the five functions may be related to a limited capacity monitoring mechanism which makes tacit visual knowledge explicitly available for conscious control and manipulation. It is suggested that the resources available to the human brain for imagining nonexistent objects are a cultural adaptation of visual mechanisms which evolved in early hominids for responding to confusing or incomplete stimuli from immediately present objects and events. Sketches are cultural inventions which artificially mimic aspects of such stimuli in order to capture these shared resources for the different purpose of imagining objects which do not yet exist. Finally the implications of the theory for the design of improved machine systems is discussed. The untidy attributes of traditional sketches are revealed to include cultural inventions which serve subtle cognitive functions. However traditional media have many short-comings which it should be possible to correct with new technology. Existing machine systems for sketching tend to imitate nonselectively the media bound properties of sketches without regard to the functions they serve. This may prove to be a mistake. It is concluded that new system designs are needed in which meaningfully structured data and specialised imagery amplify without interference or replacement the impressive but limited creative resources of the visual brain

Ancestral sequence reconstruction as an accessible tool for the engineering of biocatalyst stability

Synthetic biology is the engineering of life to imbue non-natural functionality. As such, synthetic biology has considerable commercial potential, where synthetic metabolic pathways are utilised to convert low value substrates into high value products. High temperature biocatalysis offers several system-level benefits to synthetic biology, including increased dilution of substrate, increased reaction rates and decreased contamination risk. However, the current gamut of tools available for the engineering of thermostable proteins are either expensive, unreliable, or poorly understood, meaning their adoption into synthetic biology workflows is treacherous. This thesis focuses on the development of an accessible tool for the engineering of protein thermostability, based on the evolutionary biology tool ancestral sequence reconstruction (ASR). ASR allows researchers to walk back in time along the branches of a phylogeny and predict the most likely representation of a protein family’s ancestral state. It also has simple input requirements, and its output proteins are often observed to be thermostable, making ASR tractable to protein engineering. Chapter 2 explores the applicability of multiple ASR methods to the engineering of a carboxylic acid reductase (CAR) biocatalyst. Despite the family emerging only 500 million years ago, ancestors presented considerable improvements in thermostability over their modern counterparts. We proceed to thoroughly characterise the ancestral enzymes for their inclusion into the CAR biocatalytic toolbox. Chapter 3 explores why ASR derived proteins may be thermostable despite a mesophilic history. An in silico toolbox for tracking models of protein stability over simulated evolutionary time at the sequence, protein and population level is built. We provide considerable evidence that the sequence alignments of simulated protein families that evolved at marginal stability are saturated with stabilising residues. ASR therefore derives sequences from a dataset biased toward stabilisation. Importantly, while ASR is accessible, it still requires a steep learning curve based on its requirements of phylogenetic expertise. In chapter 4, we utilise the evolutionary model produced in chapter 3 to develop a highly simplified and accessible ASR protocol. This protocol was then applied to engineer CAR enzymes that displayed dramatic increases in thermostability compared to both modern CARs and the thermostable AncCARs presented in chapter 2

A New Approach to Understanding Biological Control of Quinone Electrochemistry

Oxidoreductases play pivotal roles in energy capturing and converting processes of life. During these processes, quinones shuttle protons and reducing equivalents between membrane-bound oxidoreductases that generate the proton motive force during oxidative phosphorylation and photophosphorylation. A key mechanistic feature of these oxidoreductases is their ability to tune the reduction potentials of the hydroquinone, semiquinone and oxidized states of their substrate quinones. This level of control allows for maximization of conversion efficiency between the energy of the quinone reducing equivalents and the proton motive force, and prevents side reactions that may be fatal to cells. A half-century of experimental study and computational modeling of the respiratory and photosynthetic complexes has revealed little information on how this mechanistic control is accomplished. To obtain mechanistic insights into the control process, it is necessary to eliminate the biological complexity intrinsic to natural quinone oxidoreductases and create experimental systems that are simplified maquettes of quinone active sites. In this work, development of a naphthoquinone amino acid (Naq), modeled after vitamin K, allowed the creation of a range of quinone peptide maquettes designed to address uncertain mechanistic details of biological quinone control. In a simple heptamer, Naq acquires properties of quinone cofactors found in the three distinct classes of active sites of membrane oxidoreductases under different experimental conditions. Study of Naq in a lanthanide ion binding EF hand peptide allowed observation of the effect of a structural transition from coil to alpha-helix on the aqueous midpoint potential of Naq and measurement of the rate of electron transfer between reduced and oxidized Naq. Naq was also incorporated into a structured miniprotein based upon the TrpCage using a combination of the SCADS computational approach and iterative redesign by hand, creating a simple scaffold for evaluating effects of changing the local environment on Naq. Finally, using expressed protein ligation, Naq was incorporated into a single chain heme-binding maquette. Studies using this multi-cofactor protein to explore electron transfer reactions to and from Naq like those critical to respiration and photosynthesis are underway