High quality texture synthesis

Texture synthesis is a core process in Computer Graphics and design. It is used extensively in a wide range of applications, including computer games, virtual environments, manufacturing, and rendering. This thesis investigates a novel approach to texture synthesis in order to significantly improve speed, memory requirements, and quality. An analysis of texture properties is created, to enable the gathering a representative dataset, and a qualitative evaluation of texture synthesis algorithms. A new algorithm to make non-repeating texture synthesis on-the-fly possible is developed, tested, and evaluated. This parallel patch-based method allows repeatable sampling without cache, without creating visually noticeable repetitions, as confirmed by a perceptive objective study on quality. In order to quantify the quality of existing algorithms and to facilitate further development in the field, desired texture properties are classified and analysed, and a minimal set of textures is created according to these properties to allow subjective evaluation of texture synthesis algorithms. This dataset is then used in a user study which evaluates the quality of texture synthesis algorithms. For the first time in the field of texture synthesis, statistically significant findings quantify the quality of selected repeatable algorithms, and make it possible to evaluate new improved methods. Finally, in an effort to make these findings applicable in the British tile manufacturing industry, the developed texture synthesis technology is made available to Johnson Tiles

Texture Structure Analysis

abstract: Texture analysis plays an important role in applications like automated pattern inspection, image and video compression, content-based image retrieval, remote-sensing, medical imaging and document processing, to name a few. Texture Structure Analysis is the process of studying the structure present in the textures. This structure can be expressed in terms of perceived regularity. Our human visual system (HVS) uses the perceived regularity as one of the important pre-attentive cues in low-level image understanding. Similar to the HVS, image processing and computer vision systems can make fast and efficient decisions if they can quantify this regularity automatically. In this work, the problem of quantifying the degree of perceived regularity when looking at an arbitrary texture is introduced and addressed. One key contribution of this work is in proposing an objective no-reference perceptual texture regularity metric based on visual saliency. Other key contributions include an adaptive texture synthesis method based on texture regularity, and a low-complexity reduced-reference visual quality metric for assessing the quality of synthesized textures. In order to use the best performing visual attention model on textures, the performance of the most popular visual attention models to predict the visual saliency on textures is evaluated. Since there is no publicly available database with ground-truth saliency maps on images with exclusive texture content, a new eye-tracking database is systematically built. Using the Visual Saliency Map (VSM) generated by the best visual attention model, the proposed texture regularity metric is computed. The proposed metric is based on the observation that VSM characteristics differ between textures of differing regularity. The proposed texture regularity metric is based on two texture regularity scores, namely a textural similarity score and a spatial distribution score. In order to evaluate the performance of the proposed regularity metric, a texture regularity database called RegTEX, is built as a part of this work. It is shown through subjective testing that the proposed metric has a strong correlation with the Mean Opinion Score (MOS) for the perceived regularity of textures. The proposed method is also shown to be robust to geometric and photometric transformations and outperforms some of the popular texture regularity metrics in predicting the perceived regularity. The impact of the proposed metric to improve the performance of many image-processing applications is also presented. The influence of the perceived texture regularity on the perceptual quality of synthesized textures is demonstrated through building a synthesized textures database named SynTEX. It is shown through subjective testing that textures with different degrees of perceived regularities exhibit different degrees of vulnerability to artifacts resulting from different texture synthesis approaches. This work also proposes an algorithm for adaptively selecting the appropriate texture synthesis method based on the perceived regularity of the original texture. A reduced-reference texture quality metric for texture synthesis is also proposed as part of this work. The metric is based on the change in perceived regularity and the change in perceived granularity between the original and the synthesized textures. The perceived granularity is quantified through a new granularity metric that is proposed in this work. It is shown through subjective testing that the proposed quality metric, using just 2 parameters, has a strong correlation with the MOS for the fidelity of synthesized textures and outperforms the state-of-the-art full-reference quality metrics on 3 different texture databases. Finally, the ability of the proposed regularity metric in predicting the perceived degradation of textures due to compression and blur artifacts is also established.Dissertation/ThesisPh.D. Electrical Engineering 201

Modelling appearance and geometry from images

Acquisition of realistic and relightable 3D models of large outdoor structures, such as buildings, requires the modelling of detailed geometry and visual appearance. Recovering these material characteristics can be very time consuming and needs specially dedicated equipment. Alternatively, surface detail can be conveyed by textures recovered from images, whose appearance is only valid under the originally photographed viewing and lighting conditions. Methods to easily capture locally detailed geometry, such as cracks in stone walls, and visual appearance require control of lighting conditions, which are usually restricted to small portions of surfaces captured at close range.This thesis investigates the acquisition of high-quality models from images, using simple photographic equipment and modest user intervention. The main focus of this investigation is on approximating detailed local depth information and visual appearance, obtained using a new image-based approach, and combining this with gross-scale 3D geometry. This is achieved by capturing these surface characteristics in small accessible regions and transferring them to the complete façade. This approach yields high-quality models, imparting the illusion of measured reflectance. In this thesis, we first present two novel algorithms for surface detail and visual appearance transfer, where these material properties are captured for small exemplars, using an image-based technique. Second, we develop an interactive solution to solve the problems of performing the transfer over both a large change in scale and to the different materials contained in a complete façade. Aiming to completely automate this process, a novel algorithm to differentiate between materials in the façade and associate them with the correct exemplars is introduced with promising results. Third, we present a new method for texture reconstruction from multiple images that optimises texture quality, by choosing the best view for every point and minimising seams. Material properties are transferred from the exemplars to the texture map, approximating reflectance and meso-structure. The combination of these techniques results in a complete working system capable of producing realistic relightable models of full building façades, containing high-resolution geometry and plausible visual appearance.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Quantifying Texture Scale in Accordance With Human Perception

Visual texture has multiple perceptual attributes (e.g. regularity, isotropy, etc.), including scale. The scale of visual texture has been defined as the size of the repeating unit (or texel) of which the texture is composed. Not all textures are formed through the placement of a clearly discernible repeating unit (e.g. irregular and stochastic textures). There is currently no rigorous definition for texture scale that is applicable to textures of a wide range of regularities. We hypothesised that texture scale ought to extend to these less regular textures. Non-overlapping sample windows (or patches) taken from a texture appear increasingly similar as the size of the window gets larger. This is true irrespective of whether the texture is formed by the placement of a discernible repeating unit or not. We propose the following new characterisation for texture scale: “the smallest window size beyond within which texture appears consistently”. We perform two psychophysical studies and report data that demonstrates consensus across subjects and across methods of probing in the assessment of texture scale. We then present an empirical algorithm for the estimation of scale based on this characterisation. We demonstrate agreement between the algorithm and (subjective) human assessment with an RMS accuracy of 1.2 just-noticeable-differences, a significant improvement over previous published algorithms. We provide two ground-truth perceptual datasets, one for each of our psychophysical studies, for the texture scale of the entire Brodatz album, together with confidence levels for each of our estimates. Finally, we make available an online tool which researchers can use to obtain texture scale estimates by uploading images of textures

Ray Tracing Displacement Mapped Surface

Displacement mapping is a technique in computer graphics which allows a simple base surface to be deformed into a more complex surface by applying a texture to change the geometry. This is achieved by applying to each point on the surface a displacement, specified by a displacement function, with a magnitude specified by a height field. This technique makes it possible to transform the simple primitives used in computer graphics today into visually rich and geometrically complex surfaces. Displacement mapping is a texture mapping technique in which the texture is the height field defining the displacement magnitudes. But, unlike any other form of texture mapping, displacement mapping alters the surface geometry. This has many implications for the rendering of displacement mapped surfaces. It must be considered early in the rendering process during the visibility calculations (since it defines the actual geometry of the surface). This is in contrast to other forms of texture mapping which are applied after the visibility of the surface is known. This fact accounts for much of the power and complexity involved in rendering displacement mapped surfaces This thesis provides an investigation into ways to render such surfaces by the use of ray tracing. It is commonly believed that displacement mapped surfaces are too complex to be ray-traced due to the complex nature of the geometry they define. This myth is disproved by the algorithms contained herein. Three algorithms are presented which tackle the ray-surface intersection problem for displacement mapped surfaces (this being the core calculation in a ray tracer). The first algorithm tackles the problem geometrically by analysing the geometry of the intersection calculation. This approach provides a fast algorithm but with limited applicability. It is only suitable for simple base surfaces where the underlying geometry can be easily analysed. The second algorithm reduced the intersection calculation to a system of non-linear equations and applies existing numerical techniques to solve these. This approach, although very general, proves to unsuccessful due to the enormous amount of computation involved. The third approach polygonalises the displacement-mapped surface as it is rendered and calculates the intersections with the generated polygons. This, combined with a system to allow the efficient generation, storage and processing of the generated polygons, provides the first practical system for ray tracing displacement mapped surfaces

GPU data structures for graphics and vision

Graphics hardware has in recent years become increasingly programmable, and its programming APIs use the stream processor model to expose massive parallelization to the programmer. Unfortunately, the inherent restrictions of the stream processor model, used by the GPU in order to maintain high performance, often pose a problem in porting CPU algorithms for both video and volume processing to graphics hardware. Serial data dependencies which accelerate CPU processing are counterproductive for the data-parallel GPU. This thesis demonstrates new ways for tackling well-known problems of large scale video/volume analysis. In some instances, we enable processing on the restricted hardware model by re-introducing algorithms from early computer graphics research. On other occasions, we use newly discovered, hierarchical data structures to circumvent the random-access read/fixed write restriction that had previously kept sophisticated analysis algorithms from running solely on graphics hardware. For 3D processing, we apply known game graphics concepts such as mip-maps, projective texturing, and dependent texture lookups to show how video/volume processing can benefit algorithmically from being implemented in a graphics API. The novel GPU data structures provide drastically increased processing speed, and lift processing heavy operations to real-time performance levels, paving the way for new and interactive vision/graphics applications.Graphikhardware wurde in den letzen Jahren immer weiter programmierbar. Ihre APIs verwenden das Streamprozessor-Modell, um die massive Parallelisierung auch für den Programmierer verfügbar zu machen. Leider folgen aus dem strikten Streamprozessor-Modell, welches die GPU für ihre hohe Rechenleistung benötigt, auch Hindernisse in der Portierung von CPU-Algorithmen zur Video- und Volumenverarbeitung auf die GPU. Serielle Datenabhängigkeiten beschleunigen zwar CPU-Verarbeitung, sind aber für die daten-parallele GPU kontraproduktiv . Diese Arbeit präsentiert neue Herangehensweisen für bekannte Probleme der Video- und Volumensverarbeitung. Teilweise wird die Verarbeitung mit Hilfe von modifizierten Algorithmen aus der frühen Computergraphik-Forschung an das beschränkte Hardwaremodell angepasst. Anderswo helfen neu entdeckte, hierarchische Datenstrukturen beim Umgang mit den Schreibzugriff-Restriktionen die lange die Portierung von komplexeren Bildanalyseverfahren verhindert hatten. In der 3D-Verarbeitung nutzen wir bekannte Konzepte aus der Computerspielegraphik wie Mipmaps, projektive Texturierung, oder verkettete Texturzugriffe, und zeigen auf welche Vorteile die Video- und Volumenverarbeitung aus hardwarebeschleunigter Graphik-API-Implementation ziehen kann. Die präsentierten GPU-Datenstrukturen bieten drastisch schnellere Verarbeitung und heben rechenintensive Operationen auf Echtzeit-Niveau. Damit werden neue, interaktive Bildverarbeitungs- und Graphik-Anwendungen möglich

Multi-dimensional local binary pattern texture descriptors and their application for medical image analysis

Texture can be broadly stated as spatial variation of image intensities. Texture analysis and classification is a well researched area for its importance to many computer vision applications. Consequently, much research has focussed on deriving powerful and efficient texture descriptors. Local binary patterns (LBP) and its variants are simple yet powerful texture descriptors. LBP features describe the texture neighbourhood of a pixel using simple comparison operators, and are often calculated based on varying neighbourhood radii to provide multi-resolution texture descriptions. A comprehensive evaluation of different LBP variants on a common benchmark dataset is missing in the literature. This thesis presents the performance for different LBP variants on texture classification and retrieval tasks. The results show that multi-scale local binary pattern variance (LBPV) gives the best performance over eight benchmarked datasets. Furthermore, improvements to the Dominant LBP (D-LBP) by ranking dominant patterns over complete training set and Compound LBP (CM-LBP) by considering 16 bits binary codes are suggested which are shown to outperform their original counterparts. The main contribution of the thesis is the introduction of multi-dimensional LBP features, which preserve the relationships between different scales by building a multi-dimensional histogram. The results on benchmarked classification and retrieval datasets clearly show that the multi-dimensional LBP (MD-LBP) improves the results compared to conventional multi-scale LBP. The same principle is applied to LBPV (MD-LBPV), again leading to improved performance. The proposed variants result in relatively large feature lengths which is addressed using three different feature length reduction techniques. Principle component analysis (PCA) is shown to give the best performance when the feature length is reduced to match that of conventional multi-scale LBP. The proposed multi-dimensional LBP variants are applied for medical image analysis application. The first application is nailfold capillary (NC) image classification. Performance of MD-LBPV on NC images is highest, whereas for second application, HEp-2 cell classification, performance of MD-LBP is highest. It is observed that the proposed texture descriptors gives improved texture classification accuracy