A complete hand-drawn sketch vectorization framework

Vectorizing hand-drawn sketches is a challenging task, which is of paramount importance for creating CAD vectorized versions for the fashion and creative workflows. This paper proposes a complete framework that automatically transforms noisy and complex hand-drawn sketches with different stroke types in a precise, reliable and highly-simplified vectorized model. The proposed framework includes a novel line extraction algorithm based on a multi-resolution application of Pearson's cross correlation and a new unbiased thinning algorithm that can get rid of scribbles and variable-width strokes to obtain clean 1-pixel lines. Other contributions include variants of pruning, merging and edge linking procedures to post-process the obtained paths. Finally, a modification of the original Schneider's vectorization algorithm is designed to obtain fewer control points in the resulting Bezier splines. All the proposed steps of the framework have been extensively tested and compared with state-of-the-art algorithms, showing (both qualitatively and quantitatively) its outperformance

Variational methods and its applications to computer vision

Many computer vision applications such as image segmentation can be formulated in a ''variational'' way as energy minimization problems. Unfortunately, the computational task of minimizing these energies is usually difficult as it generally involves non convex functions in a space with thousands of dimensions and often the associated combinatorial problems are NP-hard to solve. Furthermore, they are ill-posed inverse problems and therefore are extremely sensitive to perturbations (e.g. noise). For this reason in order to compute a physically reliable approximation from given noisy data, it is necessary to incorporate into the mathematical model appropriate regularizations that require complex computations. The main aim of this work is to describe variational segmentation methods that are particularly effective for curvilinear structures. Due to their complex geometry, classical regularization techniques cannot be adopted because they lead to the loss of most of low contrasted details. In contrast, the proposed method not only better preserves curvilinear structures, but also reconnects some parts that may have been disconnected by noise. Moreover, it can be easily extensible to graphs and successfully applied to different types of data such as medical imagery (i.e. vessels, hearth coronaries etc), material samples (i.e. concrete) and satellite signals (i.e. streets, rivers etc.). In particular, we will show results and performances about an implementation targeting new generation of High Performance Computing (HPC) architectures where different types of coprocessors cooperate. The involved dataset consists of approximately 200 images of cracks, captured in three different tunnels by a robotic machine designed for the European ROBO-SPECT project.Open Acces

Image Analysis via Applied Harmonic Analysis : Perceptual Image Quality Assessment, Visual Servoing, and Feature Detection

Certain systems of analyzing functions developed in the field of applied harmonic analysis are specifically designed to yield efficient representations of structures which are characteristic of common classes of two-dimensional signals, like images. In particular, functions in these systems are typically sensitive to features that define the geometry of a signal, like edges and curves in the case of images. These properties make them ideal candidates for a wide variety of tasks in image processing and image analysis. This thesis discusses three recently developed approaches to utilizing systems of wavelets, shearlets, and alpha-molecules in specific image analysis tasks. First, a perceptual image similarity measure is introduced that is solely based on the coefficients obtained from six discrete Haar wavelet filters but yields state of the art correlations with human opinion scores on large benchmark databases. The second application concerns visual servoing, which is a technique for controlling the motion of a robot by using feedback from a visual sensor. In particular, it will be investigated how the coefficients yielded by discrete wavelet and shearlet transforms can be used as the visual features that control the motion of a robot with six degrees of freedom. Finally, a novel framework for the detection and characterization of features such as edges, ridges, and blobs in two-dimensional images is presented and evaluated in extensive numerical experiments. Here, versatile and robust feature detectors are obtained by exploiting the special symmetry properties of directionally sensitive analyzing functions in systems created within the recently introduced alpha-molecule framework

Image Analysis via Applied Harmonic Analysis : Perceptual Image Quality Assessment, Visual Servoing, and Feature Detection

Certain systems of analyzing functions developed in the field of applied harmonic analysis are specifically designed to yield efficient representations of structures which are characteristic of common classes of two-dimensional signals, like images. In particular, functions in these systems are typically sensitive to features that define the geometry of a signal, like edges and curves in the case of images. These properties make them ideal candidates for a wide variety of tasks in image processing and image analysis. This thesis discusses three recently developed approaches to utilizing systems of wavelets, shearlets, and alpha-molecules in specific image analysis tasks. First, a perceptual image similarity measure is introduced that is solely based on the coefficients obtained from six discrete Haar wavelet filters but yields state of the art correlations with human opinion scores on large benchmark databases. The second application concerns visual servoing, which is a technique for controlling the motion of a robot by using feedback from a visual sensor. In particular, it will be investigated how the coefficients yielded by discrete wavelet and shearlet transforms can be used as the visual features that control the motion of a robot with six degrees of freedom. Finally, a novel framework for the detection and characterization of features such as edges, ridges, and blobs in two-dimensional images is presented and evaluated in extensive numerical experiments. Here, versatile and robust feature detectors are obtained by exploiting the special symmetry properties of directionally sensitive analyzing functions in systems created within the recently introduced alpha-molecule framework

Applied microlocal analysis of deep neural networks for inverse problems

Deep neural networks have recently shown state-of-the-art performance in different imaging tasks. As an example, EfficientNet is today the best image classifier on the ImageNet challenge. They are also very powerful for image reconstruction, for example, deep learning currently yields the best methods for CT reconstruction. Most imaging problems, such as CT reconstruction, are ill-posed inverse problems, which hence require regularization techniques typically based on a-priori information. Also, due to the human visual system, singularities such as edge-like features are the governing structures of images. This leads to the question of how to incorporate such information into a solver of an inverse problem in imaging and how deep neural networks operate on singularities. The main research theme of this thesis is to introduce theoretically founded approaches to use deep neural networks in combination with model-based methods to solve inverse problems from imaging science. We do this by heavily exploring the singularity structure of images as a-priori information. We then develop a comprehensive analysis of how neural networks act on singularities using predominantly methods from the microlocal analysis. For analyzing the interaction of deep neural networks with singularities, we introduce a novel technique to compute the propagation of wavefront sets through convolutional residual neural networks (conv-ResNet). This is achieved in a two-fold manner: We first study the continuous case where the neural network is defined in an infinite-dimensional continuous space. This problem is tackled by using the structure of these networks as a sequential application of continuous convolutional operators and ReLU non-linearities and applying microlocal analysis techniques to track the propagation of the wavefront set through the layers. This then leads to the so-called \emph{microcanonical relation} that describes the propagation of the wavefront set under the action of such a neural network. Secondly, for studying real-world discrete problems, we digitize the necessary microlocal analysis methods via the digital shearlet transform. The key idea is the fact that the shearlet transform optimally represents Fourier integral operators hence such a discretization decays rapidly, allowing a finite approximation. Fourier integral operators play an important role in microlocal analysis, since it is well known that they preserve singularities on functions, and, in addition, they have a closed form microcanonical relation. Also, based on the newly developed theoretical analysis, we introduce a method that uses digital shearlet coefficients to compute the digital wavefront set of images by a convolutional neural network. Our approach is then used for a similar analysis of the microlocal behavior of the learned-primal dual architecture, which is formed by a sequence of conv-ResNet blocks. This architecture has shown state-of-the-art performance in inverse problem regularization, in particular, computed tomography reconstruction related to the Radon transform. Since the Radon operator is a Fourier integral operator, our microlocal techniques can be applied. Therefore, we can study with high precision the singularities propagation of this architecture. Aiming to empirically analyze our theoretical approach, we focus on the reconstruction of X-ray tomographic data. We approach this problem by using a task-adapted reconstruction framework, in which we combine the task of reconstruction with the task of computing the wavefront set of the original image as a-priori information. Our numerical results show superior performance with respect to current state-of-the-art tomographic reconstruction methods; hence we anticipate our work to also be a significant contribution to the biomedical imaging community.Tiefe neuronale Netze haben in letzter Zeit bei verschiedenen Bildverarbeitungsaufgaben Spitzenleistungen gezeigt. Zum Beispiel ist AlexNet heute der beste Bildklassifikator bei der ImageNet-Challenge. Sie sind auch sehr leistungsfaehig fue die Bildrekonstruktion, zum Beispiel liefert Deep Learning derzeit die besten Methoden fuer die CT-Rekonstruktion. Die meisten Bildgebungsprobleme wie die CT-Rekonstruktion sind schlecht gestellte inverse Probleme, die daher Regularisierungstechniken erfordern, die typischerweise auf vorherigen Informationen basieren. Auch aufgrund des menschlichen visuellen Systems sind Singularitaeten wie kantenartige Merkmale die bestimmenden Strukturen von Bildern. Dies fuehrt zu der Frage, wie man solche Informationen in einen Loeser eines inversen Problems in der Bildverarbeitung einbeziehen kann und wie tiefe neuronale Netze mit Singularitaeten arbeiten. Das Hauptforschungsthema dieser Arbeit ist die Einfuehrung theoretisch fundierter konzeptioneller Ansaetze zur Verwendung von tiefen neuronalen Netzen in Kombination mit modellbasierten Methoden zur Loesung inverser Probleme aus der Bildwissenschaft. Wir tun dies, indem wir die Singularitaetsstruktur von Bildern als Vorinformation intensiv erforschen. Dazu entwickeln wir eine umfassende Analyse, wie neuronale Netze auf Singularitaeten wirken, indem wir vorwiegend Methoden aus der mikrolokalen Analyse verwenden. Um die Interaktion von tiefen neuronalen Netzen mit Singularitaeten zu analysieren, fuehren wir eine neuartige Technik ein, um die Ausbreitung von Wellenfrontsaetzen mit Hilfe von Convolutional Residual neuronalen Netzen (Conv-ResNet) zu berechnen. Dies wird auf zweierlei Weise erreicht: Zunaechst untersuchen wir den kontinuierlichen Fall, bei dem das neuronale Netz in einem unendlich dimensionalen kontinuierlichen Raum definiert ist. Dieses Problem wird angegangen, indem wir die besondere Struktur dieser Netze als sequentielle Anwendung von kontinuierlichen Faltungsoperatoren und ReLU-Nichtlinearitaeten nutzen und mikrolokale Analyseverfahren anwenden, um die Ausbreitung einer Wellenfrontmenge durch die Schichten zu verfolgen. Dies fuehrt dann zu einer mikrokanonischen Beziehung, die die Ausbreitung der Wellenfrontmenge unter ihrer Wirkung beschreibt. Zweitens digitalisieren wir die notwendigen mikrolokalen Analysemethoden ueber die digitale Shearlet-Transformation, wobei die Digitalisierung fuer die Untersuchung realer Probleme notwendig ist. Die Schluesselidee ist die Tatsache, dass die Shearlet-Transformation Fourier-Integraloperatoren optimal repraesentiert, so dass eine solche Diskretisierung schnell abklingt und eine endliche Approximation ermoeglicht. Nebenbei stellen wir auch eine Methode vor, die digitale Shearlet-Koeffizienten verwendet, um den digitalen Wellenfrontsatz von Bildern durch ein Faltungsneuronales Netzwerk zu berechnen. Unser Ansatz wird dann fuer eine aehnliche Analyse fuer die gelernte primale-duale Architektur verwendet, die durch eine Sequenz von conv-ResNet-Bloecken gebildet wird. Diese Architektur hat bei der Rekonstruktion inverser Probleme, insbesondere bei der Rekonstruktion der Computertomographie im Zusammenhang mit der Radon-Transformation, Spitzenleistungen gezeigt. Da der Radon-Operator ein Fourier-Integraloperator ist, koennen unsere mikrolokalen Techniken angewendet werden. Um unseren theoretischen Ansatz numerisch zu analysieren, konzentrieren wir uns auf die Rekonstruktion von Roentgentomographiedaten. Wir naehern uns diesem Problem mit Hilfe eines aufgabenangepassten Rekonstruktionsrahmens, in dem wir die Aufgabe der Rekonstruktion mit der Aufgabe der Berechnung der Wellenfrontmenge des Originalbildes als Vorinformation kombinieren. Unsere numerischen Ergebnisse zeigen eine ueberragende Leistung, daher erwarten wir, dass dies auch ein interessanter Beitrag fuer die biomedizinische Bildgebung sein wird

Single-Molecule Carbon Nanotube Field-Effect Transistors for Genomic Applications

Single-molecule carbon nanotube-based field-effect transistors are promising all-electronic devices for probing interactions of various biological and chemical molecules at the single- molecule level. Such devices consist of point-functionalized carbon nanotubes which are charge sensitive in the vicinity of a generated defect on the nanotube sidewall. Of particular interest is the characterization of the kinetic rates and thermodynamics of DNA duplex formation through repeated association (hybridization) and dissociation (melting) events on timescales unmatched by conventional single-molecule methods. In this work, we study the kinetics and thermodynamics of DNA duplex formation with two types of single-walled nanotubes: CVD-grown and solution-processed. In both assessments, we are able to extract kinetic and thermodynamic parameters governing the hybridization and melting of DNA oligonucleotides. In the latter case, devices are spun onto a wafer surface from an organic suspension, revealing consistent electrical characteristics. Significant effort is made to expand this work to wafer-level, in an effort to make the fabrication manufacturable

Higher level techniques for the artistic rendering of images and video

EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Polymer Microsystems for the Enrichment of Circulating Tumor Cells and their Clinical Demonstration

Cancer research is centered on the discovery of new biomarkers that could unlock the obscurities behind the mechanisms that cause cancer or those associated with its spread (i.e., metastatic disease). Circulating tumor cells (CTCs) have emerged as attractive biomarkers for the management of many cancer-related diseases due primarily to the ease of securing them from a simple blood draw. However, their rarity (~1 CTC per mL of whole blood) makes enrichment analytically challenging. Microfluidic systems are viewed as exquisite platforms for the clinical analysis of CTCs due to their ability to be used in an automated fashion, minimizing sample loss and contamination. This has formed the basis of the reported research, which focused on the development of microfluidic systems for CTC analysis. The system reported herein consisted of a modular design and targeted the analysis of CTCs using pancreatic ductal adenocarcinoma (PDAC) as the model disease for determining the utility of the system. The system was composed of 3 functional modules; (i) a thermoplastic CTC selection module consisting of high aspect ratio (30 µm x 150 µm) channels; (ii) an impedance sensor module for label-less CTC counting; and (iii) a staining and imaging module for phenotype identification of selected CTCs. The system could exhaustively process 7.5 mL of blood in \u3c45 min with CTC recoveries \u3e90% directly from whole blood. In addition, significantly reduced assay turnaround times (8 h to 1.5 h) was demonstrated. We also show the ability to detect KRAS gene mutations from CTCs enriched by the microfluidic system. As a proof-of-concept, the ability to identify KRAS point mutations using a PCR/LDR/CE assay from as low as 10 CTCs enriched by the integrated microfluidic system was demonstrated. Finally, the clinical utility of the polymer-based microfluidic device for the analysis of circulating multiple myeloma cells (CMMCs) was demonstrated as well. Parameters such as translational velocity and recovery of CMMCs were optimized and found to be 1.1 mm/s and 71%, respectively. Also demonstrated was on-chip immunophenotyping and clonal testing of CMMCs, which has been reported to be prognostically significant. Further, a pilot study involving 26 patients was performed using the polymer microfluidic device with the aim of correlating the number of CMMCs with disease activity. An average of 347 CMMCs/mL of whole blood was recovered from blood volumes of approximately 0.5 mL

Gestural patterns: a new method of printed textile design using motion capture technology

The aim of this research is to develop a new method, Hybrid Printing System (HPS) to explore digital craft methods to create surface patterns for printed textile design. This novel method of creating ‘handcrafted’ prints is a result of the integration of two technologies such as motion-capture (MOCAP) and digital textile printing (DTP). The research towards such an innovation required a current, historical, contextual and experimental study of use of motion capture in Art &Design. The research contextualises the hand and its relationship to digital crafting methods in printed textile design, the digital medium and the process of audience participation in printed textile design to create a new conceptual framework balanced in practice and theory. The practical research then develops three new methods of motion capture such as, motion tracing, motion sensing and motion tracking to generate gestural motifs and gestural patterns. This thesis and the accompanying set of experimental work demonstrates that HPS culminates in developing new aesthetics through a new mode of creation in a new medium, which will impact the user, the designer and the product as a part of the cyclical process. HPS is an advancement of printed textile design, centred in active participation of its audience at the generative stage of design. This results in a shifting role of a designer and subverts the current model of printed textile design practice. HPS is a democratic design process where the participants design for themselves, have their own voice, which induces a sense of community, togetherness and harmony in the creative process

Are bicultural bonobos able to recognize iconic representations and produce referential signs in human cultural terms?

This is a visual communication study of the graphic sign making and recognition capabilities of bonobo-chimpanzees (Pan paniscus). The study applies principles of semiotics to assess the bonobo-chimpanzee\u27s potential for meaningful communication through figurative signs. This research sees a distinction between art (expressive) and visual communication (informative), with emphasis on meaningful information exchange. Three tests are conducted. Test 1 involved the production of recognizable and representational imagery by bonobo-chimpanzees, who were asked in a standardized testing format to depict subjects from photographs. Test 2 was an assessment of their ability to recognize extremely simplified iconic signs to their respective photographs and remembering their intended meaning by selecting them from a series of two, three, and six different photographs. In Test 3, the bonobo-chimpanzees were asked for verification of each of their iconic drawings by matching to the photo each drawing from which it was depicted. These tests were statistically calculated for their significance (success rate). The capabilities needed to accomplish such tests are visual literacy and high cognition allowing for graphic representation and interpretation---an assembly of traits thought unique only to humans. These are capabilities that bonobo-chimpanzees have never shown empirically prior to this research. The results of this study show that bonobo-chimpanzees do have representational mark-making capabilities. They can recognize extremely simplified icons from photographs, and their marks have referential meaning to them across time. These results were statistically significant in Kanzi (bonobo-chimpanzee) and approaching significance in Pan Banisha (bonobo-chimpanzee). These results are deserving of a continued multidisciplinary approach to Hominid interspecies communication