Spectral-spatial classification of n-dimensional images in real-time based on segmentation and mathematical morphology on GPUs

The objective of this thesis is to develop efficient schemes for spectral-spatial n-dimensional image classification. By efficient schemes, we mean schemes that produce good classification results in terms of accuracy, as well as schemes that can be executed in real-time on low-cost computing infrastructures, such as the Graphics Processing Units (GPUs) shipped in personal computers. The n-dimensional images include images with two and three dimensions, such as images coming from the medical domain, and also images ranging from ten to hundreds of dimensions, such as the multiand hyperspectral images acquired in remote sensing. In image analysis, classification is a regularly used method for information retrieval in areas such as medical diagnosis, surveillance, manufacturing and remote sensing, among others. In addition, as the hyperspectral images have been widely available in recent years owing to the reduction in the size and cost of the sensors, the number of applications at lab scale, such as food quality control, art forgery detection, disease diagnosis and forensics has also increased. Although there are many spectral-spatial classification schemes, most are computationally inefficient in terms of execution time. In addition, the need for efficient computation on low-cost computing infrastructures is increasing in line with the incorporation of technology into everyday applications. In this thesis we have proposed two spectral-spatial classification schemes: one based on segmentation and other based on wavelets and mathematical morphology. These schemes were designed with the aim of producing good classification results and they perform better than other schemes found in the literature based on segmentation and mathematical morphology in terms of accuracy. Additionally, it was necessary to develop techniques and strategies for efficient GPU computing, for example, a block–asynchronous strategy, resulting in an efficient implementation on GPU of the aforementioned spectral-spatial classification schemes. The optimal GPU parameters were analyzed and different data partitioning and thread block arrangements were studied to exploit the GPU resources. The results show that the GPU is an adequate computing platform for on-board processing of hyperspectral information

Optical flow estimation via steered-L1 norm

Global variational methods for estimating optical flow are among the best performing methods due to the subpixel accuracy and the ‘fill-in’ effect they provide. The fill-in effect allows optical flow displacements to be estimated even in low and untextured areas of the image. The estimation of such displacements are induced by the smoothness term. The L1 norm provides a robust regularisation term for the optical flow energy function with a very good performance for edge-preserving. However this norm suffers from several issues, among these is the isotropic nature of this norm which reduces the fill-in effect and eventually the accuracy of estimation in areas near motion boundaries. In this paper we propose an enhancement to the L1 norm that improves the fill-in effect for this smoothness term. In order to do this we analyse the structure tensor matrix and use its eigenvectors to steer the smoothness term into components that are ‘orthogonal to’ and ‘aligned with’ image structures. This is done in primal-dual formulation. Results show a reduced end-point error and improved accuracy compared to the conventional L1 norm

Optical flow estimation via steered-L1 norm

Global variational methods for estimating optical flow are among the best performing methods due to the subpixel accuracy and the ‘fill-in’ effect they provide. The fill-in effect allows optical flow displacements to be estimated even in low and untextured areas of the image. The estimation of such displacements are induced by the smoothness term. The L1 norm provides a robust regularisation term for the optical flow energy function with a very good performance for edge-preserving. However this norm suffers from several issues, among these is the isotropic nature of this norm which reduces the fill-in effect and eventually the accuracy of estimation in areas near motion boundaries. In this paper we propose an enhancement to the L1 norm that improves the fill-in effect for this smoothness term. In order to do this we analyse the structure tensor matrix and use its eigenvectors to steer the smoothness term into components that are ‘orthogonal to’ and ‘aligned with’ image structures. This is done in primal-dual formulation. Results show a reduced end-point error and improved accuracy compared to the conventional L1 norm

Lagrangian coherent structures and trajectory similarity: two important tools for scientific visualization

This thesis studies the computation and visualization of Lagrangian coherent structures (LCS), an emerging technique for analyzing time-varying velocity fields (e.g. blood vessels and airflows), and the measure of similarity for trajectories (e.g. hurricane paths). LCS surfaces and trajectory-based techniques (e.g. trajectory clustering) are complementary to each other for visualization, while velocity fields and trajectories are two important types of scientific data, which are more and more accessible by virtue of the technology development for both data collection and numerical simulation. A key step for LCS computation is tracing the paths of collections of particles through a flow field. When a flow field is interpolated from the nodes of an unstructured mesh, the process of advecting a particle must first find which cell in the unstructured mesh contains the particle. Since the paths of nearby particles often diverge, the parallelization of particle advection quickly leads to incoherent memory accesses of the unstructured mesh. We have developed a new block advection GPU approach that reorganizes particles into spatially coherent bundles as they follow their advection paths, which greatly improves memory coherence and thus shared-memory GPU performance. This approach works best for flows that meet the CFL criterion on unstructured meshes of uniformly sized elements, small enough to fit at least two timesteps in GPU memory. LCS surfaces provide insight into unsteady fluid flow, but their construction has posed many challenges. These structures can be characterized as ridges of a field, but their local definition utilizes an ambiguous eigenvector direction that can point in one of two directions, and its ambiguity can lead to noise and other problems. We overcome these issues with an application of a global ridge definition, applied using the hierarchical watershed transformation. We show results on a mathematical flow model and a simulated vascular flow dataset indicating the watershed method produces less noisy structures. Trajectory similarity has been shown to be a powerful tool for visualizing and analyzing trajectories. In this paper we propose a novel measure of trajectory similarity using both spatial and directional information. The similarity is asymmetric, bounded within [0,1], affine-invariant, and efficiently computed. Asymmetric mappings between a pair of trajectories can be derived from this similarity. Experimental results demonstrate that the measure is better than existing measures in both similarity scores and trajectory mappings. The measure also inspires a simple similarity-based clustering method for effectivly visualizing a large number of trajectories, which outperforms the state-of-the-art model-based clustering method (VFKM)

Programmable Image-Based Light Capture for Previsualization

Previsualization is a class of techniques for creating approximate previews of a movie sequence in order to visualize a scene prior to shooting it on the set. Often these techniques are used to convey the artistic direction of the story in terms of cinematic elements, such as camera movement, angle, lighting, dialogue, and character motion. Essentially, a movie director uses previsualization (previs) to convey movie visuals as he sees them in his minds-eye . Traditional methods for previs include hand-drawn sketches, Storyboards, scaled models, and photographs, which are created by artists to convey how a scene or character might look or move. A recent trend has been to use 3D graphics applications such as video game engines to perform previs, which is called 3D previs. This type of previs is generally used prior to shooting a scene in order to choreograph camera or character movements. To visualize a scene while being recorded on-set, directors and cinematographers use a technique called On-set previs, which provides a real-time view with little to no processing. Other types of previs, such as Technical previs, emphasize accurately capturing scene properties but lack any interactive manipulation and are usually employed by visual effects crews and not for cinematographers or directors. This dissertation\u27s focus is on creating a new method for interactive visualization that will automatically capture the on-set lighting and provide interactive manipulation of cinematic elements to facilitate the movie maker\u27s artistic expression, validate cinematic choices, and provide guidance to production crews. Our method will overcome the drawbacks of the all previous previs methods by combining photorealistic rendering with accurately captured scene details, which is interactively displayed on a mobile capture and rendering platform. This dissertation describes a new hardware and software previs framework that enables interactive visualization of on-set post-production elements. A three-tiered framework, which is the main contribution of this dissertation is; 1) a novel programmable camera architecture that provides programmability to low-level features and a visual programming interface, 2) new algorithms that analyzes and decomposes the scene photometrically, and 3) a previs interface that leverages the previous to perform interactive rendering and manipulation of the photometric and computer generated elements. For this dissertation we implemented a programmable camera with a novel visual programming interface. We developed the photometric theory and implementation of our novel relighting technique called Symmetric lighting, which can be used to relight a scene with multiple illuminants with respect to color, intensity and location on our programmable camera. We analyzed the performance of Symmetric lighting on synthetic and real scenes to evaluate the benefits and limitations with respect to the reflectance composition of the scene and the number and color of lights within the scene. We found that, since our method is based on a Lambertian reflectance assumption, our method works well under this assumption but that scenes with high amounts of specular reflections can have higher errors in terms of relighting accuracy and additional steps are required to mitigate this limitation. Also, scenes which contain lights whose colors are a too similar can lead to degenerate cases in terms of relighting. Despite these limitations, an important contribution of our work is that Symmetric lighting can also be leveraged as a solution for performing multi-illuminant white balancing and light color estimation within a scene with multiple illuminants without limits on the color range or number of lights. We compared our method to other white balance methods and show that our method is superior when at least one of the light colors is known a priori

Computational Analysis of Tumour Microenvironment in mIHC Stained Diffuse Glioma Samples

Healthcare is a sector that has been notoriously stagnant in digital innovation, nevertheless its transformation is imminent. Digital pathology is a field that is being accentuated in light of recent technological development. With capacity to conduct high-resolution tissue imaging and managing output digitally, advanced image analysis and Machine Learning can be subsequently applied. These methods provide means to for instance automating segmentation of region-of-interests, diagnosis and knowledge discovery. Brain malignancies are particularly dire with a high fatality rate and relatively high occurrence in children. Diffuse gliomas are a subtype of brain tumours whose biological behavior range from very indolent to extremely aggressive, which is reflected in grading I - IV. The brain tumour micro-environment (TME) --- local area surrounding cancerous cells with a plethora of immune cells and other structures in interaction --- has emerged as a critical regulator of brain tumour progression. Researchers are interested in immunotherapeutic treatment of brain cancer, since modern approaches are insufficient in treatment of especially the most aggressive tumours. Additionally, the TME is rendered difficult to understand. Multiplex Immunohistochemistry (mIHC) is a novel approach in effectively mapping spatial distribution of cell types in tissue samples using multiple antibodies. In this thesis, we investigate the TME in diffuse glioma mIHC samples for three patient cases with 2-3 differing tumour grades per patient. From the 18 possibilities we selected 6 antigens (markers) of interest for further analysis. In particular, we are interested in how relative proportion of positive antigens and mean distance to nearest blood vessel vary for our selected markers in tumour progression. In order to acquire desired properties, we register each corresponding image, detect nuclei, segment cells and extract structured data from region channel intensities along with their location and distance to nearest blood vessel. Our primary finding is that M2-macrophage and T cell occurrence proportions as well as their mean distance to blood vessel grow with increasing tumour grade. The results could suggest that aforementioned cell types are of low quantity in near vicinity of blood vessels in low tumour grades, and conversely with higher quantities and more homogeneous distribution in aggressive tumours. Despite the several potential error sources and non-standardized processes in the pipeline between tissue extraction and image analysis, our results support pre-existing knowledge in that M2-macrophage proportion has a positive correlation with tumour grade.Terveydenhuollon digitaalinen kehitys on ollut hidasliikkeistä muihin sektoreihin verrattuna. Tästä huolimatta, terveydenhuollon digitaalinen muunnos on välitön ja asiaan liittyvä tutkimus jatkuvaa. Digitaalinen patologia on ala, joka viime aikaisen teknologisen kehityksen myötä on korostunut. Kudoskuvantaminen korkealla resoluutiolla ja näytteiden digitaalinen hallinta on mahdollistanut kehittyneen kuvanalysiin sekä koneoppimisen soveltamisen. Nämä metodit luovat keinot esimerkiksi biologisesti merkittävien alueiden segmentointiin, diagnoosiin ja uuden tieteellisen tiedon tuottamiseen. Aivokasvaimet ovat järkyttäviä, sillä tapauskuolleisuus ja esiintymä nuorissa ovat suhteellisen korkealla. Diffuusigliomat ovat aivokasvainten alatyyppi, jonka sisältämät kasvaimet luokitellaan niiden aggressiivisuuden perusteella eri graduksiin väliltä I - IV. Kasvaimen mikroympäristö (TME), eli syöpäsolujen paikallinen ympäristö sisältäen mm. runsaasti immuunipuolustuksen soluja vuorovaikutuksessa, on osoittautunut merkittäväksi tekijäksi kasvaimen kehityksen suhteen. Aivosyövän tutkimus painottuu immunoterapeuttisiin ratkaisuihin, sillä nykyiset hoitomuodot eivät ole tarpeeksi tehokkaita etenkään kaikista aggressiivisimpien kasvainten hoidossa. Lisäksi mikroympäristö voi olla vaikea ymmärtää. Monikanavainen immunohistokemiallinen värjäys (mIHC) on uudenlainen lähestymistapa solutyyppien spatiaalijakauman kartoittamiseen kudosnäytteissä tehokkaasti hyödyntäen useita vasta-aineita. Tässä opinnäytetyössä tutkitaan diffuusigliooma mIHC-näytteitä kolmelle potilastapaukselle. Jokaista potilasta kohti on 2-3 näytettä eri kasvainlaaduista ja yhteensä 18 mIHC-kanavaa per näyte, joista 6 otettiin tarkasteluun. Tarkalleen ottaen, solutyyppien aktivaatioiden osuudet positiivisten antigeenien perusteella ja keskimääräinen etäisyys lähimpään verisuoneen jokaista ryhmää kohti lasketaan eri kasvaimen laaduissa. Tavoitteen saavuttamiseksi näytteitä vastaavat kuvat rekisteröidään, tumat tunnistetaan, solualueet segmentoidaan ja kerätään jäsenneltyä tietoa alueiden intensiteettikanavista mukaan lukien sijainti ja sijaintia vastaava etäisyys lähimpään verisuoneen. Pääasiallinen löytö on, että M2-makrofagien ja T-solujen suhteelliset osuudet sekä keskimääräinen etäisyys lähimpään verisuoneen nousevat kasvaimen ollessa aggressiivisempi. Tulokset saattavat ehdottaa, että edellämainitut solutyypit ovat vähäisiä ja verisuonten lähellä kun kasvain on hyvänlaatuinen ja vastaavasti suurimilla osuuksilla ja enemmän homogeenisesti jakautunut kun kasvain on aggressiivisempi. Useista virhelähteistä ja kudosanalyysin liittyvistä ei-standardisoiduista prosesseista huolimatta, tuloksemme tukevat ennaltatiedettyä tietoa siitä, että M2-makrofagien osuudella on positiivinen korrelaatio kasvaimen laatuun

Advancements and Breakthroughs in Ultrasound Imaging

Ultrasonic imaging is a powerful diagnostic tool available to medical practitioners, engineers and researchers today. Due to the relative safety, and the non-invasive nature, ultrasonic imaging has become one of the most rapidly advancing technologies. These rapid advances are directly related to the parallel advancements in electronics, computing, and transducer technology together with sophisticated signal processing techniques. This book focuses on state of the art developments in ultrasonic imaging applications and underlying technologies presented by leading practitioners and researchers from many parts of the world

IMPROVE - Innovative Modelling Approaches for Production Systems to Raise Validatable Efficiency

This open access work presents selected results from the European research and innovation project IMPROVE which yielded novel data-based solutions to enhance machine reliability and efficiency in the fields of simulation and optimization, condition monitoring, alarm management, and quality prediction

Image Processing Using FPGAs

This book presents a selection of papers representing current research on using field programmable gate arrays (FPGAs) for realising image processing algorithms. These papers are reprints of papers selected for a Special Issue of the Journal of Imaging on image processing using FPGAs. A diverse range of topics is covered, including parallel soft processors, memory management, image filters, segmentation, clustering, image analysis, and image compression. Applications include traffic sign recognition for autonomous driving, cell detection for histopathology, and video compression. Collectively, they represent the current state-of-the-art on image processing using FPGAs