WPU-Net: Boundary Learning by Using Weighted Propagation in Convolution Network

Deep learning has driven a great progress in natural and biological image processing. However, in material science and engineering, there are often some flaws and indistinctions in material microscopic images induced from complex sample preparation, even due to the material itself, hindering the detection of target objects. In this work, we propose WPU-net that redesigns the architecture and weighted loss of U-Net, which forces the network to integrate information from adjacent slices and pays more attention to the topology in boundary detection task. Then, the WPU-net is applied into a typical material example, i.e., the grain boundary detection of polycrystalline material. Experiments demonstrate that the proposed method achieves promising performance and outperforms state-of-the-art methods. Besides, we propose a new method for object tracking between adjacent slices, which can effectively reconstruct 3D structure of the whole material. Finally, we present a material microscopic image dataset with the goal of advancing the state-of-the-art in image processing for material science.Comment: technical repor

Propagated image Segmentation Using Edge-Weighted Centroidal Voronoi Tessellation based Methods

Propagated image segmentation is the problem of utilizing the existing segmentation of an image for obtaining a new segmentation of, either a neighboring image in a sequence, or the same image but in different scales. We name these two cases as the inter-image propagation and the intra-image propagation respectively. The inter-image propagation is particularly important to material science, where efficient and accurate segmentation of a sequence of 2D serial-sectioned images of 3D material samples is an essential step to understand the underlying micro-structure and related physical properties. For natural images with objects in different scales, the intra-image propagation, where segmentations are propagated from the finest scale to coarser scales, is able to better capture object boundaries than single-shot segmentations on a fixed image scale. In this work, we first propose an inter-image propagation method named Edge- Weighted Centroid Voronoi Tessellation with Propagation of Consistency Constraint (CCEWCVT) to effectively segment material images. CCEWCVT segments an image sequence by repeatedly propagating a 2D segmentation from one slice to another, and in each step of this propagation, we apply the proposed consistency constraint in the pixel clustering process such that stable structures identified from the previous slice can be well-preserved. We further propose a non-rigid transformation based association method to find the correspondence of propagated stable structures in the next slice when the inter-image distance becomes large. We justify the effectiveness of the proposed CCEWCVT method on 3D material image sequences, and we compare its performance against several state-of-the-art 2D, 3D, propagated segmentation methods. Then for the intra-image propagation, we propose a superpixel construction method named Hierarchical Edge-Weighted Centroidal Voronoi Tessellation (HEWCVT) to accurately capture object boundaries in natural images. We model the problem as a multilevel clustering process: superpixels in one level are clustered to obtain larger size superpixels in the next level. The clustering energy involves both color similarities and the proposed boundary smoothness of superpixels. We further extend HEWCVT to obtain supervoxels on 3D images or videos. Both quantitative and qualitative evaluation results on several standard datasets show that the proposed HEWCVT method achieves superior or comparable performances to other state-of-the-art methods. v

Multi-Label Segmentation Propagation for Materials Science Images Incorporating Topology and Interactivity

Segmentation propagation is the problem of transferring the segmentation of an image to a neighboring image in a sequence. This problem is of particular importance to materials science, where the accurate segmentation of a series of 2D serial-sectioned images of multiple, contiguous 3D structures has important applications. Such structures may have prior-known shape, appearance, and/or topology among the underlying structures which can be considered to improve segmentation accuracy. For example, some materials images may have structures with a specific shape or appearance in each serial section slice, which only changes minimally from slice to slice; and some materials may exhibit specific topology which constrains their structure or neighboring relations. In this work, we develop a framework for materials image segmentation that segments a variety of materials image types by repeatedly propagating a 2D segmentation from one slice to another, and we formulate each step of this propagation as an optimal labeling problem that can be efficiently solved using the graph-cut algorithm. During this propagation, we propose novel strategies to enforce the shape, appearance, and topology of the segmented structures, as well as handling local topology inconsistency. Most recent works on topology-constrained image segmentation focus on binary segmentation, where the topology is often described by the connectivity of both foreground and background. We develop a new multi-labeling approach to enforce topology in multiple-label image segmentation. In this case, we not only require each segment to be a connected region (intra-segment topology), but also require specific adjacency relations between each pair of segments (inter-segment topology). Finally, we integrate an interactive approach into the proposed framework that improves the segmentation by allowing minimal and simplistic human annotations. We justify the effectiveness of the proposed framework by testing it on various 3D materials images, and we compare its performance against several existing image segmentation methods

Autonomous robotic system for thermographic detection of defects in upper layers of carbon fiber reinforced polymers

Carbon Fiber Reinforced Polymers (CFRPs) are composites whose interesting properties, like high strength-to-weight ratio and rigidity, are of interest in many industrial fields. Many defects affecting their production process are due to the wrong distribution of the thermosetting polymer in the upper layers. In this work, they are effectively and efficiently detected by automatically analyzing the thermographic images obtained by Pulsed Phase Thermography (PPT) and comparing them with a defect-free reference. The flash lamp and infrared camera needed by PPT are mounted on an industrial robot so that surfaces of CFRP automotive components, car side blades in our case, can be inspected in a series of static tests. The thermographic image analysis is based on local contrast adjustment via UnSharp Masking (USM) and takes also advantage of the high level of knowledge of the entire system provided by the calibration procedures. This system could replace manual inspection leading to a substantial increase in efficiency

Three--dimensional medical imaging: Algorithms and computer systems

This paper presents an introduction to the field of three-dimensional medical imaging It presents medical imaging terms and concepts, summarizes the basic operations performed in three-dimensional medical imaging, and describes sample algorithms for accomplishing these operations. The paper contains a synopsis of the architectures and algorithms used in eight machines to render three-dimensional medical images, with particular emphasis paid to their distinctive contributions. It compares the performance of the machines along several dimensions, including image resolution, elapsed time to form an image, imaging algorithms used in the machine, and the degree of parallelism used in the architecture. The paper concludes with general trends for future developments in this field and references on three-dimensional medical imaging

A discrete graph Laplacian for signal processing

In this thesis we exploit diffusion processes on graphs to effect two fundamental problems of image processing: denoising and segmentation. We treat these two low-level vision problems on the pixel-wise level under a unified framework: a graph embedding. Using this framework opens us up to the possibilities of exploiting recently introduced algorithms from the semi-supervised machine learning literature. We contribute two novel edge-preserving smoothing algorithms to the literature. Furthermore we apply these edge-preserving smoothing algorithms to some computational photography tasks. Many recent computational photography tasks require the decomposition of an image into a smooth base layer containing large scale intensity variations and a residual layer capturing fine details. Edge-preserving smoothing is the main computational mechanism in producing these multi-scale image representations. We, in effect, introduce a new approach to edge-preserving multi-scale image decompositions. Where as prior approaches such as the Bilateral filter and weighted-least squares methods require multiple parameters to tune the response of the filters our method only requires one. This parameter can be interpreted as a scale parameter. We demonstrate the utility of our approach by applying the method to computational photography tasks that utilise multi-scale image decompositions. With minimal modification to these edge-preserving smoothing algorithms we show that we can extend them to produce interactive image segmentation. As a result the operations of segmentation and denoising are conducted under a unified framework. Moreover we discuss how our method is related to region based active contours. We benchmark our proposed interactive segmentation algorithms against those based upon energy-minimisation, specifically graph-cut methods. We demonstrate that we achieve competitive performance

Novel Texture-based Probabilistic Object Recognition and Tracking Techniques for Food Intake Analysis and Traffic Monitoring

More complex image understanding algorithms are increasingly practical in a host of emerging applications. Object tracking has value in surveillance and data farming; and object recognition has applications in surveillance, data management, and industrial automation. In this work we introduce an object recognition application in automated nutritional intake analysis and a tracking application intended for surveillance in low quality videos. Automated food recognition is useful for personal health applications as well as nutritional studies used to improve public health or inform lawmakers. We introduce a complete, end-to-end system for automated food intake measurement. Images taken by a digital camera are analyzed, plates and food are located, food type is determined by neural network, distance and angle of food is determined and 3D volume estimated, the results are cross referenced with a nutritional database, and before and after meal photos are compared to determine nutritional intake. We compare against contemporary systems and provide detailed experimental results of our system\u27s performance. Our tracking systems consider the problem of car and human tracking on potentially very low quality surveillance videos, from fixed camera or high flying \acrfull{uav}. Our agile framework switches among different simple trackers to find the most applicable tracker based on the object and video properties. Our MAPTrack is an evolution of the agile tracker that uses soft switching to optimize between multiple pertinent trackers, and tracks objects based on motion, appearance, and positional data. In both cases we provide comparisons against trackers intended for similar applications i.e., trackers that stress robustness in bad conditions, with competitive results

Reducing the Burden of Aerial Image Labelling Through Human-in-the-Loop Machine Learning Methods

This dissertation presents an introduction to human-in-the-loop deep learning methods for remote sensing applications. It is motivated by the need to decrease the time spent by volunteers on semantic segmentation of remote sensing imagery. We look at two human-in-the-loop approaches of speeding up the labelling of the remote sensing data: interactive segmentation and active learning. We develop these methods specifically in response to the needs of the disaster relief organisations who require accurately labelled maps of disaster-stricken regions quickly, in order to respond to the needs of the affected communities. To begin, we survey the current approaches used within the field. We analyse the shortcomings of these models which include outputs ill-suited for uploading to mapping databases, and an inability to label new regions well, when the new regions differ from the regions trained on. The methods developed then look at addressing these shortcomings. We first develop an interactive segmentation algorithm. Interactive segmentation aims to segment objects with a supervisory signal from a user to assist the model. Work within interactive segmentation has focused largely on segmenting one or few objects within an image. We make a few adaptions to allow an existing method to scale to remote sensing applications where there are tens of objects within a single image that needs to be segmented. We show a quantitative improvements of up to 18% in mean intersection over union, as well as qualitative improvements. The algorithm works well when labelling new regions, and the qualitative improvements show outputs more suitable for uploading to mapping databases. We then investigate active learning in the context of remote sensing. Active learning looks at reducing the number of labelled samples required by a model to achieve an acceptable performance level. Within the context of deep learning, the utility of the various active learning strategies developed is uncertain, with conflicting results within the literature. We evaluate and compare a variety of sample acquisition strategies on the semantic segmentation tasks in scenarios relevant to disaster relief mapping. Our results show that all active learning strategies evaluated provide minimal performance increases over a simple random sample acquisition strategy. However, we present analysis of the results illustrating how the various strategies work and intuition of when certain active learning strategies might be preferred. This analysis could be used to inform future research. We conclude by providing examples of the synergies of these two approaches, and indicate how this work, on reducing the burden of aerial image labelling for the disaster relief mapping community, can be further extended