Shape and data-driven texture segmentation using local binary patterns

We propose a shape and data driven texture segmentation method using local binary patterns (LBP) and active contours. In particular, we pass textured images through a new LBP-based filter, which produces non-textured images. In this “filtered” domain each textured region of the original image exhibits a characteristic intensity distribution. In this domain we pose the segmentation problem as an optimization problem in a Bayesian framework. The cost functional contains a data-driven term, as well as a term that brings in information about the shapes of the objects to be segmented. We solve the optimization problem using level set-based active contours. Our experimental results on synthetic and real textures demonstrate the effectiveness of our approach in segmenting challenging textures as well as its robustness to missing data and occlusions

Data-driven Flood Emulation: Speeding up Urban Flood Predictions by Deep Convolutional Neural Networks

Computational complexity has been the bottleneck of applying physically-based simulations on large urban areas with high spatial resolution for efficient and systematic flooding analyses and risk assessments. To address this issue of long computational time, this paper proposes that the prediction of maximum water depth rasters can be considered as an image-to-image translation problem where the results are generated from input elevation rasters using the information learned from data rather than by conducting simulations, which can significantly accelerate the prediction process. The proposed approach was implemented by a deep convolutional neural network trained on flood simulation data of 18 designed hyetographs on three selected catchments. Multiple tests with both designed and real rainfall events were performed and the results show that the flood predictions by neural network uses only 0.5 % of time comparing with physically-based approaches, with promising accuracy and ability of generalizations. The proposed neural network can also potentially be applied to different but relevant problems including flood predictions for urban layout planning

Grounding semantics in robots for Visual Question Answering

In this thesis I describe an operational implementation of an object detection and description system that incorporates in an end-to-end Visual Question Answering system and evaluated it on two visual question answering datasets for compositional language and elementary visual reasoning

Colour Texture analysis

This chapter presents a novel and generic framework for image segmentation using a compound image descriptor that encompasses both colour and texture information in an adaptive fashion. The developed image segmentation method extracts the texture information using low-level image descriptors (such as the Local Binary Patterns (LBP)) and colour information by using colour space partitioning. The main advantage of this approach is the analysis of the textured images at a micro-level using the local distribution of the LBP values, and in the colour domain by analysing the local colour distribution obtained after colour segmentation. The use of the colour and texture information separately has proven to be inappropriate for natural images as they are generally heterogeneous with respect to colour and texture characteristics. Thus, the main problem is to use the colour and texture information in a joint descriptor that can adapt to the local properties of the image under analysis. We will review existing approaches to colour and texture analysis as well as illustrating how our approach can be successfully applied to a range of applications including the segmentation of natural images, medical imaging and product inspection