Accelerated hardware video object segmentation: From foreground detection to connected components labelling

This is the preprint version of the Article - Copyright @ 2010 ElsevierThis paper demonstrates the use of a single-chip FPGA for the segmentation of moving objects in a video sequence. The system maintains highly accurate background models, and integrates the detection of foreground pixels with the labelling of objects using a connected components algorithm. The background models are based on 24-bit RGB values and 8-bit gray scale intensity values. A multimodal background differencing algorithm is presented, using a single FPGA chip and four blocks of RAM. The real-time connected component labelling algorithm, also designed for FPGA implementation, run-length encodes the output of the background subtraction, and performs connected component analysis on this representation. The run-length encoding, together with other parts of the algorithm, is performed in parallel; sequential operations are minimized as the number of run-lengths are typically less than the number of pixels. The two algorithms are pipelined together for maximum efficiency

Hardware dedicado para sistemas empotrados de visión

La constante evolución de las Tecnologías de la Información y las Comunicaciones no solo ha permitido que más de la mitad de la población mundial esté actualmente interconectada a través de Internet, sino que ha sido el caldo de cultivo en el que han surgido nuevos paradigmas, como el ‘Internet de las cosas’ (IoT) o la ‘Inteligencia ambiental’ (AmI), que plantean la necesidad de interconectar objetos con distintas funcionalidades para lograr un entorno digital, sensible y adaptativo, que proporcione servicios de muy distinta índole a sus usuarios. La consecución de este entorno requiere el desarrollo de dispositivos electrónicos de bajo coste que, con tamaño y peso reducido, sean capaces de interactuar con el medio que los rodea, operar con máxima autonomía y proporcionar un elevado nivel de inteligencia. La funcionalidad de muchos de estos dispositivos incluirá la capacidad para adquirir, procesar y transmitir imágenes, extrayendo, interpretando o modificando la información visual que resulte de interés para una determinada aplicación. En el marco de este desafío surge la presente Tesis Doctoral, cuyo eje central es el desarrollo de hardware dedicado para la implementación de algoritmos de procesamiento de imágenes y secuencias de vídeo usados en sistemas empotrados de visión. El trabajo persigue una doble finalidad. Por una parte, la búsqueda de soluciones que, por sus prestaciones y rendimiento, puedan ser incorporadas en sistemas que satisfagan las estrictas exigencias de funcionalidad, tamaño, consumo de energía y velocidad de operación demandadas por las nuevas aplicaciones. Por otra, el diseño de una serie de bloques funcionales implementados como módulos de propiedad intelectual, que permitan aliviar la carga computacional de las unidades de procesado de los sistemas en los que se integren. En la Tesis se proponen soluciones específicas para la implementación de dos tipos de operaciones habitualmente presentes en muchos sistemas de visión artificial: la sustracción de fondo y el etiquetado de componentes conexos. Las distintas alternativas surgen como consecuencia de aplicar una adecuada relación de compromiso entre funcionalidad y coste, entendiendo este último criterio en términos de recursos de cómputo, velocidad de operación y potencia consumida, lo que permite cubrir un amplio espectro de aplicaciones. En algunas de las soluciones propuestas se han utilizado además, técnicas de inferencia basadas en Lógica Difusa con idea de mejorar la calidad de los sistemas de visión resultantes. Para la realización de los diferentes bloques funcionales se ha seguido una metodología de diseño basada en modelos, que ha permitido la realización de todo el ciclo de desarrollo en un único entorno de trabajo. Dicho entorno combina herramientas informáticas que facilitan las etapas de codificación algorítmica, diseño de circuitos, implementación física y verificación funcional y temporal de las distintas alternativas, acelerando con ello todas las fases del flujo de diseño y posibilitando una exploración más eficiente del espacio de posibles soluciones. Asimismo, con el objetivo de demostrar la funcionalidad de las distintas aportaciones de esta Tesis Doctoral, algunas de las soluciones propuestas han sido integradas en sistemas de vídeo reales, que emplean buses estándares de uso común. Los dispositivos seleccionados para llevar a cabo estos demostradores han sido FPGAs y SoPCs de Xilinx, ya que sus excelentes propiedades para el prototipado y la construcción de sistemas que combinan componentes software y hardware, los convierten en candidatos ideales para dar soporte a la implementación de este tipo de sistemas.The continuous evolution of the Information and Communication Technologies (ICT), not only has allowed more than half of the global population to be currently interconnected through Internet, but it has also been the breeding ground for new paradigms such as Internet of Things (ioT) or Ambient Intelligence (AmI). These paradigms expose the need of interconnecting elements with different functionalities in order to achieve a digital, sensitive, adaptive and responsive environment that provides services of distinct nature to the users. The development of low cost devices, with small size, light weight and a high level of autonomy, processing power and ability for interaction is required to obtain this environment. Attending to this last feature, many of these devices will include the capacity to acquire, process and transmit images, extracting, interpreting and modifying the visual information that could be of interest for a certain application. This PhD Thesis, focused on the development of dedicated hardware for the implementation of image and video processing algorithms used in embedded systems, attempts to response to this challenge. The work has a two-fold purpose: on one hand, the search of solutions that, for its performance and properties, could be integrated on systems with strict requirements of functionality, size, power consumption and speed of operation; on the other hand, the design of a set of blocks that, packaged and implemented as IP-modules, allow to alleviate the computational load of the processing units of the systems where they could be integrated. In this Thesis, specific solutions for the implementation of two kinds of usual operations in many computer vision systems are provided. These operations are background subtraction and connected component labelling. Different solutions are created as the result of applying a good performance/cost trade-off (approaching this last criteria in terms of area, speed and consumed power), able to cover a wide range of applications. Inference techniques based on Fuzzy Logic have been applied to some of the proposed solutions in order to improve the quality of the resulting systems. To obtain the mentioned solutions, a model based-design methodology has been applied. This fact has allowed us to carry out all the design flow from a single work environment. That environment combines CAD tools that facilitate the stages of code programming, circuit design, physical implementation and functional and temporal verification of the different algorithms, thus accelerating the overall processes and making it possible to explore the space of solutions. Moreover, aiming to demonstrate the functionality of this PhD Thesis’s contributions, some of the proposed solutions have been integrated on real video systems that employ common and standard buses. The devices selected to perform these demonstrators have been FPGA and SoPCs (manufactured by Xilinx) since, due to their excellent properties for prototyping and creating systems that combine software and hardware components, they are ideal to develop these applications

Arquiteturas de hardware para aceleração de algoritmos de reconstrução morfológica

Este trabalho apresenta um estudo da implementação de algoritmos para a reconstrução morfológica de imagens bio-medicas em FPGAs (Field Programmable Gate Arrays). As arquiteturas foram baseadas nos algoritmos Sequential Reconstruction (SR) e Fast Hybrid (FH) usando linguagem de descrição de hardware VHDL (Very High Description Language). A metodologia para avaliar a plataforma consistiu em verificar a arquitetura projetada no QuestaSim, fornecendo como dados de entrada as imagens a ser reconstruídas. Adicionalmente, a validação dos resultados da arquitetura foi feita usando linguagem C ou Matlab (usando a função imreconstruct). Além disso, um estudo consumo de recursos de hardware para diferentes tamanhos e conteúdos de imagens foram realizados com o intuito de verificar a aplicabilidade dos algoritmos em arquiteturas reconfiguráveis. Neste trabalho, para a aceleração do processo de reconstrução da imagem foi proposta uma arquitetura reconfigurável baseada no algoritmo FH junto com um algoritmo de aprendizagem de máquina, especificamente uma máquina de vetores de suporte (SVM). Para o treinamento da SVM foi usada uma metodologia de verificação/validação obtendo aproximadamente 20.000 dados de treinamento. Finalmente, foi implementada uma arquitetura que particiona a imagem original em quatro unidades de processamento, processando cada unidade em paralelo. O sistema final implementado fornece um pixel processado por cada ciclo de relógio, depois de um tempo de latência, sendo aproximadamente 8 vezes mais rápida que sua versão não particionada. Adicionalmente, foram feitas comparações rodando os algoritmos de reconstrução morfológica em um processador ARM embarcado dentro do FPGA.Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES).This work presents a study of the implementation of algorithms for the morphological reconstruction of bio-medical images in FPGAs (Field Programmable Gate Arrays). The architectures were based on Sequential Reconstruction (SR) algorithms and Fast Hybrid (FH), using VHDL (Very High Description Language). The methodology for the evaluation of the platform consisted of verifying the architecture designed in QuestaSim, providing the images to be reconstructed as input data. Additionally, the validation of the results of the architecture was made using C or Matlab languages (using the imreconstruct function). Additionally, a study of hardware resource consumption for different sizes and content of images was conducted, in order to verify the applicability of the algorithms in reconfigurable architectures. In this work, in order to accelerate the image reconstruction process, a reconfigurable architecture based on the FH algorithm is proposed together with machine learning, specifically a support vector machine (SVM). For the SVM training a verification/validation methodology was used, obtaining approximately 20,000 training data. Finally, an architecture was implemented that partitions the original image in four processing units, processing each unit in parallel. The final system implemented provides one pixel processed for each clock cycle, after a latency time, being approximately 8 times faster than its unpartitioned version. Lastly, comparisons were made by running the morphological reconstruction algorithms in an ARM processor embedded within the FPGA

Malay articulation system for early screening diagnostic using hidden markov model and genetic algorithm

Speech recognition is an important technology and can be used as a great aid for individuals with sight or hearing disabilities today. There are extensive research interest and development in this area for over the past decades. However, the prospect in Malaysia regarding the usage and exposure is still immature even though there is demand from the medical and healthcare sector. The aim of this research is to assess the quality and the impact of using computerized method for early screening of speech articulation disorder among Malaysian such as the omission, substitution, addition and distortion in their speech. In this study, the statistical probabilistic approach using Hidden Markov Model (HMM) has been adopted with newly designed Malay corpus for articulation disorder case following the SAMPA and IPA guidelines. Improvement is made at the front-end processing for feature vector selection by applying the silence region calibration algorithm for start and end point detection. The classifier had also been modified significantly by incorporating Viterbi search with Genetic Algorithm (GA) to obtain high accuracy in recognition result and for lexical unit classification. The results were evaluated by following National Institute of Standards and Technology (NIST) benchmarking. Based on the test, it shows that the recognition accuracy has been improved by 30% to 40% using Genetic Algorithm technique compared with conventional technique. A new corpus had been built with verification and justification from the medical expert in this study. In conclusion, computerized method for early screening can ease human effort in tackling speech disorders and the proposed Genetic Algorithm technique has been proven to improve the recognition performance in terms of search and classification task

A graph-based approach for the retrieval of multi-modality medical images

Medical imaging has revolutionised modern medicine and is now an integral aspect of diagnosis and patient monitoring. The development of new imaging devices for a wide variety of clinical cases has spurred an increase in the data volume acquired in hospitals. These large data collections offer opportunities for search-based applications in evidence-based diagnosis, education, and biomedical research. However, conventional search methods that operate upon manual annotations are not feasible for this data volume. Content-based image retrieval (CBIR) is an image search technique that uses automatically derived visual features as search criteria and has demonstrable clinical benefits. However, very few studies have investigated the CBIR of multi-modality medical images, which are making a monumental impact in healthcare, e.g., combined positron emission tomography and computed tomography (PET-CT) for cancer diagnosis. In this thesis, we propose a new graph-based method for the CBIR of multi-modality medical images. We derive a graph representation that emphasises the spatial relationships between modalities by structurally constraining the graph based on image features, e.g., spatial proximity of tumours and organs. We also introduce a graph similarity calculation algorithm that prioritises the relationships between tumours and related organs. To enable effective human interpretation of retrieved multi-modality images, we also present a user interface that displays graph abstractions alongside complex multi-modality images. Our results demonstrated that our method achieved a high precision when retrieving images on the basis of tumour location within organs. The evaluation of our proposed UI design by user surveys revealed that it improved the ability of users to interpret and understand the similarity between retrieved PET-CT images. The work in this thesis advances the state-of-the-art by enabling a novel approach for the retrieval of multi-modality medical images

Tracking the Temporal-Evolution of Supernova Bubbles in Numerical Simulations

The study of low-dimensional, noisy manifolds embedded in a higher dimensional space has been extremely useful in many applications, from the chemical analysis of multi-phase flows to simulations of galactic mergers. Building a probabilistic model of the manifolds has helped in describing their essential properties and how they vary in space. However, when the manifold is evolving through time, a joint spatio-temporal modelling is needed, in order to fully comprehend its nature. We propose a first-order Markovian process that propagates the spatial probabilistic model of a manifold at fixed time, to its adjacent temporal stages. The proposed methodology is demonstrated using a particle simulation of an interacting dwarf galaxy to describe the evolution of a cavity generated by a Supernov

Towards Refinement for Measuring Subcutaneously Transplanted Tumour Models in Mice

Evaluation using mouse subcutaneous tumour models is a key process in cancer drug development. Tumour material is implanted subcutaneously and tumour growth measured using callipers. However this methodology can have poor reproducibility and accuracy due to observer variation. Furthermore the physical pressure of using callipers can distress the mouse and lead to tumour damage. Non-invasive digital tumour imaging would reduce handling stresses and allow volume determination without physical contact. This thesis focusses on capturing 2D digital images of subcutaneous tumours, then using image processing and machine learning methods to determine 3D volume. The biggest challenge faced was lack of differentiation between tumour and surrounding skin, rendering tumour boundary identification difficult. Whilst image processing methods such as colour segmentation and edge detection were unsuccessful, machine learning proved more successful. Three convolutional neural networks, VGG-Face, VGG-19 and VGG-16 models were evaluated, with VGG-Face producing the best results. Using the layer FC7 before RELU activation for extraction in the VGC-Face model, a tumour recognition rate of 98.86% was achieved. This was increased to 100% through a semi-automatic step with detection repeated on cropped versions of negatively classified images. Finally, volume was determined through extracting image features using the VGG-Face model and conducting partial least squares regression (error of 0.1). This work has successfully demonstrated that with computational methods the volume of subcutaneous tumours can be evaluated through non-invasive digital imaging without need to have contact with the tumour itself, thus offering refinement benefits to the mice as well as eliminating observer bias

Proceedings of the 2nd European conference on disability, virtual reality and associated technologies (ECDVRAT 1998)

The proceedings of the conferenc