Split and Shift Methodology: Overcoming Hardware Limitations on Cellular Processor Arrays for Image Processing

Na era multimedia, o procesado de imaxe converteuse nun elemento de singular importancia nos dispositivos electrónicos. Dende as comunicacións (p.e. telemedicina), a seguranza (p.e. recoñecemento retiniano) ou control de calidade e de procesos industriais (p.e. orientación de brazos articulados, detección de defectos do produto), pasando pola investigación (p.e. seguimento de partículas elementais) e diagnose médica (p.e. detección de células estrañas, identificaciónn de veas retinianas), hai un sinfín de aplicacións onde o tratamento e interpretación automáticas de imaxe e fundamental. O obxectivo último será o deseño de sistemas de visión con capacidade de decisión. As tendencias actuais requiren, ademais, a combinación destas capacidades en dispositivos pequenos e portátiles con resposta en tempo real. Isto propón novos desafíos tanto no deseño hardware como software para o procesado de imaxe, buscando novas estruturas ou arquitecturas coa menor area e consumo de enerxía posibles sen comprometer a funcionalidade e o rendemento

Ultra Low Energy Analog Image Processing Using Spin Neurons

In this work we present an ultra low energy, 'on-sensor' image processing architecture, based on cellular array of spin based neurons. The 'neuron' constitutes of a lateral spin valve (LSV) with multiple input magnets, connected to an output magnet, using metal channels. The low resistance, magneto-metallic neurons operate at a small terminal voltage of ~20mV, while performing analog computation upon photo sensor inputs. The static current-flow across the device terminals is limited to small periods, corresponding to magnet switching time, and, is determined by a low duty-cycle system-clock. Thus, the energy-cost of analog-mode processing, inevitable in most image sensing applications, is reduced and made comparable to that of dynamic and leakage power consumption in peripheral CMOS units. Performance of the proposed architecture for some common image sensing and processing applications like, feature extraction, halftone compression and digitization, have been obtained through physics based device simulation framework, coupled with SPICE. Results indicate that the proposed design scheme can achieve more than two orders of magnitude reduction in computation energy, as compared to the state of art CMOS designs, that are based on conventional mixed-signal image acquisition and processing schemes. To the best of authors' knowledge, this is the first work where application of nano magnets (in LSV's) in analog signal processing has been proposed

Automatic Pixel-Parallel Extraction of the Retinal Vascular Tree Algorithm Design, On-Chip Implementation an Applications

[Resumen] La tesis doctoral propone un nuevo algoritmo para la extracción del árbol arterio-venoso en imágenes digitales de retina usando sistemas pixel paralelo que le confiere un procesamiento a alta velocidad, Inicialmente el problema de la extracción del árbol arterio venoso se estudió desde el punto de vista del procesamiento de imágenes utilizando técnicas pixel paralelo, concretamente bajo el paradigma de las Cellular Neural Networks. Este algoritmo utiliza una técnica de contornos activos, los Pixel level snakes (PLS) que permiten aprovechar las ventajas de los contornos activos, como es su capacidad de funcionamiento con contornos borrosos así como su robustez ante el ruido, y al mismo tiempo todo ello procesándose a una alta velocidad de computación. Esta técnica permite también su proyección en un dispositivo hardware específico. La primera versión del algoritmo fue diseñada basándose en el paradigma CNN. Los resultados obtenidos eran buenos bajo el punto de vista del procesado de imagen. Sin embargo, la complejidad de algunas de las operaciones propuestas en esta versión eran de una alta complejidad para ser implementados en los chips pixel paralelos actuales con capacidades SIMD (Single Instruction Multiple Data). Esta versión ha sido redefinida para ser implementada en un chip SIMD. Esta última versión ha sido analizada desde un punto de vista del ajuste de los resultados y desde el punto de vista de la velocidad de ejecución. Para el primer análisis se ha hecho uso de una base de datos pública, concretamente la DRIVE (Digital Retinal Image for Vessel Extraction). Para el análisis de los tiempos de ejecución, se implementó el algoritmo en un chip específico, el SCAMP-3 vision system. El análisis de ambos aspectos ha permitido observar, que el ajuste obtenido sobre los resultados es alto, aunque existen algoritmos con un ajuste mejor, y el tiempo de ejecución es realmente rápido y no existe ningún algoritmo en la bibliografía que mejore el tiempo obtenido con la implementación propuesta en esta tesis. Asimismo se ha realizado un estudio de la mejora que se podría obtener utilizando una técnica de solapamiento, puesto que debido a la alta resolución de las imágenes utilizadas, estas se han tenido que dividir en subventanas para su procesamiento. Este análisis ha demostrado que la mejora obtenida es mínima en comparación con el notable incremento del tiempo de ejecución, siendo descartada su utilización. Una vez demostrado el funcionamiento del algoritmo se ha procedido a su inclusión en aplicaciones prácticas que se encontraban ya funcionando utilizando algoritmos clásicos para la extracción del árbol arterio venoso. Las aplicaciones corresponden a dos ámbitos diferentes con necesidades propias, el ámbito médico y la autenticación de personas. Para la autenticación de personas se observó que el funcionamiento es igual que usando las versiones clásicas, manteniendo un 100% de efectividad en la identificación de personas. En el caso de la aplicación médica, se incluyó dentro de un sistema de estimación del índice arterio-venoso, mostrando un funcionamiento con valores similares

Dynamically reconfigurable architecture for embedded computer vision systems

The objective of this research work is to design, develop and implement a new architecture which integrates on the same chip all the processing levels of a complete Computer Vision system, so that the execution is efficient without compromising the power consumption while keeping a reduced cost. For this purpose, an analysis and classification of different mathematical operations and algorithms commonly used in Computer Vision are carried out, as well as a in-depth review of the image processing capabilities of current-generation hardware devices. This permits to determine the requirements and the key aspects for an efficient architecture. A representative set of algorithms is employed as benchmark to evaluate the proposed architecture, which is implemented on an FPGA-based system-on-chip. Finally, the prototype is compared to other related approaches in order to determine its advantages and weaknesses

Microarray image processing: A novel neural network framework

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Due to the vast success of bioengineering techniques, a series of large-scale analysis tools has been developed to discover the functional organization of cells. Among them, cDNA microarray has emerged as a powerful technology that enables biologists to cDNA microarray technology has enabled biologists to study thousands of genes simultaneously within an entire organism, and thus obtain a better understanding of the gene interaction and regulation mechanisms involved. Although microarray technology has been developed so as to offer high tolerances, there exists high signal irregularity through the surface of the microarray image. The imperfection in the microarray image generation process causes noises of many types, which contaminate the resulting image. These errors and noises will propagate down through, and can significantly affect, all subsequent processing and analysis. Therefore, to realize the potential of such technology it is crucial to obtain high quality image data that would indeed reflect the underlying biology in the samples. One of the key steps in extracting information from a microarray image is segmentation: identifying which pixels within an image represent which gene. This area of spotted microarray image analysis has received relatively little attention relative to the advances in proceeding analysis stages. But, the lack of advanced image analysis, including the segmentation, results in sub-optimal data being used in all downstream analysis methods. Although there is recently much research on microarray image analysis with many methods have been proposed, some methods produce better results than others. In general, the most effective approaches require considerable run time (processing) power to process an entire image. Furthermore, there has been little progress on developing sufficiently fast yet efficient and effective algorithms the segmentation of the microarray image by using a highly sophisticated framework such as Cellular Neural Networks (CNNs). It is, therefore, the aim of this thesis to investigate and develop novel methods processing microarray images. The goal is to produce results that outperform the currently available approaches in terms of PSNR, k-means and ICC measurements.Aleppo University, Syri

Local Binary Patterns in Focal-Plane Processing. Analysis and Applications

Feature extraction is the part of pattern recognition, where the sensor data is transformed into a more suitable form for the machine to interpret. The purpose of this step is also to reduce the amount of information passed to the next stages of the system, and to preserve the essential information in the view of discriminating the data into different classes. For instance, in the case of image analysis the actual image intensities are vulnerable to various environmental effects, such as lighting changes and the feature extraction can be used as means for detecting features, which are invariant to certain types of illumination changes. Finally, classification tries to make decisions based on the previously transformed data. The main focus of this thesis is on developing new methods for the embedded feature extraction based on local non-parametric image descriptors. Also, feature analysis is carried out for the selected image features. Low-level Local Binary Pattern (LBP) based features are in a main role in the analysis. In the embedded domain, the pattern recognition system must usually meet strict performance constraints, such as high speed, compact size and low power consumption. The characteristics of the final system can be seen as a trade-off between these metrics, which is largely affected by the decisions made during the implementation phase. The implementation alternatives of the LBP based feature extraction are explored in the embedded domain in the context of focal-plane vision processors. In particular, the thesis demonstrates the LBP extraction with MIPA4k massively parallel focal-plane processor IC. Also higher level processing is incorporated to this framework, by means of a framework for implementing a single chip face recognition system. Furthermore, a new method for determining optical flow based on LBPs, designed in particular to the embedded domain is presented. Inspired by some of the principles observed through the feature analysis of the Local Binary Patterns, an extension to the well known non-parametric rank transform is proposed, and its performance is evaluated in face recognition experiments with a standard dataset. Finally, an a priori model where the LBPs are seen as combinations of n-tuples is also presentedSiirretty Doriast

Cellular Nonlinear Networks: optimized implementation on FPGA and applications to robotics

L'objectiu principal d'aquesta tesi consisteix a estudiar la factibilitat d'implementar un sensor càmera CNN amb plena funcionalitat basat en FPGA de baix cost adequat per a aplicacions en robots mòbils. L'estudi dels fonaments de les xarxes cel•lulars no lineals (CNNs) i la seva aplicació eficaç en matrius de portes programables (FPGAs) s'ha complementat, d'una banda amb el paral•lelisme que s'estableix entre arquitectura multi-nucli de les CNNs i els eixams de robots mòbils, i per l'altre banda amb la correlació dinàmica de CNNs i arquitectures memristive. A més, els memristors es consideren els substituts dels futurs dispositius de memòria flash per la seva capacitat d'integració d'alta densitat i el seu consum d'energia prop de zero. En el nostre cas, hem estat interessats en el desenvolupament d’FPGAs que han deixat de ser simples dispositius per a la creació ràpida de prototips ASIC per esdevenir complets dispositius reconfigurables amb integració de la memòria i els elements de processament general. En particular, s'han explorat com les arquitectures implementades CNN en FPGAs poden ser optimitzades en termes d’àrea ocupada en el dispositiu i el seu consum de potència. El nostre objectiu final ens ah portat a implementar de manera eficient una CNN-UM amb complet funcionament a un baix cost i baix consum sobre una FPGA amb tecnología flash. Per tant, futurs estudis sobre l’arquitectura eficient de la CNN sobre la FPGA i la interconnexió amb els robots comercials disponibles és un dels objectius d'aquesta tesi que se seguiran en les línies de futur exposades en aquest treball.El objetivo principal de esta tesis consiste en estudiar la factibilidad de implementar un sensor cámara CNN con plena funcionalidad basado en FPGA de bajo coste adecuado para aplicaciones en robots móviles. El estudio de los fundamentos de las redes celulares no lineales (CNNs) y su aplicación eficaz en matrices de puertas programables (FPGAs) se ha complementado, por un lado con el paralelismo que se establece entre arquitectura multi -núcleo de las CNNs y los enjambres de robots móviles, y por el otro lado con la correlación dinámica de CNNs y arquitecturas memristive. Además, los memristors se consideran los sustitutos de los futuros dispositivos de memoria flash por su capacidad de integración de alta densidad y su consumo de energía cerca de cero. En nuestro caso, hemos estado interesados en el desarrollo de FPGAs que han dejado de ser simples dispositivos para la creación rápida de prototipos ASIC para convertirse en completos dispositivos reconfigurables con integración de la memoria y los elementos de procesamiento general. En particular, se han explorado como las arquitecturas implementadas CNN en FPGAs pueden ser optimizadas en términos de área ocupada en el dispositivo y su consumo de potencia. Nuestro objetivo final nos ah llevado a implementar de manera eficiente una CNN-UM con completo funcionamiento a un bajo coste y bajo consumo sobre una FPGA con tecnología flash. Por lo tanto, futuros estudios sobre la arquitectura eficiente de la CNN sobre la FPGA y la interconexión con los robots comerciales disponibles es uno de los objetivos de esta tesis que se seguirán en las líneas de futuro expuestas en este trabajo.The main goal of this thesis consists in studying the feasibility to implement a full-functionality CNN camera sensor based on low-cost FPGA device suitable for mobile robotic applications. The study of Cellular Nonlinear Networks (CNNs) fundamentals and its efficient implementation on Field Programmable Gate Arrays (FPGAs) has been complemented, on one side with the parallelism established between multi-core CNN architecture and swarm of mobile robots, and on the other side with the dynamics correlation of CNNs and memristive architectures. Furthermore, memristors are considered the future substitutes of flash memory devices because of its capability of high density integration and its close to zero power consumption. In our case, we have been interested in the development of FPGAs that have ceased to be simple devices for ASIC fast prototyping to become complete reconfigurable devices embedding memory and processing elements. In particular, we have explored how the CNN architectures implemented on FPGAs can be optimized in terms of area occupied on the device or power consumption. Our final accomplishment has been implementing efficiently a fully functional reconfigurable CNN-UM on a low-cost low-power FPGA based on flash technology. Therefore, further studies on an efficient CNN architecture on FPGA and interfacing it with commercially-available robots is one of the objectives of this thesis that will be followed in the future directions exposed in this work

Design and Real-World Application of Novel Machine Learning Techniques for Improving Face Recognition Algorithms

Recent progress in machine learning has made possible the development of real-world face recognition applications that can match face images as good as or better than humans. However, several challenges remain unsolved. In this PhD thesis, some of these challenges are studied and novel machine learning techniques to improve the performance of real-world face recognition applications are proposed. Current face recognition algorithms based on deep learning techniques are able to achieve outstanding accuracy when dealing with face images taken in unconstrained environments. However, training these algorithms is often costly due to the very large datasets and the high computational resources needed. On the other hand, traditional methods for face recognition are better suited when these requirements cannot be satisfied. This PhD thesis presents new techniques for both traditional and deep learning methods. In particular, a novel traditional face recognition method that combines texture and shape features together with subspace representation techniques is first presented. The proposed method is lightweight and can be trained quickly with small datasets. This method is used for matching face images scanned from identity documents against face images stored in the biometric chip of such documents. Next, two new techniques to increase the performance of face recognition methods based on convolutional neural networks are presented. Specifically, a novel training strategy that increases face recognition accuracy when dealing with face images presenting occlusions, and a new loss function that improves the performance of the triplet loss function are proposed. Finally, the problem of collecting large face datasets is considered, and a novel method based on generative adversarial networks to synthesize both face images of existing subjects in a dataset and face images of new subjects is proposed. The accuracy of existing face recognition algorithms can be increased by training with datasets augmented with the synthetic face images generated by the proposed method. In addition to the main contributions, this thesis provides a comprehensive literature review of face recognition methods and their evolution over the years. A significant amount of the work presented in this PhD thesis is the outcome of a 3-year-long research project partially funded by Innovate UK as part of a Knowledge Transfer Partnership between University of Hertfordshire and IDscan Biometrics Ltd (partnership number: 009547)