Flexible Hardware Architectures for Retinal Image Analysis

RÉSUMÉ Des millions de personnes autour du monde sont touchées par le diabète. Plusieurs complications oculaires telle que la rétinopathie diabétique sont causées par le diabète, ce qui peut conduire à une perte de vision irréversible ou même la cécité si elles ne sont pas traitées. Des examens oculaires complets et réguliers par les ophtalmologues sont nécessaires pour une détection précoce des maladies et pour permettre leur traitement. Comme solution préventive, un protocole de dépistage impliquant l'utilisation d'images numériques du fond de l'œil a été adopté. Cela permet aux ophtalmologistes de surveiller les changements sur la rétine pour détecter toute présence d'une maladie oculaire. Cette solution a permis d'obtenir des examens réguliers, même pour les populations des régions éloignées et défavorisées. Avec la grande quantité d'images rétiniennes obtenues, des techniques automatisées pour les traiter sont devenues indispensables. Les techniques automatisées de détection des maladies des yeux ont été largement abordées par la communauté scientifique. Les techniques développées ont atteint un haut niveau de maturité, ce qui a permis entre autre le déploiement de solutions en télémédecine. Dans cette thèse, nous abordons le problème du traitement de volumes élevés d'images rétiniennes dans un temps raisonnable dans un contexte de dépistage en télémédecine. Ceci est requis pour permettre l'utilisation pratique des techniques développées dans le contexte clinique. Dans cette thèse, nous nous concentrons sur deux étapes du pipeline de traitement des images rétiniennes. La première étape est l'évaluation de la qualité de l'image rétinienne. La deuxième étape est la segmentation des vaisseaux sanguins rétiniens. L’évaluation de la qualité des images rétinienne après acquisition est une tâche primordiale au bon fonctionnement de tout système de traitement automatique des images de la rétine. Le rôle de cette étape est de classifier les images acquises selon leurs qualités, et demander une nouvelle acquisition en cas d’image de mauvaise qualité. Plusieurs algorithmes pour évaluer la qualité des images rétiniennes ont été proposés dans la littérature. Cependant, même si l'accélération de cette tâche est requise en particulier pour permettre la création de systèmes mobiles de capture d'images rétiniennes, ce sujet n'a pas encore été abordé dans la littérature. Dans cette thèse, nous ciblons un algorithme qui calcule les caractéristiques des images pour permettre leur classification en mauvaise, moyenne ou bonne qualité. Nous avons identifié le calcul des caractéristiques de l'image comme une tâche répétitive qui nécessite une accélération. Nous nous sommes intéressés plus particulièrement à l’accélération de l’algorithme d’encodage à longueur de séquence (Run-Length Matrix – RLM). Nous avons proposé une première implémentation complètement logicielle mise en œuvre sous forme d’un système embarqué basé sur la technologie Zynq de Xilinx. Pour accélérer le calcul des caractéristiques, nous avons conçu un co-processeur capable de calculer les caractéristiques en parallèle implémenté sur la logique programmable du FPGA Zynq. Nous avons obtenu une accélération de 30,1 × pour la tâche de calcul des caractéristiques de l’algorithme RLM par rapport à son implémentation logicielle sur la plateforme Zynq. La segmentation des vaisseaux sanguins rétiniens est une tâche clé dans le pipeline du traitement des images de la rétine. Les vaisseaux sanguins et leurs caractéristiques sont de bons indicateurs de la santé de la rétine. En outre, leur segmentation peut également aider à segmenter les lésions rouges, indicatrices de la rétinopathie diabétique. Plusieurs techniques de segmentation des vaisseaux sanguins rétiniens ont été proposées dans la littérature. Des architectures matérielles ont également été proposées pour accélérer certaines de ces techniques. Les architectures existantes manquent de performances et de flexibilité de programmation, notamment pour les images de haute résolution. Dans cette thèse, nous nous sommes intéressés à deux techniques de segmentation du réseau vasculaire rétinien, la technique du filtrage adapté et la technique des opérateurs de ligne. La technique de filtrage adapté a été ciblée principalement en raison de sa popularité. Pour cette technique, nous avons proposé deux architectures différentes, une architecture matérielle personnalisée mise en œuvre sur FPGA et une architecture basée sur un ASIP. L'architecture matérielle personnalisée a été optimisée en termes de surface et de débit de traitement pour obtenir des performances supérieures par rapport aux implémentations existantes dans la littérature. Cette implémentation est plus efficace que toutes les implémentations existantes en termes de débit. Pour l'architecture basée sur un processeur à jeu d’instructions spécialisé (Application-Specific Instruction-set Processor – ASIP), nous avons identifié deux goulets d'étranglement liés à l'accès aux données et à la complexité des calculs de l'algorithme. Nous avons conçu des instructions spécifiques ajoutées au chemin de données du processeur. L'ASIP a été rendu 7.7 × plus rapide par rapport à son architecture de base. La deuxième technique pour la segmentation des vaisseaux sanguins est l'algorithme détecteur de ligne multi-échelle (Multi-Scale Ligne Detector – MSLD). L'algorithme MSLD est choisi en raison de ses performances et de son potentiel à détecter les petits vaisseaux sanguins. Cependant, l'algorithme fonctionne en multi-échelle, ce qui rend l’algorithme gourmand en mémoire. Pour résoudre ce problème et permettre l'accélération de son exécution, nous avons proposé un algorithme efficace en terme de mémoire, conçu et implémenté sur FPGA. L'architecture proposée a réduit de façon drastique les exigences de l’algorithme en terme de mémoire en réutilisant les calculs et la co-conception logicielle/matérielle. Les deux architectures matérielles proposées pour la segmentation du réseau vasculaire rétinien ont été rendues flexibles pour pouvoir traiter des images de basse et de haute résolution. Ceci a été réalisé par le développement d'un compilateur spécifique capable de générer une description HDL de bas niveau de l'algorithme à partir d'un ensemble de paramètres. Le compilateur nous a permis d’optimiser les performances et le temps de développement. Dans cette thèse, nous avons introduit deux architectures qui sont, au meilleur de nos connaissances, les seules capables de traiter des images à la fois de basse et de haute résolution.----------ABSTRACT Millions of people all around the world are affected by diabetes. Several ocular complications such as diabetic retinopathy are caused by diabetes, which can lead to irreversible vision loss or even blindness if not treated. Regular comprehensive eye exams by eye doctors are required to detect the diseases at earlier stages and permit their treatment. As a preventing solution, a screening protocol involving the use of digital fundus images was adopted. This allows eye doctors to monitor changes in the retina to detect any presence of eye disease. This solution made regular examinations widely available, even to populations in remote and underserved areas. With the resulting large amount of retinal images, automated techniques to process them are required. Automated eye detection techniques are largely addressed by the research community, and now they reached a high level of maturity, which allows the deployment of telemedicine solutions. In this thesis, we are addressing the problem of processing a high volume of retinal images in a reasonable time. This is mandatory to allow the practical use of the developed techniques in a clinical context. In this thesis, we focus on two steps of the retinal image pipeline. The first step is the retinal image quality assessment. The second step is the retinal blood vessel segmentation. The evaluation of the quality of the retinal images after acquisition is a primary task for the proper functioning of any automated retinal image processing system. The role of this step is to classify the acquired images according to their quality, which will allow an automated system to request a new acquisition in case of poor quality image. Several algorithms to evaluate the quality of retinal images were proposed in the literature. However, even if the acceleration of this task is required, especially to allow the creation of mobile systems for capturing retinal images, this task has not yet been addressed in the literature. In this thesis, we target an algorithm that computes image features to allow their classification to bad, medium or good quality. We identified the computation of image features as a repetitive task that necessitates acceleration. We were particularly interested in accelerating the Run-Length Matrix (RLM) algorithm. We proposed a first fully software implementation in the form of an embedded system based on Xilinx's Zynq technology. To accelerate the features computation, we designed a co-processor able to compute the features in parallel, implemented on the programmable logic of the Zynq FPGA. We achieved an acceleration of 30.1× over its software implementation for the features computation part of the RLM algorithm. Retinal blood vessel segmentation is a key task in the pipeline of retinal image processing. Blood vessels and their characteristics are good indicators of retina health. In addition, their segmentation can also help to segment the red lesions, indicators of diabetic retinopathy. Several techniques have been proposed in the literature to segment retinal blood vessels. Hardware architectures have also been proposed to accelerate blood vessel segmentation. The existing architectures lack in terms of performance and programming flexibility, especially for high resolution images. In this thesis, we targeted two techniques, matched filtering and line operators. The matched filtering technique was targeted mainly because of its popularity. For this technique, we proposed two different architectures, a custom hardware architecture implemented on FPGA, and an Application Specific Instruction-set Processor (ASIP) based architecture. The custom hardware architecture area and timing were optimized to achieve higher performances in comparison to existing implementations. Our custom hardware implementation outperforms all existing implementations in terms of throughput. For the ASIP based architecture, we identified two bottlenecks related to data access and computation intensity of the algorithm. We designed two specific instructions added to the processor datapath. The ASIP was made 7.7× more efficient in terms of execution time compared to its basic architecture. The second technique for blood vessel segmentation is the Multi-Scale Line Detector (MSLD) algorithm. The MSLD algorithm is selected because of its performance and its potential to detect small blood vessels. However, the algorithm works at multiple scales which makes it memory intensive. To solve this problem and allow the acceleration of its execution, we proposed a memory-efficient algorithm designed and implemented on FPGA. The proposed architecture reduces drastically the memory requirements of the algorithm by reusing the computations and SW/HW co-design. The two hardware architectures proposed for retinal blood vessel segmentation were made flexible to be able to process low and high resolution images. This was achieved by the development of a specific compiler able to generate low-level HDL descriptions of the algorithm from a set of the algorithm parameters. The compiler enabled us to optimize performance and development time. In this thesis, we introduce two novel architectures which are, to the best of our knowledge, the only ones able to process both low and high resolution images

A Feasibility Study to Develop an Integrated Diabetic Retinopathy Screening Programme in the Western Province of Sri Lanka

Background: Diabetic retinopathy (DR) is a common microvascular complication of diabetes mellitus which can lead to sight loss, if not detected and treated in time. Objectives: This study aimed to assess the feasibility of integrating DR screening (DRS) services into free public sector health care in Sri Lanka. The objectives were to identify barriers to access DRS, to determine the most appropriate DRS modality and to assess acceptability of a health educational intervention (HEI). Methods: The study was conducted using mixed methods. The barriers were assessed through systematic literature search and qualitative studies. A systematic literature review and meta-analysis was conducted to assess the diagnostic accuracy of DRS using digital retinal imaging. Based on the results of the formative stages, a local context specific DRS modality was defined and validated at a tertiary level medical clinic by trained physician graders. Finally, a HEI was adapted and acceptability was assessed using participatory approach. Results: The formative studies revealed that lack of knowledge and awareness on DR, lack of skilled human resources and DRS imaging infrastructure as the main barriers. In the meta-analysis, highest sensitivity was observed in mydriatic more than two field strategy (92%, 95% CI 90-94%). In the validation study, sensitivity of the defined referable DR was 88.7% for grader 1 and 92.5% for grader 2, using mydriatic imaging. The specificity was 94.9% for grader 1 and 96.4% for grader 2. The overall acceptability of the HEI material was satisfactory. Conclusions: Knowing the barriers to access DRS is a pre-requisite in development of a DRS program. Non-mydriatic 2-field strategy is a more pragmatic approach in implementing DRS programs in low income non-ophthalmic settings, with dilatation of pupils of those who have ungradable images. The process of adapting HEI was not simply translation into local language, instead a tailored approach for the local context

Gaze-Based Human-Robot Interaction by the Brunswick Model

We present a new paradigm for human-robot interaction based on social signal processing, and in particular on the Brunswick model. Originally, the Brunswick model copes with face-to-face dyadic interaction, assuming that the interactants are communicating through a continuous exchange of non verbal social signals, in addition to the spoken messages. Social signals have to be interpreted, thanks to a proper recognition phase that considers visual and audio information. The Brunswick model allows to quantitatively evaluate the quality of the interaction using statistical tools which measure how effective is the recognition phase. In this paper we cast this theory when one of the interactants is a robot; in this case, the recognition phase performed by the robot and the human have to be revised w.r.t. the original model. The model is applied to Berrick, a recent open-source low-cost robotic head platform, where the gazing is the social signal to be considered

Telemedicine

Telemedicine is a rapidly evolving field as new technologies are implemented for example for the development of wireless sensors, quality data transmission. Using the Internet applications such as counseling, clinical consultation support and home care monitoring and management are more and more realized, which improves access to high level medical care in underserved areas. The 23 chapters of this book present manifold examples of telemedicine treating both theoretical and practical foundations and application scenarios

Aspectos do rastreamento do glaucoma auxiliados por técnicas automatizadas em imagens com menor qualidade do disco óptico

O glaucoma é uma neuropatia óptica cuja progressão pode levar a cegueira. Representa a principal causa de perda visual de caráter irreversível em todo o mundo para homens e mulheres. A detecção precoce através de programas de rastreamento feita por especialistas é baseada nas características do nervo óptico, em biomarcadores oftalmológicos (destacando-se a pressão ocular) e exames subsidiários, com destaque ao campo visual e OCT. Após o reconhecimento dos casos é feito o tratamento com finalidade de estacionar a progressão da doença e melhorar a qualidade de vida dos pacientes. Contudo, estes programas têm limitações, principalmente em locais mais distantes dos grandes centros de tratamento especializado, insuficiência de equipamentos básicos e pessoal especializado para oferecer o rastreamento a toda a população, faltam meios para locomoção a estes centros, desinformação e desconhecimento da doença, além de características de progressão assintomática da doença. Esta tese aborda soluções inovadoras que podem contribuir para a automação do rastreamento do glaucoma utilizando aparelhos portáteis e mais baratos, considerando as necessidades reais dos clínicos durante o rastreamento. Para isso foram realizadas revisões sistemáticas sobre os métodos e equipamentos para apoio à triagem automática do glaucoma e os métodos de aprendizado profundo para a segmentação e classificação aplicáveis. Também foi feito um levantamento de questões médicas relativas à triagem do glaucoma e associá-las ao campo da inteligência artificial, para dar mais sentido as metodologias automatizadas. Além disso, foi criado um banco de dados privado, com vídeos e imagens de retina adquiridos por um smartphone acoplado a lente de baixo custo para o rastreamento do glaucoma e avaliado com métodos do estado da arte. Foram avaliados e analisados métodos de detecção automática de glaucoma utilizando métodos de aprendizado profundo de segmentação do disco e do copo óptico em banco de dados públicos de imagens de retina. Finalmente, foram avaliadas técnicas de mosaico e de detecção da cabeça do nervo óptico em imagens de baixa qualidade obtidas para pré-processamento de imagens adquiridas por smartphones acoplados a lente de baixo custo.Glaucoma is an optic neuropathy whose progression can lead to blindness. It represents the leading cause of irreversible visual loss worldwide for men and women. Early detection through screening programs carried out by specialists is based on the characteristics of the optic papilla, ophthalmic biomarkers (especially eye pressure), and subsidiary exams, emphasizing the visual field and optical coherence tomography (OCT). After recognizing the cases, the treatment is carried out to stop the progression of the disease and improve the quality of patients’ life. However, these screening programs have limitations, particularly in places further away from the sizeable, specialized treatment centers, due to the lack of essential equipment and technical personnel to offer screening to the entire population, due to the lack of means of transport to these centers, due to lack of information and lack of knowledge about the disease, considering the characteristics of asymptomatic progression of the disease. This thesis aims to develop innovative approaches to contribute to the automation of glaucoma screening using portable and cheaper devices, considering the real needs of clinicians during screening. For this, systematic reviews were carried out on the methods and equipment to support automatic glaucoma screening, and the applicable deep learning methods for segmentation and classification. A survey of medical issues related to glaucoma screening was carried out and associated with the field of artificial intelligence to make automated methodologies more effective. In addition, a private dataset was created, with videos and retina images acquired using a low-cost lens-coupled cell phone, for glaucoma screening and evaluated with state-of-the-art methods. Methods of automatic detection of glaucoma using deep learning methods of segmentation of the disc and optic cup were evaluated and analyzed in a public database of retinal images. In the case of deep learning classification methods, these were evaluated in public databases of retina images and in a private database with low-cost images. Finally, mosaic and object detection techniques were evaluated in low-quality images obtained for pre-processing images acquired by cell phones coupled with low-cost lenses

Psychophysiological indices of recognition memory

It has recently been found that during recognition memory tests participants’ pupils dilate more when they view old items compared to novel items. This thesis sought to replicate this novel ‘‘Pupil Old/New Effect’’ (PONE) and to determine its relationship to implicit and explicit mnemonic processes, the veracity of participants’ responses, and the analogous Event-Related Potential (ERP) old/new effect. Across 9 experiments, pupil-size was measured with a video-based eye-tracker during a variety of recognition tasks, and, in the case of Experiment 8, with concurrent Electroencephalography (EEG). The main findings of this thesis are that: - the PONE occurs in a standard explicit test of recognition memory but not in “implicit” tests of either perceptual fluency or artificial grammar learning; - the PONE is present even when participants are asked to give false behavioural answers in a malingering task, or are asked not to respond at all; - the PONE is present when attention is divided both at learning and during recognition; - the PONE is accompanied by a posterior ERP old/new effect; - the PONE does not occur when participants are asked to read previously encountered words without making a recognition decision; - the PONE does not occur if participants preload an “old/new” response; - the PONE is not enhanced by repetition during learning. These findings are discussed in the context of current models of recognition memory and other psychophysiological indices of mnemonic processes. It is argued that together these findings suggest that the increase in pupil-size which occurs when participants encounter previously studied items is not under conscious control and may reflect primarily recollective processes associated with recognition memory