24 research outputs found

    Comprehensive retinal image analysis: image processing and feature extraction techniques oriented to the clinical task

    Get PDF
    Medical digital imaging has become a key element of modern health care procedures. It provides a visual documentation, a permanent record for the patients, and most importantly the ability to extract information about many diseases. Ophthalmology is a field that is heavily dependent on the analysis of digital images because they can aid in establishing an early diagnosis even before the first symptoms appear. This dissertation contributes to the digital analysis of such images and the problems that arise along the imaging pipeline, a field that is commonly referred to as retinal image analysis. We have dealt with and proposed solutions to problems that arise in retinal image acquisition and longitudinal monitoring of retinal disease evolution. Specifically, non-uniform illumination, poor image quality, automated focusing, and multichannel analysis. However, there are many unavoidable situations in which images of poor quality, like blurred retinal images because of aberrations in the eye, are acquired. To address this problem we have proposed two approaches for blind deconvolution of blurred retinal images. In the first approach, we consider the blur to be space-invariant and later in the second approach we extend the work and propose a more general space-variant scheme. For the development of the algorithms we have built preprocessing solutions that have enabled the extraction of retinal features of medical relevancy, like the segmentation of the optic disc and the detection and visualization of longitudinal structural changes in the retina. Encouraging experimental results carried out on real retinal images coming from the clinical setting demonstrate the applicability of our proposed solutions

    Effects of EEG-neurofeedback training on brain functional connectivity

    Get PDF
    Tese de Mestrado Integrado, Engenharia Biomédica e Biofísica, 2022, Universidade de Lisboa, Faculdade de CiênciasO neurofeedback (NF) consiste em medir a atividade cerebral, usando técnicas como a eletroencefalografia (EEG) ou a imagem por ressonância magnética funcional (fMRI), e apresentar ao participante, em tempo real, uma representação de um padrão de atividade de interesse, enquanto lhe é pedido para manipular essa mesma representação através da autorregulação da atividade cerebral (Sitaram et al., 2017). As bases neurofisiológicas desta técnica ainda não são conhecidas na sua totalidade, apesar de vários estudos terem demonstrado que o treino através de NF tende a reorganizar as redes cerebrais. Posto isto, existem poucos estudos que tentam comparar a influência da utilização de diferentes modalidades sensoriais de apresentação do “feedback” nos resultados do treino por NF em EEG, e os poucos estudos existentes não investigam possíveis efeitos nas métricas de conectividade funcional do cérebro. Neste projeto, pretendemos avaliar o efeito da utilização de diferentes modalidades de “feedback” no treino de NF através EEG (EEG-NF) para o incremento da amplitude relativa da banda alfa superior no canal Cz, e investigar se existe um efeito significativo nos padrões de conectividade funcional do cérebro. Para esse fim, será efetuada a análise de dados previamente recolhidos em 20 participantes saudáveis que realizaram quatro sessões de treino por EEG-NF, que visava incrementar a densidade espectral na banda alfa superior, e que utilizaram diferentes modalidades de feedback (visual, realidade virtual (VR), e auditiva). Os dados de EEG foram pré-processados, com remoção de artefactos através de análise de componentes independentes. Adicionalmente, duas técnicas de re-referenciação do sinal EEG foram utilizadas para comparação posterior, sendo estas a re-referenciação para a média de todos os canais EEG, e a re-referenciação através da aplicação de um Laplaciano de Superfície com parâmetro de rigidez de valores 4 e 3, respetivamente. A avaliação dos resultados foi efetuada a diversos níveis, com a análise: i) das variações intra-sessão da amplitude relativa da banda alfa superior no canal Cz, ii) da distribuição topológica da banda alfa superior no decorrer do treino, iii) das variações intrasessão dos padrões de conectividade funcional da banda alfa superior, utilizando a parte imaginária da coerência como métrica de conectividade, e iv) por fim, em termos de uma análise de redes, que visava avaliar a importância de nodos de rede, verificada através das métricas como betweeness centrality e força, da atividade segregada, verificada através da métrica de transitividade, e da atividade integrada, verificada através de métricas como caminho característico e eficiência global da rede cerebral. Relativamente aos resultados para a análise espectral e topológica, encontram-se correlações estatisticamente significativas entre o valor da amplitude relativa da banda alfa superior e o número de set, em todos os grupos, principalmente nas duas primeiras sessões, sendo cada set composto por 6 trials com duração de 30 segundos Posto isto, não são registadas diferenças estatisticamente significativas intra-sessão, isto é, do set 1 para o set 5 de cada sessão. Para a análise topológica, não se realizaram testes de significância, mas é possível visualizar uma acentuação da amplitude relativa da banda alfa superior em zonas parietais/occipitais, e é também possível verificar que o treino realizado, não afetou somente a banda de interesse mas também a banda theta, cuja atividade não focal diminui, a banda alfa inferior, cuja amplitude relativa parece incrementar. Relativamente aos resultados da análise de conetividade, os mesmos sugerem que o treino de EEG-NF para o incremento da banda alfa superior resulta num incremento mais pronunciado nas fases iniciais do treino, isto é, nas duas primeiras sessões de treino. Este incremento é representado pelo do número de canais que apresentam conectividade funcional com a zona parietal central, com canais como o Pz, e com a zona parietal direita, CP6, P4, entre outros, independentemente da modalidade de feedback, ou seja, para a generalidade dos "Learners”. De facto, os próprios canais parietais direitos, P4, P8, CP6, TP10 aumentam de forma estatisticamente significativa a conectividade entre eles. Isto parece indiciar a criação de um complexo focado na zona parietal direita. Em todas as modalidades, à exceção da VR, verifica-se ainda um aumento significativo intra-sessão da transitividade e eficiência global enquanto uma diminuição estatisticamente significativa intra-sessão é observada para a métrica caminho característico. Posto isto, a metodologia de neurofeedback no contexto experimental que foi implementado, parece promover a atividade cerebral segregada, isto é, a atividade que resulta de uma atividade cerebral mais localizada, e também integrada, isto é, que resulta da integração da atividade de áreas cerebrais dispersas. A não existência de variações significativas na modalidade VR não parece estar relacionada com a modalidade em si, mas sim devido a uma menor amostra do respetivo grupo. Assim, futuramente será necessário aumentar a amostra, pelo menos para este grupo, por forma a poderem ser extraídos resultados significativos da análise do mesmo. Interessantemente, e independentemente do método de rereferenciação utilizado, enquanto para o grupo do treino NF para a modalidade visual se observa a partir da terceira sessão de treino a estabilização do número de conexões funcionais entre os diferentes elétrodos, ou seja deixa de haver um crescimento acentuado da transitividade e da eficiência global com diminuição simultânea do caminho característico, para o grupo do treino NF com a modalidade auditiva a generalidade dos incrementos verificados, estão presentes em todas as sessões, incluindo a última sessão. No referente ao estudo sobre o método de re-referenciação dos dados EEG, com interesse específico na utilização de um Laplaciano de superfície comparativamente à simples utilização da média dos sinais EEG, a análise topológica das diferentes bandas cerebrais confirma que a utilização do Laplaciano de superfície contribuiu para aumento da resolução espacial dos dados de EEG, uma vez que atenuou para as diferentes bandas a amplitude relativa da atividade periférica, ou seja não focal, que estará relacionada com frequências espaciais mais baixas. Relativamente à análise da conectividade funcional intra-sessão, verifica-se que a aplicação do Laplaciano se reflete na mudança das configurações de variações de conexões funcionais no cérebro, nomeadamente eliminando determinados aumentos estatisticamente significativos, por exemplo para a sessão 1 dos “Learners”, após a aplicação do Laplaciano de superfície, o incremento da conectividade funcional entre Pz e O2 deixa de ser estatisticamente significativo. Possivelmente, isto poderá estar relacionado com uma eliminação de conexões espúrias. Também na análise de redes, a aplicação do Laplaciano afeta a configuração dos dados e outputs embora não se consiga precisar uma relação causa efeito. Posto isto, a variação da própria configuração do Laplaciano, no que se refere à rigidez do mesmo, de parâmetro m=4 para m=3, não se traduz em resultados tão diferentes, pese embora algumas alterações notadas na análise de redes. De facto, para análise de conectividade funcional, os heatmaps resultantes da aplicação de Laplaciano de superfície com m=4, são exatamente iguais aos heatmaps resultantes da aplicação de Laplaciano de superfície com m=3. Quanto à análise de redes, nomeadamente nas métricas de transitividade, caminho característico e eficiência global, se verificarmos os gráficos e tabelas apresentadas, apesar de serem notados ligeiros desvios quer nas curvas quer em valores de correlação ou variação intra-sessão, o nível de significância é quase sempre atingido, independentemente da rigidez do Laplaciano aplicado, para a mesma sessão. Posto isto, não é possível reportar claramente uma relação causa-efeito vantajosa decorrente da aplicação do Laplaciano de superfície nos dados aqui tratados. De facto, reitera-se que, pela análise topológica se confirma que este possa estar associado a um filtro espacial, mas nas restantes análises não se consegue confirmar se este “melhorou ou não” os nossos dados.Neurofeedback (NF) consists in measuring brain activity and presenting a real-time representation of a brain activity pattern of interest to an individual, while instructing him to manipulate the feedback representation through self-regulation. The neurophysiological basis for NF remains to be fully elucidated, whereas several studies support that NF training tends to reorganize the brain networks. Only a handful of studies compare how different feedback sensory modalities affect the outcomes of EEG-based NF training, and none of them analyzes such effect on the functional connectivity or network metrics. In this project, we evaluate how using different feedback modalities on the EEG-based NFtraining will affect the brain’s functional connectivity, by analyzing previously collected data from a total of 20 healthy subjects, who underwent four sessions of upper-alpha (UA) band EEG-based NF training, with different feedback modalities (visual, auditory, or virtual reality (VR)). The EEG data was preprocessed and re-referenced with three different methods for posterior comparison, the common average reference (avgREF), and spline lines Surface Laplacian with stiffness parameters equals 4 and 3. The data were evaluated in terms of: i) the within-sessions’ variations of the relative amplitude of the UA at the Cz channel, ii) relative band amplitude topological distribution across sets and sessions, iii) the within-sessions’ variations of the UA functional connectivity patterns, computed with the imaginary part of coherency, and iv) an UA band network analysis of the metrics betweenness centrality, strength, transitivity, charpath and global efficiency. Our results suggest that the UA EEG-based NF-training is associated with an early increment of functional connections with channels over parietal areas (e.g. Pz), independently of the feedback sensory modality. All the modalities, except the VR, which had a reduced sample, verify statistically significant intra-session increases in the transitivity and global efficiency, while showing statistically significant intra-session decreases of the charpath, suggesting that this protocol promotes both clustered and integrated brain activity. While for the visual NF-training group the third session seems to be a breakthrough point, where the number of functional connections stabilize, for the auditory NF-training group longer lasting “variations” are reported. Through the topological analysis we confirm that the application of Laplacian leads to higher spatial resolutions on the EEG data. Regarding the connectivity analysis and network analysis, we note that the application of the Surface Laplacian creates different values when compared to the avgREF data, yet no advantageous outcome can be reported

    Image Restoration Under Adverse Illumination for Various Applications

    Get PDF
    Many images are captured in sub-optimal environment, resulting in various kinds of degradations, such as noise, blur, and shadow. Adverse illumination is one of the most important factors resulting in image degradation with color and illumination distortion or even unidentified image content. Degradation caused by the adverse illumination makes the images suffer from worse visual quality, which might also lead to negative effects on high-level perception tasks, e.g., object detection. Image restoration under adverse illumination is an effective way to remove such kind of degradations to obtain visual pleasing images. Existing state-of-the-art deep neural networks (DNNs) based image restoration methods have achieved impressive performance for image visual quality improvement. However, different real-world applications require the image restoration under adverse illumination to achieve different goals. For example, in the computational photography field, visually pleasing image is desired in the smartphone photography. Nevertheless, for traffic surveillance and autonomous driving in the low light or nighttime scenario, high-level perception tasks, \e.g., object detection, become more important to ensure safe and robust driving performance. Therefore, in this dissertation, we try to explore DNN-based image restoration solutions for images captured under adverse illumination in three important applications: 1) image visual quality enhancement, 2) object detection improvement, and 3) enhanced image visual quality and better detection performance simultaneously. First, in the computational photography field, visually pleasing images are desired. We take shadow removal task as an example to fully explore image visual quality enhancement. Shadow removal is still a challenging task due to its inherent background-dependent and spatial-variant properties, leading to unknown and diverse shadow patterns. We propose a novel solution by formulating this task as an exposure fusion problem to address the challenges. We propose shadow-aware FusionNet to `smartly\u27 fuse multiple over-exposure images with pixel-wise fusion weight maps, and boundary-aware RefineNet to eliminate the remaining shadow trace further. Experiment results show that our method outperforms other CNN-based methods in three datasets. Second, we explore the application of CNN-based night-to-day image translation for improving vehicle detection in traffic surveillance that is important for safe and robust driving. We propose a detail-preserving method to implement the nighttime to daytime image translation and thus adapt daytime trained detection model to nighttime vehicle detection. We utilize StyleMix method to acquire paired images of daytime and nighttime for the nighttime to daytime image translation training. The translation is implemented based on kernel prediction network to avoid texture corruption. Experimental results showed that the proposed method can better address the nighttime vehicle detection task by reusing the daytime domain knowledge. Third, we explore the image visual quality and facial landmark detection improvement simultaneously. For the portrait images captured in the wild, the facial landmark detection can be affected by the cast shadow. We construct a novel benchmark SHAREL covering diverse face shadow patterns with different intensities, sizes, shapes, and locations to study the effects of shadow removal on facial landmark detection. Moreover, we propose a novel adversarial shadow attack to mine hard shadow patterns. We conduct extensive analysis on three shadow removal methods and three landmark detectors. Then, we design a novel landmark detection-aware shadow removal framework, which empowers shadow removal to achieve higher restoration quality and enhances the shadow robustness of deployed facial landmark detectors

    Data-guided statistical sparse measurements modeling for compressive sensing

    Get PDF
    Digital image acquisition can be a time consuming process for situations where high spatial resolution is required. As such, optimizing the acquisition mechanism is of high importance for many measurement applications. Acquiring such data through a dynamically small subset of measurement locations can address this problem. In such a case, the measured information can be regarded as incomplete, which necessitates the application of special reconstruction tools to recover the original data set. The reconstruction can be performed based on the concept of sparse signal representation. Recovering signals and images from their sub-Nyquist measurements forms the core idea of compressive sensing (CS). In this work, a CS-based data-guided statistical sparse measurements method is presented, implemented and evaluated. This method significantly improves image reconstruction from sparse measurements. In the data-guided statistical sparse measurements approach, signal sampling distribution is optimized for improving image reconstruction performance. The sampling distribution is based on underlying data rather than the commonly used uniform random distribution. The optimal sampling pattern probability is accomplished by learning process through two methods - direct and indirect. The direct method is implemented for learning a nonparametric probability density function directly from the dataset. The indirect learning method is implemented for cases where a mapping between extracted features and the probability density function is required. The unified model is implemented for different representation domains, including frequency domain and spatial domain. Experiments were performed for multiple applications such as optical coherence tomography, bridge structure vibration, robotic vision, 3D laser range measurements and fluorescence microscopy. Results show that the data-guided statistical sparse measurements method significantly outperforms the conventional CS reconstruction performance. Data-guided statistical sparse measurements method achieves much higher reconstruction signal-to-noise ratio for the same compression rate as the conventional CS. Alternatively, Data-guided statistical sparse measurements method achieves similar reconstruction signal-to-noise ratio as the conventional CS with significantly fewer samples

    Deep learning-based improvement for the outcomes of glaucoma clinical trials

    Get PDF
    Glaucoma is the leading cause of irreversible blindness worldwide. It is a progressive optic neuropathy in which retinal ganglion cell (RGC) axon loss, probably as a consequence of damage at the optic disc, causes a loss of vision, predominantly affecting the mid-peripheral visual field (VF). Glaucoma results in a decrease in vision-related quality of life and, therefore, early detection and evaluation of disease progression rates is crucial in order to assess the risk of functional impairment and to establish sound treatment strategies. The aim of my research is to improve glaucoma diagnosis by enhancing state of the art analyses of glaucoma clinical trial outcomes using advanced analytical methods. This knowledge would also help better design and analyse clinical trials, providing evidence for re-evaluating existing medications, facilitating diagnosis and suggesting novel disease management. To facilitate my objective methodology, this thesis provides the following contributions: (i) I developed deep learning-based super-resolution (SR) techniques for optical coherence tomography (OCT) image enhancement and demonstrated that using super-resolved images improves the statistical power of clinical trials, (ii) I developed a deep learning algorithm for segmentation of retinal OCT images, showing that the methodology consistently produces more accurate segmentations than state-of-the-art networks, (iii) I developed a deep learning framework for refining the relationship between structural and functional measurements and demonstrated that the mapping is significantly improved over previous techniques, iv) I developed a probabilistic method and demonstrated that glaucomatous disc haemorrhages are influenced by a possible systemic factor that makes both eyes bleed simultaneously. v) I recalculated VF slopes, using the retinal never fiber layer thickness (RNFLT) from the super-resolved OCT as a Bayesian prior and demonstrated that use of VF rates with the Bayesian prior as the outcome measure leads to a reduction in the sample size required to distinguish treatment arms in a clinical trial

    Topics in Adaptive Optics

    Get PDF
    Advances in adaptive optics technology and applications move forward at a rapid pace. The basic idea of wavefront compensation in real-time has been around since the mid 1970s. The first widely used application of adaptive optics was for compensating atmospheric turbulence effects in astronomical imaging and laser beam propagation. While some topics have been researched and reported for years, even decades, new applications and advances in the supporting technologies occur almost daily. This book brings together 11 original chapters related to adaptive optics, written by an international group of invited authors. Topics include atmospheric turbulence characterization, astronomy with large telescopes, image post-processing, high power laser distortion compensation, adaptive optics and the human eye, wavefront sensors, and deformable mirrors

    Coherent and Holographic Imaging Methods for Immersive Near-Eye Displays

    Get PDF
    Lähinäytöt on suunniteltu tarjoamaan realistisia kolmiulotteisia katselukokemuksia, joille on merkittävää tarvetta esimerkiksi työkoneiden etäkäytössä ja 3D-suunnittelussa. Nykyaikaiset lähinäytöt tuottavat kuitenkin edelleen ristiriitaisia visuaalisia vihjeitä, jotka heikentävät immersiivistä kokemusta ja haittaavat niiden miellyttävää käyttöä. Merkittävänä ratkaisuvaihtoehtona pidetään koherentin valon, kuten laservalon, käyttöä näytön valaistukseen, millä voidaan korjata nykyisten lähinäyttöjen puutteita. Erityisesti koherentti valaistus mahdollistaa holografisen kuvantamisen, jota käyttävät holografiset näytöt voivat tarkasti jäljitellä kolmiulotteisten mallien todellisia valoaaltoja. Koherentin valon käyttäminen näyttöjen valaisemiseen aiheuttaa kuitenkin huomiota vaativaa korkean kontrastin häiriötä pilkkukuvioiden muodossa. Lisäksi holografisten näyttöjen laskentamenetelmät ovat laskennallisesti vaativia ja asettavat uusia haasteita analyysin, pilkkuhäiriön ja valon mallintamisen suhteen. Tässä väitöskirjassa tutkitaan laskennallisia menetelmiä lähinäytöille koherentissa kuvantamisjärjestelmässä käyttäen signaalinkäsittelyä, koneoppimista sekä geometrista (säde) ja fysikaalista (aalto) optiikan mallintamista. Työn ensimmäisessä osassa keskitytään holografisten kuvantamismuotojen analysointiin sekä kehitetään hologrammien laskennallisia menetelmiä. Holografian korkeiden laskentavaatimusten ratkaisemiseksi otamme käyttöön holografiset stereogrammit holografisen datan likimääräisenä esitysmuotona. Tarkastelemme kyseisen esitysmuodon visuaalista oikeellisuutta kehittämällä analyysikehyksen holografisen stereogrammin tarjoamien visuaalisten vihjeiden tarkkuudelle akkommodaatiota varten suhteessa sen suunnitteluparametreihin. Lisäksi ehdotamme signaalinkäsittelyratkaisua pilkkuhäiriön vähentämiseksi, ratkaistaksemme nykyisten menetelmien valon mallintamiseen liittyvät visuaalisia artefakteja aiheuttavat ongelmat. Kehitämme myös uudenlaisen holografisen kuvantamismenetelmän, jolla voidaan mallintaa tarkasti valon käyttäytymistä haastavissa olosuhteissa, kuten peiliheijastuksissa. Väitöskirjan toisessa osassa lähestytään koherentin näyttökuvantamisen laskennallista taakkaa koneoppimisen avulla. Kehitämme koherentin akkommodaatioinvariantin lähinäytön suunnittelukehyksen, jossa optimoidaan yhtäaikaisesti näytön staattista optiikka ja näytön kuvan esikäsittelyverkkoa. Lopuksi nopeutamme ehdottamaamme uutta holografista kuvantamismenetelmää koneoppimisen avulla reaaliaikaisia sovelluksia varten. Kyseiseen ratkaisuun sisältyy myös tehokkaan menettelyn kehittäminen funktionaalisten satunnais-3D-ympäristöjen tuottamiseksi. Kehittämämme menetelmä mahdollistaa suurten synteettisten moninäkökulmaisten kuvien datasettien tuottamisen, joilla voidaan kouluttaa sopivia neuroverkkoja mallintamaan holografista kuvantamismenetelmäämme reaaliajassa. Kaiken kaikkiaan tässä työssä kehitettyjen menetelmien osoitetaan olevan erittäin kilpailukykyisiä uusimpien koherentin valon lähinäyttöjen laskentamenetelmien kanssa. Työn tuloksena nähdään kaksi vaihtoehtoista lähestymistapaa ristiriitaisten visuaalisten vihjeiden aiheuttamien nykyisten lähinäyttöongelmien ratkaisemiseksi joko staattisella tai dynaamisella optiikalla ja reaaliaikaiseen käyttöön soveltuvilla laskentamenetelmillä. Esitetyt tulokset ovat näin ollen tärkeitä seuraavan sukupolven immersiivisille lähinäytöille.Near-eye displays have been designed to provide realistic 3D viewing experience, strongly demanded in applications, such as remote machine operation, entertainment, and 3D design. However, contemporary near-eye displays still generate conflicting visual cues which degrade the immersive experience and hinders their comfortable use. Approaches using coherent, e.g., laser light for display illumination have been considered prominent for tackling the current near-eye display deficiencies. Coherent illumination enables holographic imaging whereas holographic displays are expected to accurately recreate the true light waves of a desired 3D scene. However, the use of coherent light for driving displays introduces additional high contrast noise in the form of speckle patterns, which has to be taken care of. Furthermore, imaging methods for holographic displays are computationally demanding and impose new challenges in analysis, speckle noise and light modelling. This thesis examines computational methods for near-eye displays in the coherent imaging regime using signal processing, machine learning, and geometrical (ray) and physical (wave) optics modeling. In the first part of the thesis, we concentrate on analysis of holographic imaging modalities and develop corresponding computational methods. To tackle the high computational demands of holography, we adopt holographic stereograms as an approximative holographic data representation. We address the visual correctness of such representation by developing a framework for analyzing the accuracy of accommodation visual cues provided by a holographic stereogram in relation to its design parameters. Additionally, we propose a signal processing solution for speckle noise reduction to overcome existing issues in light modelling causing visual artefacts. We also develop a novel holographic imaging method to accurately model lighting effects in challenging conditions, such as mirror reflections. In the second part of the thesis, we approach the computational complexity aspects of coherent display imaging through deep learning. We develop a coherent accommodation-invariant near-eye display framework to jointly optimize static display optics and a display image pre-processing network. Finally, we accelerate the corresponding novel holographic imaging method via deep learning aimed at real-time applications. This includes developing an efficient procedure for generating functional random 3D scenes for forming a large synthetic data set of multiperspective images, and training a neural network to approximate the holographic imaging method under the real-time processing constraints. Altogether, the methods developed in this thesis are shown to be highly competitive with the state-of-the-art computational methods for coherent-light near-eye displays. The results of the work demonstrate two alternative approaches for resolving the existing near-eye display problems of conflicting visual cues using either static or dynamic optics and computational methods suitable for real-time use. The presented results are therefore instrumental for the next-generation immersive near-eye displays
    corecore