Implicit Neural Representation for Cooperative Low-light Image Enhancement

The following three factors restrict the application of existing low-light image enhancement methods: unpredictable brightness degradation and noise, inherent gap between metric-favorable and visual-friendly versions, and the limited paired training data. To address these limitations, we propose an implicit Neural Representation method for Cooperative low-light image enhancement, dubbed NeRCo. It robustly recovers perceptual-friendly results in an unsupervised manner. Concretely, NeRCo unifies the diverse degradation factors of real-world scenes with a controllable fitting function, leading to better robustness. In addition, for the output results, we introduce semantic-orientated supervision with priors from the pre-trained vision-language model. Instead of merely following reference images, it encourages results to meet subjective expectations, finding more visual-friendly solutions. Further, to ease the reliance on paired data and reduce solution space, we develop a dual-closed-loop constrained enhancement module. It is trained cooperatively with other affiliated modules in a self-supervised manner. Finally, extensive experiments demonstrate the robustness and superior effectiveness of our proposed NeRCo. Our code is available at https://github.com/Ysz2022/NeRCo

Artificial Intelligence in Civil Infrastructure Health Monitoring—historical Perspectives, Current Trends, and Future Visions

Over the past 2 decades, the use of artificial intelligence (AI) has exponentially increased toward complete automation of structural inspection and assessment tasks. This trend will continue to rise in image processing as unmanned aerial systems (UAS) and the internet of things (IoT) markets are expected to expand at a compound annual growth rate of 57.5% and 26%, respectively, from 2021 to 2028. This paper aims to catalog the milestone development work, summarize the current research trends, and envision a few future research directions in the innovative application of AI in civil infrastructure health monitoring. A blow-by-blow account of the major technology progression in this research field is provided in a chronological order. Detailed applications, key contributions, and performance measures of each milestone publication are presented. Representative technologies are detailed to demonstrate current research trends. A road map for future research is outlined to address contemporary issues such as explainable and physics-informed AI. This paper will provide readers with a lucid memoir of the historical progress, a good sense of the current trends, and a clear vision for future research

Wavelets and sparse methods for image reconstruction and classification in neuroimaging

This dissertation contributes to neuroimaging literature in the fields of compressed sensing magnetic resonance imaging (CS-MRI) and image-based detection of Alzheimer’s disease (AD). It consists of three main contributions, based on wavelets and sparse methods. The first contribution is a method for wavelet packet basis optimisation for sparse approximation and compressed sensing reconstruction of magnetic resonance (MR) images of the brain. The proposed method is based on the basis search algorithm developed by Coifman and Wickerhauser, with a cost function designed specifically for compressed sensing. It is tested on MR images available from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). The second contribution consists of evaluating and comparing several sparse classification methods in an application to detection of AD based on positron emission tomography (PET) images of the brain. This comparison includes univariate feature selection, feature clustering and classifiers that automatically select a small subset of features due to their mathematical or algorithmic construction. The evaluation is based on PET images available from ADNI. The third contribution is proposing an extension of wavelet-based scattering networks (originally proposed by Mallat and Bruna) to three-dimensional tomographic images. The proposed extension is evaluated as a feature representation in an application to detection of AD based on MR images available from ADNI. There are several possible extensions of the work presented in this dissertation. The wavelet packet basis search method proposed in the first contribution can be improved to take into account the coherence between the sparse approximation basis and the sensing basis. The evaluation presented in the second contribution can be extended with additional algorithms to make it more comprehensive. The three-dimensional scattering networks that are the core part of the third contribution can be combined with other machine learning methods, such as manifold learning or deep convolutional neural networks. As a whole, the methods proposed in this dissertation contribute to the work towards efficient screening for Alzheimer’s disease, by making MRI scans of the brain faster and helping to automate image analysis for AD detection. The first contribution is a method for wavelet packet basis optimisation for sparse approximation and compressed sensing reconstruction of magnetic resonance (MR) images of the brain. The proposed method is based on the basis search algorithm developed by Coifman and Wickerhauser, with a cost function designed specifically for compressed sensing. It is tested on MR images available from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). The second contribution consists of evaluating and comparing several sparse classification methods in an application to detection of AD based on positron emission tomography (PET) images of the brain. This comparison includes univariate feature selection, feature clustering and classifiers that automatically select a small subset of features due to their mathematical or algorithmic construction. The evaluation is based on PET images available from ADNI. The third contribution is proposing an extension of wavelet-based scattering networks (originally proposed by Mallat and Bruna) to three-dimensional tomographic images. The proposed extension is evaluated as a feature representation in an application to detection of AD based on MR images available from ADNI. There are several possible extensions of the work presented in this dissertation. The wavelet packet basis search method proposed in the first contribution can be improved to take into account the coherence between the sparse approximation basis and the sensing basis. The evaluation presented in the second contribution can be extended with additional algorithms to make it more comprehensive. The three-dimensional scattering networks that are the core part of the third contribution can be combined with other machine learning methods, such as manifold learning or deep convolutional neural networks. This dissertation contributes to neuroimaging literature in the fields of compressed sensing magnetic resonance imaging (CS-MRI) and image-based detection of Alzheimer’s disease (AD). It consists of three main contributions, based on wavelets and sparse methods. The first contribution is a method for wavelet packet basis optimisation for sparse approximation and compressed sensing reconstruction of magnetic resonance (MR) images of the brain. The proposed method is based on the basis search algorithm developed by Coifman and Wickerhauser, with a cost function designed specifically for compressed sensing. It is tested on MR images available from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). The second contribution consists of evaluating and comparing several sparse classification methods in an application to detection of AD based on positron emission tomography (PET) images of the brain. This comparison includes univariate feature selection, feature clustering and classifiers that automatically select a small subset of features due to their mathematical or algorithmic construction. The evaluation is based on PET images available from ADNI. The third contribution is proposing an extension of wavelet-based scattering networks (originally proposed by Mallat and Bruna) to three-dimensional tomographic images. The proposed extension is evaluated as a feature representation in an application to detection of AD based on MR images available from ADNI. There are several possible extensions of the work presented in this dissertation. The wavelet packet basis search method proposed in the first contribution can be improved to take into account the coherence between the sparse approximation basis and the sensing basis. The evaluation presented in the second contribution can be extended with additional algorithms to make it more comprehensive. The three-dimensional scattering networks that are the core part of the third contribution can be combined with other machine learning methods, such as manifold learning or deep convolutional neural networks. As a whole, the methods proposed in this dissertation contribute to the work towards efficient screening for Alzheimer’s disease, by making MRI scans of the brain faster and helping to automate image analysis for AD detection.Open Acces

Review : Deep learning in electron microscopy

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy

Low-Rank and Sparse Decomposition for Hyperspectral Image Enhancement and Clustering

In this dissertation, some new algorithms are developed for hyperspectral imaging analysis enhancement. Tensor data format is applied in hyperspectral dataset sparse and low-rank decomposition, which could enhance the classification and detection performance. And multi-view learning technique is applied in hyperspectral imaging clustering. Furthermore, kernel version of multi-view learning technique has been proposed, which could improve clustering performance. Most of low-rank and sparse decomposition algorithms are based on matrix data format for HSI analysis. As HSI contains high spectral dimensions, tensor based extended low-rank and sparse decomposition (TELRSD) is proposed in this dissertation for better performance of HSI classification with low-rank tensor part, and HSI detection with sparse tensor part. With this tensor based method, HSI is processed in 3D data format, and information between spectral bands and pixels maintain integrated during decomposition process. This proposed algorithm is compared with other state-of-art methods. And the experiment results show that TELRSD has the best performance among all those comparison algorithms. HSI clustering is an unsupervised task, which aims to group pixels into different groups without labeled information. Low-rank sparse subspace clustering (LRSSC) is the most popular algorithms for this clustering task. The spatial-spectral based multi-view low-rank sparse subspace clustering (SSMLC) algorithms is proposed in this dissertation, which extended LRSSC with multi-view learning technique. In this algorithm, spectral and spatial views are created to generate multi-view dataset of HSI, where spectral partition, morphological component analysis (MCA) and principle component analysis (PCA) are applied to create others views. Furthermore, kernel version of SSMLC (k-SSMLC) also has been investigated. The performance of SSMLC and k-SSMLC are compared with sparse subspace clustering (SSC), low-rank sparse subspace clustering (LRSSC), and spectral-spatial sparse subspace clustering (S4C). It has shown that SSMLC could improve the performance of LRSSC, and k-SSMLC has the best performance. The spectral clustering has been proved that it equivalent to non-negative matrix factorization (NMF) problem. In this case, NMF could be applied to the clustering problem. In order to include local and nonlinear features in data source, orthogonal NMF (ONMF), graph-regularized NMF (GNMF) and kernel NMF (k-NMF) has been proposed for better clustering performance. The non-linear orthogonal graph NMF combine both kernel, orthogonal and graph constraints in NMF (k-OGNMF), which push up the clustering performance further. In the HSI domain, kernel multi-view based orthogonal graph NMF (k-MOGNMF) is applied for subspace clustering, where k-OGNMF is extended with multi-view algorithm, and it has better performance and computation efficiency

Dissimilarity-based multiple instance classification and dictionary learning for bioacoustic signal recognition

In this thesis, two promising and actively researched fields from pattern recognition (PR) and digital signal processing (DSP) are studied, adapted and applied for the automated recognition of bioacoustic signals: (i) learning from weakly-labeled data, and (ii) dictionary-based decomposition. The document begins with an overview of the current methods and techniques applied for the automated recognition of bioacoustic signals, and an analysis of the impact of this technology at global and local scales. This is followed by a detailed description of my research on studying two approaches from the above-mentioned fields, multiple instance learning (MIL) and dictionary learning (DL), as solutions to particular challenges in bioacoustic data analysis. The most relevant contributions and findings of this thesis are the following ones: 1) the proposal of an unsupervised recording segmentation method of audio birdsong recordings that improves species classification with the benefit of an easier implementation since no manual handling of recordings is required; 2) the confirmation that, in the analyzed audio datasets, appropriate dissimilarity measures are those which capture most of the overall differences between bags, such as the modified Hausdorff distance and the mean minimum distance; 3) the adoption of dissimilarity adaptation techniques for the enhancement of dissimilarity-based multiple instance classification, along with the potential further enhancement of the classification performance by building dissimilarity spaces and increasing training set sizes; 4) the proposal of a framework for solving MIL problems by using the one nearest neighbor (1-NN) classifier; 5) a novel convolutive DL method for learning a representative dictionary from a collection of multiple-bird audio recordings; 6) such a DL method is successfully applied to spectrogram denoising and species classification; and, 7) an efficient online version of the DL method that outperforms other state-of-the-art batch and online methods, in both, computational cost and quality of the discovered patternsResumen : En esta tesis se estudian, adaptan y aplican dos prometedoras y activas áreas del reconocimiento de patrones (PR) y procesamiento digital de señales (DSP): (i) aprendizaje débilmente supervisado y (ii) descomposiciones basadas en diccionarios. Inicialmente se hace una revisión de los métodos y técnicas que actualmente se aplican en tareas de reconocimiento automatizado de señales bioacústicas y se describe el impacto de esta tecnología a escalas nacional y global. Posteriormente, la investigación se enfoca en el estudio de dos técnicas de las áreas antes mencionadas, aprendizaje multi-instancia (MIL) y aprendizaje de diccionarios (DL), como soluciones a retos particulares del análisis de datos bioacústicos. Las contribuciones y hallazgos ms relevantes de esta tesis son los siguientes: 1) se propone un método de segmentacin de grabaciones de audio que mejora la clasificación automatizada de especies, el cual es fácil de implementar ya que no necesita información supervisada de entrenamiento; 2) se confirma que, en los conjuntos de datos analizados, las medidas de disimilitudes que capturan las diferencias globales entre bolsas funcionan apropiadamente, tales como la distancia modificada de Hausdorff y la distancia media de los mínimos; 3) la adopción de técnicas de adaptación de disimilitudes para mejorar la clasificación multi-instancia, junto con el incremento potencial del desempeño por medio de la construcción de espacios de disimilitudes y el aumento del tamaño de los conjuntos de entrenamiento; 4) se presenta un esquema para la solución de problemas MIL por medio del clasificador del vecino ms cercano (1-NN); 5) se propone un método novedoso de DL, basado en convoluciones, para el aprendizaje automatizado de un diccionario representativo a partir de un conjunto de grabaciones de audio de múltiples vocalizaciones de aves; 6) dicho mtodo DL se utiliza exitosamente como técnica de reducción de ruido en espectrogramas y clasificación de grabaciones bioacústicas; y 7) un método DL, de procesamiento en línea, que supera otros métodos del estado del arte en costo computacional y calidad de los patrones descubiertosDoctorad