Adapting Computer Vision Models To Limitations On Input Dimensionality And Model Complexity

When considering instances of distributed systems where visual sensors communicate with remote predictive models, data traffic is limited to the capacity of communication channels, and hardware limits the processing of collected data prior to transmission. We study novel methods of adapting visual inference to limitations on complexity and data availability at test time, wherever the aforementioned limitations exist. Our contributions detailed in this thesis consider both task-specific and task-generic approaches to reducing the data requirement for inference, and evaluate our proposed methods on a wide range of computer vision tasks. This thesis makes four distinct contributions: (i) We investigate multi-class action classification via two-stream convolutional neural networks that directly ingest information extracted from compressed video bitstreams. We show that selective access to macroblock motion vector information provides a good low-dimensional approximation of the underlying optical flow in visual sequences. (ii) We devise a bitstream cropping method by which AVC/H.264 and H.265 bitstreams are reduced to the minimum amount of necessary elements for optical flow extraction, while maintaining compliance with codec standards. We additionally study the effect of codec rate-quality control on the sparsity and noise incurred on optical flow derived from resulting bitstreams, and do so for multiple coding standards. (iii) We demonstrate degrees of variability in the amount of data required for action classification, and leverage this to reduce the dimensionality of input volumes by inferring the required temporal extent for accurate classification prior to processing via learnable machines. (iv) We extend the Mixtures-of-Experts (MoE) paradigm to adapt the data cost of inference for any set of constituent experts. We postulate that the minimum acceptable data cost of inference varies for different input space partitions, and consider mixtures where each expert is designed to meet a different set of constraints on input dimensionality. To take advantage of the flexibility of such mixtures in processing different input representations and modalities, we train biased gating functions such that experts requiring less information to make their inferences are favoured to others. We finally note that, our proposed data utility optimization solutions include a learnable component which considers specified priorities on the amount of information to be used prior to inference, and can be realized for any combination of tasks, modalities, and constraints on available data

Computational Intelligence Methodologies for Soft Sensors Development in Industrial Processes

Tese de doutoramento em Engenharia Electrotécnica e de Computadores, no ramo de especialização em Automação e Robótica, apresentada ao Departamento de Engenharia Electrotécnica e de Computadores da Faculdade de Ciências e Tecnologia da Universidade de Coimbra.Sensores virtuais são modelos inferenciais que utilizam sensores disponíveis online (e.g.\ temperatura, pressão, vazão, etc) para prever variáveis relacionadas com a qualidade do processo, que não podem ser medidas de forma automática, ou só podem ser medidas por um custo elevado, de forma esporádica, ou com longos atrasos (e.g.\ análises laboratoriais). Sensores virtuais são construídos usando usando os dados históricos de processo, geralmente fornecidos pelo sistema de controle de supervisão e aquisição de dados (SCADA) e pelas anotações das medições de laboratório. No desenvolvimento dos sensores virtuais, há muitas questões para lidar. As principais questões são o tratamento de dados em falta, a detecção de outliers, a selecção das variáveis de entrada, o treino do modelo, a validação, e a manutenção do sensor virtual. Esta tese centra-se em três destas questões, nomeadamente, a selecção de variáveis de entrada, o treino do modelo e a manutenção do sensor virtual. Novas metodologias são propostas em cada uma destas áreas. A selecção das variáveis de entrada é baseada na rede neuronal multilayer perceptron (o modelo de regressão não linear mais popular em aplicações de sensores virtuais). A segunda questão, o treino do modelo, é tratado no contexto de múltiplos modos de operação. Exemplos de múltiplos modos de operação são a variação da carga diurna de uma central de produção energia, a operação verão-inverno de uma refinaria, etc. Nesta tese, para treinar um modelo no contexto dos múltiplos modos de operação, o modelo de regressão por mínimos quadrados parciais (PLS), um método muito difundido na literatura de quimiometria e um dos métodos mais utilizados na indústria, é inserido no método de mistura de especialistas (ME), derivando assim o método a mistura de modelos mínimos quadrados parciais (Mix-PLS) especialistas. O terceiro problema está relacionado com a manutenção do sensor virtual. Na manutenção do sensor virtual, o modelo é actualizado utilizando amostras recentes do processo. A maneira mais comum de fazê-lo é através de aprendizagem exponencial-recursiva dos parâmetros do modelo. Na aprendizagem exponencial-recursiva, é utilizado um factor de esquecimento para dar exponencialmente menos pesos para as amostras mais antigas. O factor de esquecimento está directamente relacionado com o número "eficaz" de amostras, e valores baixos do factor de esquecimento podem proporcionar os mesmos problemas enfrentados na modelação de sistemas estáticos, tais como overfitting, mau desempenho de predição, etc. Para resolver este problema, um novo modelo, baseado na mistura de modelos de regressão linear univariados (portanto de baixa dimensionalidade), é proposto, permitindo a utilização de baixos valores de factor de esquecimento. Todos os métodos propostos nesta tese são testado em conjuntos de dados obtidos de processos reais. Cada método proposto é comparado com os respectivos métodos do estado da arte, validando assim as abordagens propostas.Data-driven soft sensors are inferential models that use on-line available sensors (e.g. temperature, pressure, flow rate, etc) to predict quality variables which cannot be automatically measured at all, or can only be measured at high cost, sporadically, or with high delays (e.g. laboratory analysis). Soft sensors are built using historical data of the process, usually provided from the supervisory control and data acquisition (SCADA) system or obtained from the laboratory annotations/measurements. In the soft sensor development, there are many issues to deal with. The main issues are the treatment of missing data, outliers detection, selection of input variables, model training, validation, and soft sensor maintenance. This thesis focuses on three of these issues, namely the selection of input variables, model training, and soft sensor maintenance. Novel methodologies are proposed in each of these areas. The selection of input variables is based on the multilayer perceptron (MLP) neural network model (the most popular non-linear regression model in soft sensors applications). The second issue, the model training, is addressed in the context of multiple operating modes. Examples of multiple operating modes are diurnal load variation of a power plant, summer-winter operation of a refinery, etc. In this thesis, to train a model in the context of multiple modes context, the partial least squares regression (PLS), a well know method in the chemometrics literature and one of the mostly used methods in industry, is inserted into the mixture of experts (ME) framework, deriving so the mixture of partial least square (Mix-PLS) regression. The Mix-PLS is able to characterize multiple operating modes. The third problem is related to soft sensor maintenance. In soft sensor maintenance, the model is updated using recent samples of the process. The most common way to do so is by the exponentially recursive learning of parameters, using the incoming samples of the process. In exponentially recursive learning, a forgetting factor is used to give exponentially less weight to older samples. In many applications, small values of the forgetting factor can lead to better predictive performance. However, the forgetting factor is directly related to the “effective” number of samples, and low values of forgetting factor can bring the same problem faced when modeling static systems, such as overfitting, poor prediction performance, etc. To solve this problem, a new model, based on the mixture of univariate (thus low dimensional) linear regression models is proposed (MULRM), allowing the use of small values of forgetting factor. All the methods proposed in this thesis are evaluated in soft sensors data sets coming from real-world processes. Each of the proposed methods is compared with the corresponding state of the art methods, thus validating the proposed approaches.FCT - SFRH/BD/63454/200

Computer-Aided Assessment of Tuberculosis with Radiological Imaging: From rule-based methods to Deep Learning

Mención Internacional en el título de doctorTuberculosis (TB) is an infectious disease caused by Mycobacterium tuberculosis (Mtb.) that produces pulmonary damage due to its airborne nature. This fact facilitates the disease fast-spreading, which, according to the World Health Organization (WHO), in 2021 caused 1.2 million deaths and 9.9 million new cases. Traditionally, TB has been considered a binary disease (latent/active) due to the limited specificity of the traditional diagnostic tests. Such a simple model causes difficulties in the longitudinal assessment of pulmonary affectation needed for the development of novel drugs and to control the spread of the disease. Fortunately, X-Ray Computed Tomography (CT) images enable capturing specific manifestations of TB that are undetectable using regular diagnostic tests, which suffer from limited specificity. In conventional workflows, expert radiologists inspect the CT images. However, this procedure is unfeasible to process the thousands of volume images belonging to the different TB animal models and humans required for a suitable (pre-)clinical trial. To achieve suitable results, automatization of different image analysis processes is a must to quantify TB. It is also advisable to measure the uncertainty associated with this process and model causal relationships between the specific mechanisms that characterize each animal model and its level of damage. Thus, in this thesis, we introduce a set of novel methods based on the state of the art Artificial Intelligence (AI) and Computer Vision (CV). Initially, we present an algorithm to assess Pathological Lung Segmentation (PLS) employing an unsupervised rule-based model which was traditionally considered a needed step before biomarker extraction. This procedure allows robust segmentation in a Mtb. infection model (Dice Similarity Coefficient, DSC, 94%±4%, Hausdorff Distance, HD, 8.64mm±7.36mm) of damaged lungs with lesions attached to the parenchyma and affected by respiratory movement artefacts. Next, a Gaussian Mixture Model ruled by an Expectation-Maximization (EM) algorithm is employed to automatically quantify the burden of Mtb.using biomarkers extracted from the segmented CT images. This approach achieves a strong correlation (R2 ≈ 0.8) between our automatic method and manual extraction. Consequently, Chapter 3 introduces a model to automate the identification of TB lesions and the characterization of disease progression. To this aim, the method employs the Statistical Region Merging algorithm to detect lesions subsequently characterized by texture features that feed a Random Forest (RF) estimator. The proposed procedure enables a selection of a simple but powerful model able to classify abnormal tissue. The latest works base their methodology on Deep Learning (DL). Chapter 4 extends the classification of TB lesions. Namely, we introduce a computational model to infer TB manifestations present in each lung lobe of CT scans by employing the associated radiologist reports as ground truth. We do so instead of using the classical manually delimited segmentation masks. The model adjusts the three-dimensional architecture, V-Net, to a multitask classification context in which loss function is weighted by homoscedastic uncertainty. Besides, the method employs Self-Normalizing Neural Networks (SNNs) for regularization. Our results are promising with a Root Mean Square Error of 1.14 in the number of nodules and F1-scores above 0.85 for the most prevalent TB lesions (i.e., conglomerations, cavitations, consolidations, trees in bud) when considering the whole lung. In Chapter 5, we present a DL model capable of extracting disentangled information from images of different animal models, as well as information of the mechanisms that generate the CT volumes. The method provides the segmentation mask of axial slices from three animal models of different species employing a single trained architecture. It also infers the level of TB damage and generates counterfactual images. So, with this methodology, we offer an alternative to promote generalization and explainable AI models. To sum up, the thesis presents a collection of valuable tools to automate the quantification of pathological lungs and moreover extend the methodology to provide more explainable results which are vital for drug development purposes. Chapter 6 elaborates on these conclusions.Programa de Doctorado en Multimedia y Comunicaciones por la Universidad Carlos III de Madrid y la Universidad Rey Juan CarlosPresidenta: María Jesús Ledesma Carbayo.- Secretario: David Expósito Singh.- Vocal: Clarisa Sánchez Gutiérre

Detection and description of pulmonary nodules through 2D and 3D clustering

Precise 3D automated detection, description and classification of pulmonary nodules offer the potential for early diagnosis of cancer and greater efficiency in the reading of computerised tomography (CT) images. CT scan centres are currently experiencing high loads and experts shortage, especially in developing countries such as Iraq where the results of the current research will be used. This motivates the researchers to address these problems and challenges by developing automated processes for the early detection and efficient description of cancer cases. This research attempts to reduce workloads, enhance the patient throughput and improve the diagnosis performance. To achieve this goal, the study selects techniques for segmentation, classification, detection and implements the best candidates alongside a novel automated approach. Techniques for each stage in the process are quantitatively evaluated to select the best performance against standard data for lung cancer. In addition, the ideal approach is identified by comparing them against other works in detecting and describing pulmonary nodules. This work detects and describes the nodules and their characteristics in several stages: automated lung segmentation from the background, automated 2D and 3D clustering of vessels and nodules, applying shape and textures features, classification and automatic measurement of nodule characteristics. This work is tested on standard CT lung image data and shows promising results, matching or close to experts’ diagnosis in the nodules number and their features (size/volume, location) and in terms the accuracy and automation. It also achieved a classification accuracy of 98% and efficient results in measuring the nodules’ volume automatically

Combined supervised and unsupervised learning to identify subclasses of disease for better prediction

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonDisease subtyping, which aids in the development of personalised treatments, remains a challenge in data analysis because of the many different ways to group patients based upon their data. However, if I can identify subclasses of disease, this will help to develop better models that are more specific to individuals and should therefore improve prediction and understanding of the underlying characteristics of the disease in question. In addition, patients might suffer from multiple disease complications. Models that are tailored to individuals could improve both prediction of multiple complications and understanding of underlying disease characteristics. However, AI models can become outdated over time due to either sudden changes in the underlying data, such as those caused by new measurement methods, or incremental changes, such as the ageing of the study population. This thesis proposes a new algorithm that integrates consensus clustering methods with classification in order to overcome issues with sample bias. The method was tested on a freely available dataset of real-world breast cancer cases and data from a London hospital on systemic sclerosis, a rare and potentially fatal condition. The results show that nearest consensus clustering classification improves accuracy and prediction significantly when this algorithm is compared with competitive similar methods. In addition, this thesis proposes a new algorithm that integrates latent class models with classification. The new algorithm uses latent class models to cluster patients within groups; this results in improved classification and aids in the understanding of the underlying differences of the discovered groups. The method was tested on data from patients with systemic sclerosis (SSc), a rare and potentially fatal condition, and coronary heart disease. Results show that the latent class multi-label classification (MLC) model improves accuracy when compared with competitive similar methods. Finally, this thesis implemented the updated concept drift method (DDM) to monitor AI models over time and detect drifts when they occur. The method was tested on data from patients with SSc and patients with coronavirus disease (COVID)

A review of machine learning applications for the proton MR spectroscopy workflow

This literature review presents a comprehensive overview of machine learning (ML) applications in proton MR spectroscopy (MRS). As the use of ML techniques in MRS continues to grow, this review aims to provide the MRS community with a structured overview of the state-of-the-art methods. Specifically, we examine and summarize studies published between 2017 and 2023 from major journals in the MR field. We categorize these studies based on a typical MRS workflow, including data acquisition, processing, analysis, and artificial data generation. Our review reveals that ML in MRS is still in its early stages, with a primary focus on processing and analysis techniques, and less attention given to data acquisition. We also found that many studies use similar model architectures, with little comparison to alternative architectures. Additionally, the generation of artificial data is a crucial topic, with no consistent method for its generation. Furthermore, many studies demonstrate that artificial data suffers from generalization issues when tested on in vivo data. We also conclude that risks related to ML models should be addressed, particularly for clinical applications. Therefore, output uncertainty measures and model biases are critical to investigate. Nonetheless, the rapid development of ML in MRS and the promising results from the reviewed studies justify further research in this field.</p

Gene selection for classification of microarray data based on the Bayes error

<p>Abstract</p> <p>Background</p> <p>With DNA microarray data, selecting a compact subset of discriminative genes from thousands of genes is a critical step for accurate classification of phenotypes for, e.g., disease diagnosis. Several widely used gene selection methods often select top-ranked genes according to their individual discriminative power in classifying samples into distinct categories, without considering correlations among genes. A limitation of these gene selection methods is that they may result in gene sets with some redundancy and yield an unnecessary large number of candidate genes for classification analyses. Some latest studies show that incorporating gene to gene correlations into gene selection can remove redundant genes and improve classification accuracy.</p> <p>Results</p> <p>In this study, we propose a new method, Based Bayes error Filter (BBF), to select relevant genes and remove redundant genes in classification analyses of microarray data. The effectiveness and accuracy of this method is demonstrated through analyses of five publicly available microarray datasets. The results show that our gene selection method is capable of achieving better accuracies than previous studies, while being able to effectively select relevant genes, remove redundant genes and obtain efficient and small gene sets for sample classification purposes.</p> <p>Conclusion</p> <p>The proposed method can effectively identify a compact set of genes with high classification accuracy. This study also indicates that application of the Bayes error is a feasible and effective wayfor removing redundant genes in gene selection.</p