Scene Analysis Using Scale Invariant Feature Extraction and Probabilistic Modeling

Conventional pattern recognition systems have two components: feature analysis and pattern classification. For any object in an image, features could be considered as the major characteristic of the object either for object recognition or object tracking purpose. Features extracted from a training image, can be used to identify the object when attempting to locate the object in a test image containing many other objects. To perform reliable scene analysis, it is important that the features extracted from the training image are detectable even under changes in image scale, noise and illumination. Scale invariant feature has wide applications such as image classification, object recognition and object tracking in the image processing area. In this thesis, color feature and SIFT (scale invariant feature transform) are considered to be scale invariant feature. The classification, recognition and tracking result were evaluated with novel evaluation criterion and compared with some existing methods. I also studied different types of scale invariant feature for the purpose of solving scene analysis problems. I propose probabilistic models as the foundation of analysis scene scenario of images. In order to differential the content of image, I develop novel algorithms for the adaptive combination for multiple features extracted from images. I demonstrate the performance of the developed algorithm on several scene analysis tasks, including object tracking, video stabilization, medical video segmentation and scene classification

Hookworm and Bleeding Detection in WCE Images using Rusboost Classifier

Now-a-days, million ranges of individuals are having helminthiasis and this number has been increasing day by day. Automatic hookworm recognition could be a difficult task in medical field. Here projected a completely unique technique for detective work the helminthiasis from wireless capsule examination (WCE) pictures. During this paper initial adopted for WCE image with sweetening method by mistreatment Multi-scale twin Matched Filter (MDMF). Then, Piecewise Parallel Region Detection (PPRD) is employed to discover the parallel edges. This technique is extremely appropriate for detective work hookworm when put next to different standard technique

Anatomical Classification of the Gastrointestinal Tract Using Ensemble Transfer Learning

Endoscopy is a procedure used to visualize disorders of the gastrointestinal (GI) lumen. GI disorders can occur without symptoms, which is why gastroenterologists often recommend routine examinations of the GI tract. It allows a doctor to directly visualize the inside of the GI tract and identify the cause of symptoms, reducing the need for exploratory surgery or other invasive procedures. It can also detect the early stages of GI disorders, such as cancer, enabling prompt treatment that can improve outcomes. Endoscopic examinations generate significant numbers of GI images. Because of this vast amount of endoscopic image data, relying solely on human interpretation can be problematic. Artificial intelligence is gaining popularity in clinical medicine. Assist in medical image analysis and early detection of diseases, help with personalized treatment planning by analyzing a patient’s medical history and genomic data, and be used by surgical robots to improve precision and reduce invasiveness. It enables automated diagnosis, provides physicians with assistance, and may improve performance. One of the significant challenges is defining the specific anatomic locations of GI tract abnormalities. Clinicians can then determine appropriate treatment options, reducing the need for repetitive endoscopy. Due to the difficulty of collecting annotated data, very limited research has been conducted on the localization of anatomical locations by classification of endoscopy images. In this study, we present a classification of GI tract anatomical localization based on transfer learning and ensemble learning. Our approach involves the use of an autoencoder and the Xception model. The autoencoder was initially trained on thousands of unlabeled images, and the encoder then separated and used as a feature extractor. The Xception model was also used as a second model to extract features from the input images. The extracted feature vectors were then concatenated and fed into a Convolutional Neural Network for classification. This combination of models provides a powerful and versatile solution for image classification. By using the encoder as a feature extractor that can transfer the learned knowledge, it is possible to improve learning by allowing the model to focus on more relevant and useful data, which is extremely valuable when there are not enough appropriately labelled data. On the other hand, the Xception model provides additional feature extraction capabilities. Sometimes, one classifier is not enough in machine learning, as it depends on the problem we are trying to solve and the quality and quantity of data available. With ensemble learning, multiple learning networks can work together to create a stronger classifier. The final classification results are obtained by combining the information from both models through the CNN model. This approach demonstrates the potential for combining multiple models to improve the accuracy of image classification tasks in the medical domain. The HyperKvasir dataset is the main dataset used in this study. It contains 4,104 labelled and 99,417 unlabeled images taken at six different locations in the GI tract, including the cecum, ileum, pylorus, rectum, stomach, and Z line. After dataset preprocessing, which includes noise deduction and similarity removal, 871 labelled images remained for the purpose of this study. Our method was more accurate than state-of-the-art studies and had a higher F1 score while categorizing the input images into six different anatomical locations with less than a thousand labelled images. According to the results, feature extraction and ensemble learning increase accuracy by 5%, and a comparison with existing methods using the same dataset indicate improved performance and reduced cross entropy loss. The proposed method can therefore be used in the classification of endoscopy images

Uncertainty, interpretability and dataset limitations in Deep Learning

[eng] Deep Learning (DL) has gained traction in the last years thanks to the exponential increase in compute power. New techniques and methods are published at a daily basis, and records are being set across multiple disciplines. Undeniably, DL has brought a revolution to the machine learning field and to our lives. However, not everything has been resolved and some considerations must be taken into account. For instance, obtaining uncertainty measures and bounds is still an open problem. Models should be able to capture and express the confidence they have in their decisions, and Artificial Neural Networks (ANN) are known to lack in this regard. Be it through out of distribution samples, adversarial attacks, or simply unrelated or nonsensical inputs, ANN models demonstrate an unfounded and incorrect tendency to still output high probabilities. Likewise, interpretability remains an unresolved question. Some fields not only need but rely on being able to provide human interpretations of the thought process of models. ANNs, and specially deep models trained with DL, are hard to reason about. Last but not least, there is a tendency that indicates that models are getting deeper and more complex. At the same time, to cope with the increasing number of parameters, datasets are required to be of higher quality and, usually, larger. Not all research, and even less real world applications, can keep with the increasing demands. Therefore, taking into account the previous issues, the main aim of this thesis is to provide methods and frameworks to tackle each of them. These approaches should be applicable to any suitable field and dataset, and are employed with real world datasets as proof of concept. First, we propose a method that provides interpretability with respect to the results through uncertainty measures. The model in question is capable of reasoning about the uncertainty inherent in data and leverages that information to progressively refine its outputs. In particular, the method is applied to land cover segmentation, a classification task that aims to assign a type of land to each pixel in satellite images. The dataset and application serve to prove that the final uncertainty bound enables the end-user to reason about the possible errors in the segmentation result. Second, Recurrent Neural Networks are used as a method to create robust models towards lacking datasets, both in terms of size and class balance. We apply them to two different fields, road extraction in satellite images and Wireless Capsule Endoscopy (WCE). The former demonstrates that contextual information in the temporal axis of data can be used to create models that achieve comparable results to state-of-the-art while being less complex. The latter, in turn, proves that contextual information for polyp detection can be crucial to obtain models that generalize better and obtain higher performance. Last, we propose two methods to leverage unlabeled data in the model creation process. Often datasets are easier to obtain than to label, which results in many wasted opportunities with traditional classification approaches. Our approaches based on self-supervised learning result in a novel contrastive loss that is capable of extracting meaningful information out of pseudo-labeled data. Applying both methods to WCE data proves that the extracted inherent knowledge creates models that perform better in extremely unbalanced datasets and with lack of data. To summarize, this thesis demonstrates potential solutions to obtain uncertainty bounds, provide reasonable explanations of the outputs, and to combat lack of data or unbalanced datasets. Overall, the presented methods have a positive impact on the DL field and could have a real and tangible effect for the society.[cat] És innegable que el Deep Learning ha causat una revolució en molts aspectes no solament de l’aprenentatge automàtic però també de les nostres vides diàries. Tot i així, encara queden aspectes a millorar. Les xarxes neuronals tenen problemes per estimar la seva confiança en les prediccions, i sovint reporten probabilitats altes en casos que no tenen relació amb el model o que directament no tenen sentit. De la mateixa forma, interpretar els resultats d’un model profund i complex resulta una tasca extremadament complicada. Aquests mateixos models, cada cop amb més paràmetres i més potents, requereixen també de dades més ben etiquetades i més completes. Tenint en compte aquestes limitacions, l’objectiu principal és el de buscar mètodes i algoritmes per trobar-ne solució. Primerament, es proposa la creació d’un mètode capaç d’obtenir incertesa en imatges satèl·lit i d’utilitzar-la per crear models més robustos i resultats interpretables. En segon lloc, s’utilitzen Recurrent Neural Networks (RNN) per combatre la falta de dades mitjançant l’obtenció d’informació contextual de dades temporals. Aquestes s’apliquen per l’extracció de carreteres d’imatges satèl·lit i per la classificació de pòlips en imatges obtingudes amb Wireless Capsule Endoscopy (WCE). Finalment, es plantegen dos mètodes per tractar amb la falta de dades etiquetades i desbalancejos en les classes amb l’ús de Self-supervised Learning (SSL). Seqüències no etiquetades d’imatges d’intestins s’incorporen en el models en una fase prèvia a la classificació tradicional. Aquesta tesi demostra que les solucions proposades per obtenir mesures d’incertesa són efectives per donar explicacions raonables i interpretables sobre els resultats. Igualment, es prova que el context en dades de caràcter temporal, obtingut amb RNNs, serveix per obtenir models més simples que poden arribar a solucionar els problemes derivats de la falta de dades. Per últim, es mostra que SSL serveix per combatre de forma efectiva els problemes de generalització degut a dades no balancejades en diversos dominis de WCE. Concloem que aquesta tesi presenta mètodes amb un impacte real en diversos aspectes de DL a la vegada que demostra la capacitat de tenir un impacte positiu en la societat

Wireless capsule gastrointestinal endoscopy: direction of arrival estimation based localization survey

One of the significant challenges in Capsule Endoscopy (CE) is to precisely determine the pathologies location. The localization process is primarily estimated using the received signal strength from sensors in the capsule system through its movement in the gastrointestinal (GI) tract. Consequently, the wireless capsule endoscope (WCE) system requires improvement to handle the lack of the capsule instantaneous localization information and to solve the relatively low transmission data rate challenges. Furthermore, the association between the capsule’s transmitter position, capsule location, signal reduction and the capsule direction should be assessed. These measurements deliver significant information for the instantaneous capsule localization systems based on TOA (time of arrival) approach, PDOA (phase difference of arrival), RSS (received signal strength), electromagnetic, DOA (direction of arrival) and video tracking approaches are developed to locate the WCE precisely. The current article introduces the acquisition concept of the GI medical images using the endoscopy with a comprehensive description of the endoscopy system components. Capsule localization and tracking are considered to be the most important features of the WCE system, thus the current article emphasizes the most common localization systems generally, highlighting the DOA-based localization systems and discusses the required significant research challenges to be addressed

Exploring variability in medical imaging

Although recent successes of deep learning and novel machine learning techniques improved the perfor- mance of classification and (anomaly) detection in computer vision problems, the application of these methods in medical imaging pipeline remains a very challenging task. One of the main reasons for this is the amount of variability that is encountered and encapsulated in human anatomy and subsequently reflected in medical images. This fundamental factor impacts most stages in modern medical imaging processing pipelines. Variability of human anatomy makes it virtually impossible to build large datasets for each disease with labels and annotation for fully supervised machine learning. An efficient way to cope with this is to try and learn only from normal samples. Such data is much easier to collect. A case study of such an automatic anomaly detection system based on normative learning is presented in this work. We present a framework for detecting fetal cardiac anomalies during ultrasound screening using generative models, which are trained only utilising normal/healthy subjects. However, despite the significant improvement in automatic abnormality detection systems, clinical routine continues to rely exclusively on the contribution of overburdened medical experts to diagnosis and localise abnormalities. Integrating human expert knowledge into the medical imaging processing pipeline entails uncertainty which is mainly correlated with inter-observer variability. From the per- spective of building an automated medical imaging system, it is still an open issue, to what extent this kind of variability and the resulting uncertainty are introduced during the training of a model and how it affects the final performance of the task. Consequently, it is very important to explore the effect of inter-observer variability both, on the reliable estimation of model’s uncertainty, as well as on the model’s performance in a specific machine learning task. A thorough investigation of this issue is presented in this work by leveraging automated estimates for machine learning model uncertainty, inter-observer variability and segmentation task performance in lung CT scan images. Finally, a presentation of an overview of the existing anomaly detection methods in medical imaging was attempted. This state-of-the-art survey includes both conventional pattern recognition methods and deep learning based methods. It is one of the first literature surveys attempted in the specific research area.Open Acces

A hybrid localization method for Wireless Capsule Endoscopy (WCE)

Wireless capsule endoscopy (WCE) is a well-established diagnostic tool for visualizing the Gastrointestinal (GI) tract. WCE provides a unique view of the GI system with minimum discomfort for patients. Doctors can determine the type and severity of abnormality by analyzing the taken images. Early diagnosis helps them act and treat the disease in its earlier stages. However, the location information is missing in the frames. Pictures labeled by their location assist doctors in prescribing suitable medicines. The disease progress can be monitored, and the best treatment can be advised for the patients. Furthermore, at the time of surgery, it indicates the correct position for operation. Several attempts have been performed to localize the WCE accurately. Methods such as Radio frequency (RF), magnetic, image processing, ultrasound, and radiative imaging techniques have been investigated. Each one has its strengths and weaknesses. RF-based and magnetic-based localization methods need an external reference point, such as a belt or box around the patient, which limits their activities and causes discomfort. Computing the location solely based on an external reference could not distinguish between GI motion from capsule motion. Hence, this relative motion causes errors in position estimation. The GI system can move inside the body, while the capsule is stationary inside the intestine. This proposal presents two pose fusion methods, Method 1 and Method 2, that compensate for the relative motion of the GI tract with respect to the body. Method 1 is based on the data fusion from the Inertial measurement unit (IMU) sensor and side wall cameras. The IMU sensor consists of 9 Degree-Of-Freedom (DOF), including a gyroscope, an accelerometer, and a magnetometer to monitor the capsule’s orientation and its heading vector (the heading vector is a three-dimensional vector pointing to the direction of the capsule's head). Four monochromic cameras are placed at the side of the capsule to measure the displacement. The proposed method computes the heading vector using IMU data. Then, the heading vector is fused with displacements to estimate the 3D trajectory. This method has high accuracy in the short term. Meanwhile, due to the accumulation of errors from side wall cameras, the estimated trajectory tends to drift over time. Method 2 was developed to resolve the drifting issue while keeping the same positioning error. The capsule is equipped with four side wall cameras and a magnet. Magnetic localization acquires the capsule’s global position using 9 three-axis Hall effect sensors. However, magnetic localization alone cannot distinguish between the capsule’s and GI tract’s motions. To overcome this issue and increase tracking accuracy, side wall cameras are utilized, which are responsible for measuring the capsule’s movement, not the involuntary motion of the GI system. A complete setup is designed to test the capsule and perform the experiments. The results show that Method 2 has an average position error of only 3.5 mm and can compensate for the GI tract’s relative movements. Furthermore, environmental parameters such as magnetic interference and the nonhomogeneous structure of the GI tract have little influence on our system compared to the available magnetic localization methods. The experiment showed that Method 2 is suitable for localizing the WCE inside the body