Medical Diagnosis with Multimodal Image Fusion Techniques

Image Fusion is an effective approach utilized to draw out all the significant information from the source images, which supports experts in evaluation and quick decision making. Multi modal medical image fusion produces a composite fused image utilizing various sources to improve quality and extract complementary information. It is extremely challenging to gather every piece of information needed using just one imaging method. Therefore, images obtained from different modalities are fused Additional clinical information can be gleaned through the fusion of several types of medical image pairings. This study's main aim is to present a thorough review of medical image fusion techniques which also covers steps in fusion process, levels of fusion, various imaging modalities with their pros and cons, and  the major scientific difficulties encountered in the area of medical image fusion. This paper also summarizes the quality assessments fusion metrics. The various approaches used by image fusion algorithms that are presently available in the literature are classified into four broad categories i) Spatial fusion methods ii) Multiscale Decomposition based methods iii) Neural Network based methods and iv) Fuzzy Logic based methods. the benefits and pitfalls of the existing literature are explored and Future insights are suggested. Moreover, this study is anticipated to create a solid platform for the development of better fusion techniques in medical applications

Use of Coherent Point Drift in computer vision applications

This thesis presents the novel use of Coherent Point Drift in improving the robustness of a number of computer vision applications. CPD approach includes two methods for registering two images - rigid and non-rigid point set approaches which are based on the transformation model used. The key characteristic of a rigid transformation is that the distance between points is preserved, which means it can be used in the presence of translation, rotation, and scaling. Non-rigid transformations - or affine transforms - provide the opportunity of registering under non-uniform scaling and skew. The idea is to move one point set coherently to align with the second point set. The CPD method finds both the non-rigid transformation and the correspondence distance between two point sets at the same time without having to use a-priori declaration of the transformation model used. The first part of this thesis is focused on speaker identification in video conferencing. A real-time, audio-coupled video based approach is presented, which focuses more on the video analysis side, rather than the audio analysis that is known to be prone to errors. CPD is effectively utilised for lip movement detection and a temporal face detection approach is used to minimise false positives if face detection algorithm fails to perform. The second part of the thesis is focused on multi-exposure and multi-focus image fusion with compensation for camera shake. Scale Invariant Feature Transforms (SIFT) are first used to detect keypoints in images being fused. Subsequently this point set is reduced to remove outliers, using RANSAC (RANdom Sample Consensus) and finally the point sets are registered using CPD with non-rigid transformations. The registered images are then fused with a Contourlet based image fusion algorithm that makes use of a novel alpha blending and filtering technique to minimise artefacts. The thesis evaluates the performance of the algorithm in comparison to a number of state-of-the-art approaches, including the key commercial products available in the market at present, showing significantly improved subjective quality in the fused images. The final part of the thesis presents a novel approach to Vehicle Make & Model Recognition in CCTV video footage. CPD is used to effectively remove skew of vehicles detected as CCTV cameras are not specifically configured for the VMMR task and may capture vehicles at different approaching angles. A LESH (Local Energy Shape Histogram) feature based approach is used for vehicle make and model recognition with the novelty that temporal processing is used to improve reliability. A number of further algorithms are used to maximise the reliability of the final outcome. Experimental results are provided to prove that the proposed system demonstrates an accuracy in excess of 95% when tested on real CCTV footage with no prior camera calibration

Fusi Citra Multi-Fokus Menggunakan Stationary Wavelet Transform dan Himpunan Fuzzy

Masalah utama pada fusi citra multi-fokus ialah bagaimana caranya untuk mengekstraksi fitur dari citra sumber dan menggabungkan koefisien tersebut secara akurat sehingga menghasilkan piksel citra yang berkualitas tinggi. Akan tetapi, yang disebut dengan berkualitas tinggi merupakan definisi yang tidak pasti, oleh karena itu teori fuzzy sangat sesuai digunakan untuk menyelesaikan permasalahan tersebut. Tugas akhir ini mengusulkan skema fusi citra multi-fokus yang dapat menggabungkan fitur berkualitas tinggi dari dua citra sumber yang berbeda menjadi satu citra gabungan menggunakan integrasi dari Stationary Wavelet Transform dan Himpunan Fuzzy. Pertama, sumber citra didekomposisi menggunakan Stationary Wavelet Transform (SWT) untuk mendapatkan kumpulan sub-citra dengan fitur rinci yang berbeda. Kedua, Gaussian Membership Function (GMF) dimanfaatkan untuk mendapatkan himpunan fuzzy dari data sub-citra. Ketiga, Local Spatial Frequency (LSF) diaplikasikan untuk mendapatkan fitur lokal sub-citra dari himpunan fuzzy. Akhirnya, aturan fusi dirancang berdasarkan hasil verifikasi konsistensi untuk menggabungkan sub-citra, lalu Inverse Stationary Wavelet Transform (ISWT) diimplementasikan untuk merekonstruksi citra gabungan. Uji coba dilakukan pada 20 pasang citra RGB dan 10 pasang citra grayscale. Berdasarkan hasil uji coba, metode ini dapat menghasilkan citra gabungan yang akurat dengan rata-rata Root Mean Square Error (RMSE) dan Mutual Information (MI) pada citra RGB yaitu 0,1091 dan 9,2625 dan pada citra grayscale yaitu 0,0996 dan 8,4949. ================================================================================= The key issue of multi-focus image fusion is how to accurately extract features from source images and fuse those coefficients to create high-quality image. Nevertheless, the so-called high-quality is an uncertain definition, which is very suitable for fuzzy theory to address this problem. This research proposes multi-focus image fusion scheme which can merge the high-quality coefficients of two different source images into a fused image by the integration of Stationary Wavelet Transform (SWT) and Fuzzy Sets. First, the source images are decomposed by Stationary Wavelet Transform (SWT) to get a set of sub-images with different detailed features. Second, the Gaussian Membership Function (GMF) is utilized to get the fuzzy sets of sub-images data. Third, the Local Spatial Frequency (LSF) is employed to extract the local features of the fuzzy sets. At last, the fusion rule is designed based on consistency verification to fuse the sub-images, and then Inverse Stationary Wavelet Transform (ISWT) is implemented to reconstruct the fused image. The experimental is done to 20 pairs of RGB image and 10 pairs of grayscale image. Based on the experiments, this method generate an accurate fused image with average of Root Mean Square Error (RMSE) and Mutual Information (MI) for RGB images are 0,1091 and 9,2625 and for grayscale images are 0,0996 and 8,4949

HETEROGENEOUS MULTI-SENSOR FUSION FOR 2D AND 3D POSE ESTIMATION

Sensor fusion is a process in which data from different sensors is combined to acquire an output that cannot be obtained from individual sensors. This dissertation first considers a 2D image level real world problem from rail industry and proposes a novel solution using sensor fusion, then proceeds further to the more complicated 3D problem of multi sensor fusion for UAV pose estimation. One of the most important safety-related tasks in the rail industry is an early detection of defective rolling stock components. Railway wheels and wheel bearings are two components prone to damage due to their interactions with the brakes and railway track, which makes them a high priority when rail industry investigates improvements to current detection processes. The main contribution of this dissertation in this area is development of a computer vision method for automatically detecting the defective wheels that can potentially become a replacement for the current manual inspection procedure. The algorithm fuses images taken by wayside thermal and vision cameras and uses the outcome for the wheel defect detection. As a byproduct, the process will also include a method for detecting hot bearings from the same images. We evaluate our algorithm using simulated and real data images from UPRR in North America and it will be shown in this dissertation that using sensor fusion techniques the accuracy of the malfunction detection can be improved. After the 2D application, the more complicated 3D application is addressed. Precise, robust and consistent localization is an important subject in many areas of science such as vision-based control, path planning, and SLAM. Each of different sensors employed to estimate the pose have their strengths and weaknesses. Sensor fusion is a known approach that combines the data measured by different sensors to achieve a more accurate or complete pose estimation and to cope with sensor outages. In this dissertation, a new approach to 3D pose estimation for a UAV in an unknown GPS-denied environment is presented. The proposed algorithm fuses the data from an IMU, a camera, and a 2D LiDAR to achieve accurate localization. Among the employed sensors, LiDAR has not received proper attention in the past; mostly because a 2D LiDAR can only provide pose estimation in its scanning plane and thus it cannot obtain full pose estimation in a 3D environment. A novel method is introduced in this research that enables us to employ a 2D LiDAR to improve the full 3D pose estimation accuracy acquired from an IMU and a camera. To the best of our knowledge 2D LiDAR has never been employed for 3D localization without a prior map and it is shown in this dissertation that our method can significantly improve the precision of the localization algorithm. The proposed approach is evaluated and justified by simulation and real world experiments

Machine learning and deep learning based methods toward Industry 4.0 predictive maintenance in induction motors: Α state of the art survey

Purpose: Developments in Industry 4.0 technologies and Artificial Intelligence (AI) have enabled data-driven manufacturing. Predictive maintenance (PdM) has therefore become the prominent approach for fault detection and diagnosis (FD/D) of induction motors (IMs). The maintenance and early FD/D of IMs are critical processes, considering that they constitute the main power source in the industrial production environment. Machine learning (ML) methods have enhanced the performance and reliability of PdM. Various deep learning (DL) based FD/D methods have emerged in recent years, providing automatic feature engineering and learning and thereby alleviating drawbacks of traditional ML based methods. This paper presents a comprehensive survey of ML and DL based FD/D methods of IMs that have emerged since 2015. An overview of the main DL architectures used for this purpose is also presented. A discussion of the recent trends is given as well as future directions for research. Design/methodology/approach: A comprehensive survey has been carried out through all available publication databases using related keywords. Classification of the reviewed works has been done according to the main ML and DL techniques and algorithms Findings: DL based PdM methods have been mainly introduced and implemented for IM fault diagnosis in recent years. Novel DL FD/D methods are based on single DL techniques as well as hybrid techniques. DL methods have also been used for signal preprocessing and moreover, have been combined with traditional ML algorithms to enhance the FD/D performance in feature engineering. Publicly available datasets have been mostly used to test the performance of the developed methods, however industrial datasets should become available as well. Multi-agent system (MAS) based PdM employing ML classifiers has been explored. Several methods have investigated multiple IM faults, however, the presence of multiple faults occurring simultaneously has rarely been investigated. Originality/value: The paper presents a comprehensive review of the recent advances in PdM of IMs based on ML and DL methods that have emerged since 2015Peer Reviewe

A Review on the use of Artificial Intelligence Techniques in Brain MRI Analysis

Over the past 20 years, the global research going on in Artificial Intelligence in applica-tions in medication is a venue internationally, for medical trade and creating an ener-getic research community. The Artificial Intelligence in Medicine magazine has posted a massive amount. This paper gives an overview of the history of AI applications in brain MRI analysis to research its effect at the wider studies discipline and perceive de-manding situations for its destiny. Analysis of numerous articles to create a taxono-my of research subject matters and results was done. The article is classed which might be posted between 2000 and 2018 with this taxonomy. Analyzed articles have excessive citations. Efforts are useful in figuring out popular studies works in AI primarily based on mind MRI analysis throughout specific issues. The biomedical prognosis was ruled by way of knowledge engineering research in its first decade, whilst gadget mastering, and records mining prevailed thereafter. Together these two topics have contributed a lot to the latest medical domain

Deep fusion of multi-channel neurophysiological signal for emotion recognition and monitoring

How to fuse multi-channel neurophysiological signals for emotion recognition is emerging as a hot research topic in community of Computational Psychophysiology. Nevertheless, prior feature engineering based approaches require extracting various domain knowledge related features at a high time cost. Moreover, traditional fusion method cannot fully utilise correlation information between different channels and frequency components. In this paper, we design a hybrid deep learning model, in which the 'Convolutional Neural Network (CNN)' is utilised for extracting task-related features, as well as mining inter-channel and inter-frequency correlation, besides, the 'Recurrent Neural Network (RNN)' is concatenated for integrating contextual information from the frame cube sequence. Experiments are carried out in a trial-level emotion recognition task, on the DEAP benchmarking dataset. Experimental results demonstrate that the proposed framework outperforms the classical methods, with regard to both of the emotional dimensions of Valence and Arousal

Epileptic seizure detection and prediction based on EEG signal

Epilepsy is a kind of chronic brain disfunction, manifesting as recurrent seizures which is caused by sudden and excessive discharge of neurons. Electroencephalogram (EEG) recordings is regarded as the golden standard for clinical diagnosis of epilepsy disease. The diagnosis of epilepsy disease by professional doctors clinically is time-consuming. With the help artificial intelligence algorithms, the task of automatic epileptic seizure detection and prediction is called a research hotspot. The thesis mainly contributes to propose a solution to overfitting problem of EEG signal in deep learning and a method of multiple channels fusion for EEG features. The result of proposed method achieves outstanding performance in seizure detection task and seizure prediction task. In seizure detection task, this paper mainly explores the effect of the deep learning in small data size. This thesis designs a hybrid model of CNN and SVM for epilepsy detection compared with end-to-end classification by deep learning. Another technique for overfitting is new EEG signal generation based on decomposition and recombination of EEG in time-frequency domain. It achieved a classification accuracy of 98.8%, a specificity of 98.9% and a sensitivity of 98.4% on the classic Bonn EEG data. In seizure prediction task, this paper proposes a feature fusion method for multi-channel EEG signals. We extract a three-order tensor feature in temporal, spectral and spatial domain. UMLDA is a tensor-to-vector projection method, which ensures minimal redundancy between feature dimensions. An excellent experimental result was finally obtained, including an average accuracy of 95%, 94% F1-measure and 90% Kappa index