Longitudinal detection of radiological abnormalities with time-modulated LSTM

Convolutional neural networks (CNNs) have been successfully employed in recent years for the detection of radiological abnormalities in medical images such as plain x-rays. To date, most studies use CNNs on individual examinations in isolation and discard previously available clinical information. In this study we set out to explore whether Long-Short-Term-Memory networks (LSTMs) can be used to improve classification performance when modelling the entire sequence of radiographs that may be available for a given patient, including their reports. A limitation of traditional LSTMs, though, is that they implicitly assume equally-spaced observations, whereas the radiological exams are event-based, and therefore irregularly sampled. Using both a simulated dataset and a large-scale chest x-ray dataset, we demonstrate that a simple modification of the LSTM architecture, which explicitly takes into account the time lag between consecutive observations, can boost classification performance. Our empirical results demonstrate improved detection of commonly reported abnormalities on chest x-rays such as cardiomegaly, consolidation, pleural effusion and hiatus hernia.Comment: Submitted to 4th MICCAI Workshop on Deep Learning in Medical Imaging Analysi

A hybrid CNN-RNN approach for survival analysis in a Lung Cancer Screening study

In this study, we present a hybrid CNN-RNN approach to investigate long-term survival of subjects in a lung cancer screening study. Subjects who died of cardiovascular and respiratory causes were identified whereby the CNN model was used to capture imaging features in the CT scans and the RNN model was used to investigate time series and thus global information. To account for heterogeneity in patients' follow-up times, two different variants of LSTM models were evaluated, each incorporating different strategies to address irregularities in follow-up time. The models were trained on subjects who underwent cardiovascular and respiratory deaths and a control cohort matched to participant age, gender, and smoking history. The combined model can achieve an AUC of 0.76 which outperforms humans at cardiovascular mortality prediction. The corresponding F1 and Matthews Correlation Coefficient are 0.63 and 0.42 respectively. The generalisability of the model is further validated on an 'external' cohort. The same models were applied to survival analysis with the Cox Proportional Hazard model. It was demonstrated that incorporating the follow-up history can lead to improvement in survival prediction. The Cox neural network can achieve an IPCW C-index of 0.75 on the internal dataset and 0.69 on an external dataset. Delineating subjects at increased risk of cardiorespiratory mortality can alert clinicians to request further more detailed functional or imaging studies to improve the assessment of cardiorespiratory disease burden. Such strategies may uncover unsuspected and under-recognised pathologies thereby potentially reducing patient morbidity

Development and validation of a deep learning algorithm for longitudinal change detection in sequential chest X-ray images

최근 그래픽 처리 장치 및 빅데이터가 발전하면서, 의료 영상처리 분야에 인공지능 알고리즘을 접목시켜 주요 질병을 진단 및 검출하는 연구가 활발히 진행되고 있다. 그러나 현재까지 제안된 인공지능 기반의 분류 모델들은 주어진 단일 영상만을 독립적으로 이용하여 결과를 도출한다. 즉, 현재 촬영된 영상은 이전 기록과 잠재적으로 관련이 있음에도 불구하고, 사전에 정의된 비정상 범주만을 예측하는 횡단면적 분석을 시행하는 것이다. 분류 성능은 영상의학 임상의의 수준에 근접하였지만, 병변의 구체적인 변화에 대응하지 못한다. 이는 환자가 이전에 촬영한 영상을 분류하더라도 단순 질병의 출현 유무로는 병변의 변화를 파악하기가 어렵기 때문이다. 특히 일부 주요 질병의 경우, 동일한 질병 내에서도 그 패턴의 종류가 다양할 뿐 만 아니라, 장기간 또는 급성 변화 등 변화 양상이 환자의 임상기록에 따라 매우 다르다. 따라서 횡단면적 분석만으로는 시간에 따른 특정 변화를 검출하는 것은 불가능하기 때문에 종단면적 분석이 함께 요구된다. 본 연구는 주어진 두 영상(전,후) 간의 병변의 특정 변화를 감지하는 새로운 인공지능 알고리즘을 제안한다. 본 알고리즘의 핵심 기법은 정합되지 않은 두 영상의 기하 상관관계도를 구하여 영상 간 변화 유무에 따른 기하 상관관계도 변화 패턴을 파악하고, 변화유무를 이진 분류하는 것이다. 현재까지 종단면적 분석을 위한 기계학습용 참조표준 데이터베이스가 공개된 것이 없기 때문에, 본 연구에서는 영상 판독문을 분석하여 병변의 변화기준을 확립하고, 질환의 종류, 경과시간, 변화 형태 등에 따른 데이터 분류 체계를 구축하여 순차적 흉부 X-선 영상에 대한 참조표준 데이터베이스를 자체적으로 확보하였다. 본 연구는 알고리즘 성능을 객관적으로 분석하기 위하여 수신자조작특성(ROC)의 하의 면적(AUC)을 산출하고, 기존 개발된 알고리즘 및 유사 연구와 정량적으로 비교하였다. 본 연구에서 제안하는 기하 상관관계도를 이용한 알고리즘이 AUC=0.89 (95% 신뢰구간, 0.86-0.92) 및 Youden's index에서의 민감도=0.83, 특이도=0.82으로 가장 뛰어난 성능을 보였다. 또한 주어진 두 영상에서 특정 병변의 변화에 따른 기하 상관관계도를 정성적으로 분석함으로써 실제로 해당 변화가 발생한 위치를 역추적 및 설명할 수 있는 가능성을 제시하였다.The diagnostic decision for chest X-ray image generally considers a probable change in a lesion, compared to the previous examination. We propose a novel algorithm to detect the change in longitudinal chest X-ray images. We extract feature maps from a pair of input images through two streams of convolutional neural networks. Next, we generate the geometric correlation map computing matching scores for every possible match of local descriptors in two feature maps. This correlation map is fed into a binary classifier to detect specific patterns of the map representing the change in the lesion. Since no public dataset offers proper information to train the proposed network, we also build our own dataset by analyzing reports in examinations at a tertiary hospital. Experimental results show our approach outperforms previous methods in quantitative comparison. We also provide various case examples visualizing the effect of the proposed geometric correlation map.1. 서론 7 1.1. 배경 7 1.2. 연구의 목적 9 2. 본론 12 2.1. 알고리즘 구조 12 2.2.1. 특징 추출 14 2.2.2. 기하 상관관계도 14 2.2.3. 이진 분류기 16 2.2. 참조표준 데이터베이스 17 3. 결과 및 분석 21 3.1. 판독문 가공 21 3.2. 알고리즘 성능 22 4. 고찰 28 4.1. 실험 결과 고찰 28 4.2. 알고리즘 고찰 28 5. 결론 30 참고 문헌 31 Abstract 33석

Deep learning in medical imaging and radiation therapy

Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/146980/1/mp13264_am.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/146980/2/mp13264.pd

Reflections on the nature of measurement in language-based automated assessments of patients' mental state and cognitive function

Modern advances in computational language processing methods have enabled new approaches to the measurement of mental processes. However, the field has primarily focused on model accuracy in predicting performance on a task or a diagnostic category. Instead the field should be more focused on determining which computational analyses align best with the targeted neurocognitive/psychological functions that we want to assess. In this paper we reflect on two decades of experience with the application of language-based assessment to patients' mental state and cognitive function by addressing the questions of what we are measuring, how it should be measured and why we are measuring the phenomena. We address the questions by advocating for a principled framework for aligning computational models to the constructs being assessed and the tasks being used, as well as defining how those constructs relate to patient clinical states. We further examine the assumptions that go into the computational models and the effects that model design decisions may have on the accuracy, bias and generalizability of models for assessing clinical states. Finally, we describe how this principled approach can further the goal of transitioning language-based computational assessments to part of clinical practice while gaining the trust of critical stakeholders

Reflections on the nature of measurement in language-based automated assessments of patients' mental state and cognitive function

Modern advances in computational language processing methods have enabled new approaches to the measurement of mental processes. However, the field has primarily focused on model accuracy in predicting performance on a task or a diagnostic category. Instead the field should be more focused on determining which computational analyses align best with the targeted neurocognitive/psychological functions that we want to assess. In this paper we reflect on two decades of experience with the application of language-based assessment to patients' mental state and cognitive function by addressing the questions of what we are measuring, how it should be measured and why we are measuring the phenomena. We address the questions by advocating for a principled framework for aligning computational models to the constructs being assessed and the tasks being used, as well as defining how those constructs relate to patient clinical states. We further examine the assumptions that go into the computational models and the effects that model design decisions may have on the accuracy, bias and generalizability of models for assessing clinical states. Finally, we describe how this principled approach can further the goal of transitioning language-based computational assessments to part of clinical practice while gaining the trust of critical stakeholders

Machine Learning for Biomedical Application

Biomedicine is a multidisciplinary branch of medical science that consists of many scientific disciplines, e.g., biology, biotechnology, bioinformatics, and genetics; moreover, it covers various medical specialties. In recent years, this field of science has developed rapidly. This means that a large amount of data has been generated, due to (among other reasons) the processing, analysis, and recognition of a wide range of biomedical signals and images obtained through increasingly advanced medical imaging devices. The analysis of these data requires the use of advanced IT methods, which include those related to the use of artificial intelligence, and in particular machine learning. It is a summary of the Special Issue “Machine Learning for Biomedical Application”, briefly outlining selected applications of machine learning in the processing, analysis, and recognition of biomedical data, mostly regarding biosignals and medical images

Improved 3D MR Image Acquisition and Processing in Congenital Heart Disease

Congenital heart disease (CHD) is the most common type of birth defect, affecting about 1% of the population. MRI is an essential tool in the assessment of CHD, including diagnosis, intervention planning and follow-up. Three-dimensional MRI can provide particularly rich visualization and information. However, it is often complicated by long scan times, cardiorespiratory motion, injection of contrast agents, and complex and time-consuming postprocessing. This thesis comprises four pieces of work that attempt to respond to some of these challenges. The first piece of work aims to enable fast acquisition of 3D time-resolved cardiac imaging during free breathing. Rapid imaging was achieved using an efficient spiral sequence and a sparse parallel imaging reconstruction. The feasibility of this approach was demonstrated on a population of 10 patients with CHD, and areas of improvement were identified. The second piece of work is an integrated software tool designed to simplify and accelerate the development of machine learning (ML) applications in MRI research. It also exploits the strengths of recently developed ML libraries for efficient MR image reconstruction and processing. The third piece of work aims to reduce contrast dose in contrast-enhanced MR angiography (MRA). This would reduce risks and costs associated with contrast agents. A deep learning-based contrast enhancement technique was developed and shown to improve image quality in real low-dose MRA in a population of 40 children and adults with CHD. The fourth and final piece of work aims to simplify the creation of computational models for hemodynamic assessment of the great arteries. A deep learning technique for 3D segmentation of the aorta and the pulmonary arteries was developed and shown to enable accurate calculation of clinically relevant biomarkers in a population of 10 patients with CHD