Object Detection in medical imaging

A thesis submitted in partial fulfillment of the requirements for the degree of Doctor in Information Management, specialization in Information and Decision SystemsArtificial Intelligence, assisted by deep learning, has emerged in various fields of our society. These systems allow the automation and the improvement of several tasks, even surpassing, in some cases, human capability. Object detection methods are used nowadays in several areas, including medical imaging analysis. However, these methods are susceptible to errors, and there is a lack of a universally accepted method that can be applied across all types of applications with the needed precision in the medical field. Additionally, the application of object detectors in medical imaging analysis has yet to be thoroughly analyzed to achieve a richer understanding of the state of the art. To tackle these shortcomings, we present three studies with distinct goals. First, a quantitative and qualitative analysis of academic research was conducted to gather a perception of which object detectors are employed, the modality of medical imaging used, and the particular body parts under investigation. Secondly, we propose an optimized version of a widely used algorithm to overcome limitations commonly addressed in medical imaging by fine-tuning several hyperparameters. Thirdly, we develop a novel stacking approach to augment the precision of detections on medical imaging analysis. The findings show that despite the late arrival of object detection in medical imaging analysis, the number of publications has increased in recent years, demonstrating the significant potential for growth. Additionally, we establish that it is possible to address some constraints on the data through an exhaustive optimization of the algorithm. Finally, our last study highlights that there is still room for improvement in these advanced techniques, using, as an example, stacking approaches. The contributions of this dissertation are several, as it puts forward a deeper overview of the state-of-the-art applications of object detection algorithms in the medical field and presents strategies for addressing typical constraints in this area.A Inteligência Artificial, auxiliada pelo deep learning, tem emergido em diversas áreas da nossa sociedade. Estes sistemas permitem a automatização e a melhoria de diversas tarefas, superando mesmo, em alguns casos, a capacidade humana. Os métodos de detecção de objetos são utilizados atualmente em diversas áreas, inclusive na análise de imagens médicas. No entanto, esses métodos são suscetíveis a erros e falta um método universalmente aceite que possa ser aplicado em todos os tipos de aplicações com a precisão necessária na área médica. Além disso, a aplicação de detectores de objetos na análise de imagens médicas ainda precisa ser analisada minuciosamente para alcançar uma compreensão mais rica do estado da arte. Para enfrentar essas limitações, apresentamos três estudos com objetivos distintos. Inicialmente, uma análise quantitativa e qualitativa da pesquisa acadêmica foi realizada para obter uma percepção de quais detectores de objetos são empregues, a modalidade de imagem médica usada e as partes específicas do corpo sob investigação. Num segundo estudo, propomos uma versão otimizada de um algoritmo amplamente utilizado para superar limitações comumente abordadas em imagens médicas por meio do ajuste fino de vários hiperparâmetros. Em terceiro lugar, desenvolvemos uma nova abordagem de stacking para aumentar a precisão das detecções na análise de imagens médicas. Os resultados demostram que, apesar da chegada tardia da detecção de objetos na análise de imagens médicas, o número de publicações aumentou nos últimos anos, evidenciando o significativo potencial de crescimento. Adicionalmente, estabelecemos que é possível resolver algumas restrições nos dados por meio de uma otimização exaustiva do algoritmo. Finalmente, o nosso último estudo destaca que ainda há espaço para melhorias nessas técnicas avançadas, usando, como exemplo, abordagens de stacking. As contribuições desta dissertação são várias, apresentando uma visão geral em maior detalhe das aplicações de ponta dos algoritmos de detecção de objetos na área médica e apresenta estratégias para lidar com restrições típicas nesta área

Machine learning based small bowel video capsule endoscopy analysis: Challenges and opportunities

YesVideo capsule endoscopy (VCE) is a revolutionary technology for the early diagnosis of gastric disorders. However, owing to the high redundancy and subtle manifestation of anomalies among thousands of frames, the manual construal of VCE videos requires considerable patience, focus, and time. The automatic analysis of these videos using computational methods is a challenge as the capsule is untamed in motion and captures frames inaptly. Several machine learning (ML) methods, including recent deep convolutional neural networks approaches, have been adopted after evaluating their potential of improving the VCE analysis. However, the clinical impact of these methods is yet to be investigated. This survey aimed to highlight the gaps between existing ML-based research methodologies and clinically significant rules recently established by gastroenterologists based on VCE. A framework for interpreting raw frames into contextually relevant frame-level findings and subsequently merging these findings with meta-data to obtain a disease-level diagnosis was formulated. Frame-level findings can be more intelligible for discriminative learning when organized in a taxonomical hierarchy. The proposed taxonomical hierarchy, which is formulated based on pathological and visual similarities, may yield better classification metrics by setting inference classes at a higher level than training classes. Mapping from the frame level to the disease level was structured in the form of a graph based on clinical relevance inspired by the recent international consensus developed by domain experts. Furthermore, existing methods for VCE summarization, classification, segmentation, detection, and localization were critically evaluated and compared based on aspects deemed significant by clinicians. Numerous studies pertain to single anomaly detection instead of a pragmatic approach in a clinical setting. The challenges and opportunities associated with VCE analysis were delineated. A focus on maximizing the discriminative power of features corresponding to various subtle lesions and anomalies may help cope with the diverse and mimicking nature of different VCE frames. Large multicenter datasets must be created to cope with data sparsity, bias, and class imbalance. Explainability, reliability, traceability, and transparency are important for an ML-based diagnostics system in a VCE. Existing ethical and legal bindings narrow the scope of possibilities where ML can potentially be leveraged in healthcare. Despite these limitations, ML based video capsule endoscopy will revolutionize clinical practice, aiding clinicians in rapid and accurate diagnosis

Advanced Endoscopic Navigation:Surgical Big Data,Methodology,and Applications

随着科学技术的飞速发展，健康与环境问题日益成为人类面临的最重大问题之一。信息科学、计算机技术、电子工程与生物医学工程等学科的综合应用交叉前沿课题，研究现代工程技术方法，探索肿瘤癌症等疾病早期诊断、治疗和康复手段。本论文综述了计算机辅助微创外科手术导航、多模态医疗大数据、方法论及其临床应用：从引入微创外科手术导航概念出发，介绍了医疗大数据的术前与术中多模态医学成像方法、阐述了先进微创外科手术导航的核心流程包括计算解剖模型、术中实时导航方案、三维可视化方法及交互式软件技术，归纳了各类微创外科手术方法的临床应用。同时，重点讨论了全球各种手术导航技术在临床应用中的优缺点，分析了目前手术导航领域内的最新技术方法。在此基础上，提出了微创外科手术方法正向数字化、个性化、精准化、诊疗一体化、机器人化以及高度智能化的发展趋势。【Abstract】Interventional endoscopy (e.g., bronchoscopy, colonoscopy, laparoscopy, cystoscopy) is a widely performed procedure that involves either diagnosis of suspicious lesions or guidance for minimally invasive surgery in a variety of organs within the body cavity. Endoscopy may also be used to guide the introduction of certain items (e.g., stents) into the body. Endoscopic navigation systems seek to integrate big data with multimodal information (e.g., computed tomography, magnetic resonance images, endoscopic video sequences, ultrasound images, external trackers) relative to the patient's anatomy, control the movement of medical endoscopes and surgical tools, and guide the surgeon's actions during endoscopic interventions. Nevertheless, it remains challenging to realize the next generation of context-aware navigated endoscopy. This review presents a broad survey of various aspects of endoscopic navigation, particularly with respect to the development of endoscopic navigation techniques. First, we investigate big data with multimodal information involved in endoscopic navigation. Next, we focus on numerous methodologies used for endoscopic navigation. We then review different endoscopic procedures in clinical applications. Finally, we discuss novel techniques and promising directions for the development of endoscopic navigation.X.L. acknowledges funding from the Fundamental Research Funds for the Central Universities. T.M.P. acknowledges funding from the Canadian Foundation for Innovation, the Canadian Institutes for Health Research, the National Sciences and Engineering Research Council of Canada, and a grant from Intuitive Surgical Inc

Deep Learning in Cardiology

The medical field is creating large amount of data that physicians are unable to decipher and use efficiently. Moreover, rule-based expert systems are inefficient in solving complicated medical tasks or for creating insights using big data. Deep learning has emerged as a more accurate and effective technology in a wide range of medical problems such as diagnosis, prediction and intervention. Deep learning is a representation learning method that consists of layers that transform the data non-linearly, thus, revealing hierarchical relationships and structures. In this review we survey deep learning application papers that use structured data, signal and imaging modalities from cardiology. We discuss the advantages and limitations of applying deep learning in cardiology that also apply in medicine in general, while proposing certain directions as the most viable for clinical use.Comment: 27 pages, 2 figures, 10 table

Explainable artificial intelligence (XAI) in deep learning-based medical image analysis

With an increase in deep learning-based methods, the call for explainability of such methods grows, especially in high-stakes decision making areas such as medical image analysis. This survey presents an overview of eXplainable Artificial Intelligence (XAI) used in deep learning-based medical image analysis. A framework of XAI criteria is introduced to classify deep learning-based medical image analysis methods. Papers on XAI techniques in medical image analysis are then surveyed and categorized according to the framework and according to anatomical location. The paper concludes with an outlook of future opportunities for XAI in medical image analysis.Comment: Submitted for publication. Comments welcome by email to first autho

Building up the Future of Colonoscopy – A Synergy between Clinicians and Computer Scientists

Recent advances in endoscopic technology have generated an increasing interest in strengthening the collaboration between clinicians and computers scientist to develop intelligent systems that can provide additional information to clinicians in the different stages of an intervention. The objective of this chapter is to identify clinical drawbacks of colonoscopy in order to define potential areas of collaboration. Once areas are defined, we present the challenges that colonoscopy images present in order computational methods to provide with meaningful output, including those related to image formation and acquisition, as they are proven to have an impact in the performance of an intelligent system. Finally, we also propose how to define validation frameworks in order to assess the performance of a given method, making an special emphasis on how databases should be created and annotated and which metrics should be used to evaluate systems correctly

Triage-driven diagnosis for early detection of oesophageal cancer

In this thesis I present my work to advance the early detection of oesophageal adenocarcinoma by investigating translational aspects of a minimally invasive oesophageal cell sampling technology for the detection of Barrett oesophagus. Most oesophageal adenocarcinoma patients present with advanced disease, requiring treatment with chemotherapy with or without radiotherapy, followed by surgery to remove the oesophagus, and even then the overall five-year survival is less than 20%. However, if the cancer can be diagnosed at an early, superficial stage then treatment can be performed endoscopically and over 80% of patients survive beyond 5 years. The disease has a clear pre-cancer stage called Barrett oesophagus, making early detection feasible. A novel test called Cytosponge for diagnosing Barrett by cell collection coupled with an immunohistochemical test (Trefoil factor 3 / TFF3) has been developed. I have investigated two distinct topics, which are key to implement the Cytosponge-TFF3 test in primary and secondary care. First, I analysed and interpreted data of a pragmatic, prospective, multicentre, randomised controlled trial (BEST3) in order to evaluate the use of Cytosponge in primary care. The study aim was to investigate whether offering this test to patients on medication for gastro-oesophageal reflux disease (GERD) would increase the detection of Barrett oesophagus compared with usual care. We were able to show that in patients with GERD the offer of Cytosponge-TFF3 testing results in improved detection (in excess of 10-fold) of Barrett oesophagus. Second, I devised and implemented a machine learning framework applied to Cytosponge samples with the objective to reduce the pathologists' screening time. I trained and independently validated the framework on data from two clinical trials, analysing a combined total of 4,662 pathology slides from 2,331 patients. The approach exploits screening patterns of expert gastrointestinal pathologists and established decision pathways to define eight triage classes of varying priority for manual expert review. By substitution of manual review with automated review in low-priority classes, I can reduce pathologist workload by 57% while matching the diagnostic performance of expert pathologists