A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

Phase Aberration Correction for in vivo Ultrasound Localization Microscopy Using a Spatiotemporal Complex-Valued Neural Network

Ultrasound Localization Microscopy (ULM) can map microvessels at a resolution of a few micrometers ({\mu}m). Transcranial ULM remains challenging in presence of aberrations caused by the skull, which lead to localization errors. Herein, we propose a deep learning approach based on recently introduced complex-valued convolutional neural networks (CV-CNNs) to retrieve the aberration function, which can then be used to form enhanced images using standard delay-and-sum beamforming. Complex-valued convolutional networks were selected as they can apply time delays through multiplication with in-phase quadrature input data. Predicting the aberration function rather than corrected images also confers enhanced explainability to the network. In addition, 3D spatiotemporal convolutions were used for the network to leverage entire microbubble tracks. For training and validation, we used an anatomically and hemodynamically realistic mouse brain microvascular network model to simulate the flow of microbubbles in presence of aberration. We then confirmed the capability of our network to generalize to transcranial in vivo data in the mouse brain (n=2). Qualitatively, vascular reconstructions using a pixel-wise predicted aberration function included additional and sharper vessels. The spatial resolution was evaluated by using the Fourier ring correlation (FRC). After correction, we measured a resolution of 16.7 {\mu}m in vivo, representing an improvement of up to 27.5 %. This work leads to different applications for complex-valued convolutions in biomedical imaging and strategies to perform transcranial ULM

Ultrasound Signal Processing: From Models to Deep Learning

Medical ultrasound imaging relies heavily on high-quality signal processing algorithms to provide reliable and interpretable image reconstructions. Hand-crafted reconstruction methods, often based on approximations of the underlying measurement model, are useful in practice, but notoriously fall behind in terms of image quality. More sophisticated solutions, based on statistical modelling, careful parameter tuning, or through increased model complexity, can be sensitive to different environments. Recently, deep learning based methods have gained popularity, which are optimized in a data-driven fashion. These model-agnostic methods often rely on generic model structures, and require vast training data to converge to a robust solution. A relatively new paradigm combines the power of the two: leveraging data-driven deep learning, as well as exploiting domain knowledge. These model-based solutions yield high robustness, and require less trainable parameters and training data than conventional neural networks. In this work we provide an overview of these methods from the recent literature, and discuss a wide variety of ultrasound applications. We aim to inspire the reader to further research in this area, and to address the opportunities within the field of ultrasound signal processing. We conclude with a future perspective on these model-based deep learning techniques for medical ultrasound applications

Fast and selective super-resolution ultrasound in vivo with acoustically activated nanodroplets

Perfusion by the microcirculation is key to the development, maintenance and pathology of tissue. Its measurement with high spatiotemporal resolution is consequently valuable but remains a challenge in deep tissue. Ultrasound Localization Microscopy (ULM) provides very high spatiotemporal resolution but the use of microbubbles requires low contrast agent concentrations, a long acquisition time, and gives little control over the spatial and temporal distribution of the microbubbles. The present study is the first to demonstrate Acoustic Wave Sparsely-Activated Localization Microscopy (AWSALM) and fast-AWSALM for in vivo super-resolution ultrasound imaging, offering contrast on demand and vascular selectivity. Three different formulations of acoustically activatable contrast agents were used. We demonstrate their use with ultrasound mechanical indices well within recommended safety limits to enable fast on-demand sparse activation and destruction at very high agent concentrations. We produce super-localization maps of the rabbit renal vasculature with acquisition times between 5.5 s and 0.25 s, and a 4-fold improvement in spatial resolution. We present the unique selectivity of AWSALM in visualizing specific vascular branches and downstream microvasculature, and we show super-localized kidney structures in systole (0.25 s) and diastole (0.25 s) with fast-AWSALM outdoing microbubble based ULM. In conclusion, we demonstrate the feasibility of fast and selective measurement of microvascular dynamics in vivo with subwavelength resolution using ultrasound and acoustically activatable nanodroplet contrast agents

Review of photoacoustic imaging plus X

Photoacoustic imaging (PAI) is a novel modality in biomedical imaging technology that combines the rich optical contrast with the deep penetration of ultrasound. To date, PAI technology has found applications in various biomedical fields. In this review, we present an overview of the emerging research frontiers on PAI plus other advanced technologies, named as PAI plus X, which includes but not limited to PAI plus treatment, PAI plus new circuits design, PAI plus accurate positioning system, PAI plus fast scanning systems, PAI plus novel ultrasound sensors, PAI plus advanced laser sources, PAI plus deep learning, and PAI plus other imaging modalities. We will discuss each technology's current state, technical advantages, and prospects for application, reported mostly in recent three years. Lastly, we discuss and summarize the challenges and potential future work in PAI plus X area

BUbble Flow Field: a Simulation Framework for Evaluating Ultrasound Localization Microscopy Algorithms

Ultrasound contrast enhanced imaging has seen widespread uptake in research and clinical diagnostic imaging. This includes applications such as vector flow imaging, functional ultrasound and super-resolution Ultrasound Localization Microscopy (ULM). All of these require testing and validation during development of new algorithms with ground truth data. In this work we present a comprehensive simulation platform BUbble Flow Field (BUFF) that generates contrast enhanced ultrasound images in vascular tree geometries with realistic flow characteristics and validation algorithms for ULM. BUFF allows complex micro-vascular network generation of random and user-defined vascular networks. Blood flow is simulated with a fast Computational Fluid Dynamics (CFD) solver and allows arbitrary input and output positions and custom pressures. The acoustic field simulation is combined with non-linear Microbubble (MB) dynamics and simulates a range of point spread functions based on user-defined MB characteristics. The validation combines both binary and quantitative metrics. BFF's capacity to generate and validate user-defined networks is demonstrated through its implementation in the Ultrasound Localisation and TRacking Algorithms for Super Resolution (ULTRA-SR) Challenge at the International Ultrasonics Symposium (IUS) 2022 of the Institute of Electrical and Electronics Engineers (IEEE). The ability to produce ULM images, and the availability of a ground truth in localisation and tracking enables objective and quantitative evaluation of the large number of localisation and tracking algorithms developed in the field. BUFF can also benefit deep learning based methods by automatically generating datasets for training. BUFF is a fully comprehensive simulation platform for testing and validation of novel ULM techniques and is open source.Comment: 10 Pages, 9 Figure