Image Processing Using FPGAs

This book presents a selection of papers representing current research on using field programmable gate arrays (FPGAs) for realising image processing algorithms. These papers are reprints of papers selected for a Special Issue of the Journal of Imaging on image processing using FPGAs. A diverse range of topics is covered, including parallel soft processors, memory management, image filters, segmentation, clustering, image analysis, and image compression. Applications include traffic sign recognition for autonomous driving, cell detection for histopathology, and video compression. Collectively, they represent the current state-of-the-art on image processing using FPGAs

Nanoscale imaging of clinical specimens using pathology-optimized expansion microscopy

Expansion microscopy (ExM), a method for improving the resolution of light microscopy by physically expanding the specimen, has not been applied to clinical tissue samples. Here we report a clinically optimized form of ExM that supports nanoscale imaging of human tissue specimens that have been fixed with formalin, embedded in paraffin, stained with hematoxylin and eosin (H&E), and/or fresh frozen. The method, which we call expansion pathology (ExPath), converts clinical samples into an ExM-compatible state, then applies an ExM protocol with protein anchoring and mechanical homogenization steps optimized for clinical samples. ExPath enables ~70 nm resolution imaging of diverse biomolecules in intact tissues using conventional diffraction-limited microscopes, and standard antibody and fluorescent DNA in situ hybridization reagents. We use ExPath for optical diagnosis of kidney minimal-change disease, which previously required electron microscopy (EM), and demonstrate high-fidelity computational discrimination between early breast neoplastic lesions that to date have challenged human judgment. ExPath may enable the routine use of nanoscale imaging in pathology and clinical research

A Pragmatic Machine Learning Approach to Quantify Tumor-Infiltrating Lymphocytes in Whole Slide Images

Increased levels of tumor-infiltrating lymphocytes (TILs) indicate favorable outcomes in many types of cancer. The manual quantification of immune cells is inaccurate and time-consuming for pathologists. Our aim is to leverage a computational solution to automatically quantify TILs in standard diagnostic hematoxylin and eosin-stained sections (H&E slides) from lung cancer patients. Our approach is to transfer an open-source machine learning method for the segmentation and classification of nuclei in H&E slides trained on public data to TIL quantification without manual labeling of the data. Our results show that the resulting TIL quantification correlates to the patient prognosis and compares favorably to the current state-of-the-art method for immune cell detection in non-small cell lung cancer (current standard CD8 cells in DAB-stained TMAs HR 0.34, 95% CI 0.17–0.68 vs. TILs in HE WSIs: HoVer-Net PanNuke Aug Model HR 0.30, 95% CI 0.15–0.60 and HoVer-Net MoNuSAC Aug model HR 0.27, 95% CI 0.14–0.53). Our approach bridges the gap between machine learning research, translational clinical research and clinical implementation. However, further validation is warranted before implementation in a clinical setting

Pattern recognition systems design on parallel GPU architectures for breast lesions characterisation employing multimodality images

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University London.The aim of this research was to address the computational complexity in designing multimodality Computer-Aided Diagnosis (CAD) systems for characterising breast lesions, by harnessing the general purpose computational potential of consumer-level Graphics Processing Units (GPUs) through parallel programming methods. The complexity in designing such systems lies on the increased dimensionality of the problem, due to the multiple imaging modalities involved, on the inherent complexity of optimal design methods for securing high precision, and on assessing the performance of the design prior to deployment in a clinical environment, employing unbiased system evaluation methods. For the purposes of this research, a Pattern Recognition (PR)-system was designed to provide highest possible precision by programming in parallel the multiprocessors of the NVIDIA’s GPU-cards, GeForce 8800GT or 580GTX, and using the CUDA programming framework and C++. The PR-system was built around the Probabilistic Neural Network classifier and its performance was evaluated by a re-substitution method, for estimating the system’s highest accuracy, and by the external cross validation method, for assessing the PR-system’s unbiased accuracy to new, “unseen” by the system, data. Data comprised images of patients with histologically verified (benign or malignant) breast lesions, who underwent both ultrasound (US) and digital mammography (DM). Lesions were outlined on the images by an experienced radiologist, and textural features were calculated. Regarding breast lesion classification, the accuracies for discriminating malignant from benign lesions were, 85.5% using US-features alone, 82.3% employing DM-features alone, and 93.5% combining US and DM features. Mean accuracy to new “unseen” data for the combined US and DM features was 81%. Those classification accuracies were about 10% higher than accuracies achieved on a single CPU, using sequential programming methods, and 150-fold faster. In addition, benign lesions were found smoother, more homogeneous, and containing larger structures. Additionally, the PR-system design was adapted for tackling other medical problems, as a proof of its generalisation. These included classification of rare brain tumours, (achieving 78.6% for overall accuracy (OA) and 73.8% for estimated generalisation accuracy (GA), and accelerating system design 267 times), discrimination of patients with micro-ischemic and multiple sclerosis lesions (90.2% OA and 80% GA with 32-fold design acceleration), classification of normal and pathological knee cartilages (93.2% OA and 89% GA with 257-fold design acceleration), and separation of low from high grade laryngeal cancer cases (93.2% OA and 89% GA, with 130-fold design acceleration). The proposed PR-system improves breast-lesion discrimination accuracy, it may be redesigned on site when new verified data are incorporated in its depository, and it may serve as a second opinion tool in a clinical environment

Embedded machine learning using microcontrollers in wearable and ambulatory systems for health and care applications: a review

The use of machine learning in medical and assistive applications is receiving significant attention thanks to the unique potential it offers to solve complex healthcare problems for which no other solutions had been found. Particularly promising in this field is the combination of machine learning with novel wearable devices. Machine learning models, however, suffer from being computationally demanding, which typically has resulted on the acquired data having to be transmitted to remote cloud servers for inference. This is not ideal from the system’s requirements point of view. Recently, efforts to replace the cloud servers with an alternative inference device closer to the sensing platform, has given rise to a new area of research Tiny Machine Learning (TinyML). In this work, we investigate the different challenges and specifications trade-offs associated to existing hardware options, as well as recently developed software tools, when trying to use microcontroller units (MCUs) as inference devices for health and care applications. The paper also reviews existing wearable systems incorporating MCUs for monitoring, and management, in the context of different health and care intended uses. Overall, this work addresses the gap in literature targeting the use of MCUs as edge inference devices for healthcare wearables. Thus, can be used as a kick-start for embedding machine learning models on MCUs, focusing on healthcare wearables

A phantom for the quantitative determination and improvement of the spatial resolution in slice-selective 2D-FT magnetic resonance micro-imaging and -microscopy based on Deep X-ray Lithography (DXRL)

Introduction: The most important assessed quality-control (QC) criteria for improvements in high-resolution imaging are represented by the contrast-to-noise-ratio and spatial resolution. Ultra-High-Field (UHF) Magnetic-Resonance-scanners (B ≥ 7 T) for medical research allowed for the improvement in spatial resolution up to the microimaging and nominal microscopy range [pixel-size: ps < (100 μm)2], even in-vivo on humans just recently. Preclinical MRI- and dedicated MR-microscopy (MRM) scanners already allow for microimaging and MRM (1-256 μm) but lack a sensible spatial resolution phantom for QC and performance improvements in hardware, pulse-sequencing and MRprotocols. In most scientific MRI articles, the spatial resolution is characterized by the ps, though this measurement parameter only limits the actual resolution. Methods: Here the Modulation-Transfer-Function (MTF) is used as evaluation concept for the determination of the spatial resolution in MRM using simple intensity profiles. The resolution limit is defined using a critical modulation-level. In approaching visual impressions on spatial resolution an additional criterion derived from the Modulation-depth-to-Noise-Ratio (MNR) is proposed. A practical method for assessment based on a concrete phantom design and its realization is shown. Results: The phantom design consists of several sets of fine grids, specifically featuring high structural anisotropy for optimum SNR and CNR, with different spatial periods ranging from a1 = 256 μm down to a8 = 2 μm, not only for a quick visual qualitative check, but also for quantification of resolution using the MTF for two different spatial encodings in two orthogonal in-plane directions. The challenging demands on the manufacturing technology especially with regard to the aspect-ratio are approached using Deep-X-Ray-Lithography (DXRL) relying on the high brilliance of Synchroton-radiation. Smallest grid plates with width of 4 μm corresponding to 125 line pairs/mm at a plate depth of 100 μm were achieved. Discussion: MR-microscopic images, originating from a microscopy insert on a human UHF-MR-scanner, were used for demonstration of the evaluation process with two independent resolution-criteria. The developed prototype offers unique possibilities for quantitative resolution QC on UHF human and preclinical MR-scanners. Such a resolution-phantom might be very important for the improvement of MR-pulse-sequences, MR-protocols and even hardware. In principle the phantom can also be used for other microscopic imaging-modalities as for instance μCT and Optical-Coherence-Tomography (OCT)

SYNCHROTRON RADIATION INLINE PROPAGATION BASED PHASE CONTRAST COMPUTERIZED TOMOGRAPHY (PC-CT) OF HUMAN PROSTATE SAMPLE

The human prostate is an accessory male reproductive gland located below the neck of the urinary bladder. Benign prostatic hyperplasia (BPH) and prostate cancer are the frequently encountered pathological conditions of the prostate. It is estimated that 50% of men will develop BPH by age 50 with the incidence increasing to 90% by age 90. Prostate cancer is the second most common cause of cancer in men worldwide after lung cancer. In this study, we examined the ability of synchrotron radiation propagation phase-contrast computerized tomography (PC-CT) in comparison to ultrasound (US), magnetic resonance imaging (MRI) and histology, to characterize and differentiate various structural features and pathological lesions in 61 prostate tissues from 13 human patients collected during trans-urethral resection of the prostate. We compared the PC-CT, MRI, US and histology images of the same tissues from the same plane to determine if different structures like blood vessels, dilated acini etc. could be observed with each modality. The PC-CT was found to be a powerful imaging technique compared to MRI and US in identifying and resolving small structures located near each other. With PC-CT imaging, the same structures could be correctly identified almost 4 times and 15 times more often than MRI and US respectively. While comparing the ability to identify and resolve the nearby structures in PC-CT images reconstructed from 100%, 50% and 25% of the number of total projections collected (i.e. 2250 projections over 180 degree rotation of a sample on imaging stage), the ranking was as follows: 100% PC-CT>50% PC-CT>25% PC-CT (p<0.05). Radiation data recorded during a previous study while imaging dog cadavers with PC-CT were also analyzed. It was found that the average effective radiation dose imparted in a medium-sized dog during PC-CT imaging of one view of 7.8 mm height with 2000 projections in the biomedical imaging and therapy – insertion device (BMIT-ID) beamline of Canadian light source (CLS) beamline was 1,481.7 mSv, which is very high compared to the standard clinical CT examination deposits in human clinical medicine. The dose could be reduced by performing sparse view imaging i.e. 50% projection PC-CT or 25% projection PC-CT, but these amounts are still hazardous, such that a similar protocol used in human would have the potential to induce cancer later in life in approximately 0.5 % of the patients. For PC-CT imaging of human prostate in situ, a human positioning device was also designed. Due to the limitation in the weight-bearing capacity of the stage in the beamline, the positioning device was designed to be able to hold only a human pelvis or pelvis phantom up to 50 kg of weight in an upright position. The results from this work demonstrate that the synchrotron radiation-based inline PC-CT is a promising technique that offers closer-to-histology grade non-invasive diagnostic imaging of prostate tissue. Further study in conducting in-vivo prostate imaging to reduce the radiation dose is the next step to move forward in this direction

Aerospace Medicine and Biology: A continuing bibliography with indexes, supplement 174

This bibliography lists 181 reports, articles, and other documents introduced into the NASA scientific and technical information system in November 1977

Superparamagnetic iron oxide nanoparticles for imaging and drug delivery

In the past years, nanoparticle usage and research increased enormously in different fields, especially in the clinical sciences for drug delivery and imaging. Selection of the most suitable type of nanoparticle is not always easy because of a broad material variety and different physicochemical characteristics. Depending on the purpose, the safe usage of nanoparticles in vivo needs to be ensured first to predict and eliminate unwanted effects like agglomeration, loss of function, immune system responses (i.e. inflammation), or toxicity after intravenous application. To guarantee the safe usage of nanoparticles, there are various hazard evaluation strategies for different scenarios for example for nanomedicines. Drug delivery with nanomedicines has advantages like increased absorbability, increased in vivo half-life, and decreased drug dosage needs to reach the same therapeutic effect. Amongst others, superparamagnetic iron oxide nanoparticles (SPIONs) and liposomes are already used as medicinal nanoparticles and are approved from the US Food and Drug Administration (FDA). Examples are Feraheme®, which is a ferumoxytol injection for iron deficiency treatments and Doxil®, a liposomal doxorubicin hydrochloride chemotherapy drug. SPIONs are also used as contrast agents because of their high-dense core and the possibility to synthesize very small diameters below 10 nm, which can even penetrate into smallest fenestrations of i.e. the kidneys. This work is divided into two main parts: Part one was the analysis of different nanoparticle safety evaluation strategies to propose a new hazard evaluation guideline for intravenously applied nanoparticles. This was part of the NanoREG II European Union’s Horizon 2020 research and innovation program under grant agreement 646221. Part two started with the synthesis and physicochemical characterization of hybrid nanoparticles made out of SPION cores and liposome coatings. The coatings were differently modified with additions like polyethyleneglycol for increased in vivo half-life or folic acid for renal targeting. Injected into zebrafish (Danio rerio) embryos, their biodistribution and toxicity was analyzed. Various methods like confocal laser scanning microscopy and synchrotron X-ray radiation micro-computed phase-contrast tomography were used and compared with each other. Finally, those hybrid nanoparticles were manipulated in vivo with external magnets to increase phagocytic uptake and also with electromagnetic fields and acoustic waves for controlling the nanoparticles in vivo in terms of agglomeration and rotation

Aerospace medicine and biology: A continuing bibliography with indexes, supplement 190, February 1979

This bibliography lists 235 reports, articles, and other documents introduced into the NASA scientific and technical information system in January 1979