Simulation and Synthesis in Medical Imaging

This editorial introduces the Special Issue on Simulation and Synthesis in Medical Imaging. In this editorial, we define so-far ambiguous terms of simulation and synthesis in medical imaging. We also briefly discuss the synergistic importance of mechanistic (hypothesis-driven) and phenomenological (data-driven) models of medical image generation. Finally, we introduce the twelve papers published in this issue covering both mechanistic (5) and phenomenological (7) medical image generation. This rich selection of papers covers applications in cardiology, retinopathy, histopathology, neurosciences, and oncology. It also covers all mainstream diagnostic medical imaging modalities. We conclude the editorial with a personal view on the field and highlight some existing challenges and future research opportunities

AutoRoot: open-source software employing a novel image analysis approach to support fully-automated plant phenotyping

Background: Computer-based phenotyping of plants has risen in importance in recent years. Whilst much software has been written to aid phenotyping using image analysis, to date the vast majority has been only semi-automatic. However, such interaction is not desirable in high throughput approaches. Here, we present a system designed to analyse plant images in a completely automated manner, allowing genuine high throughput measurement of root traits. To do this we introduce a new set of proxy traits. Results: We test the system on a new, automated image capture system, the Microphenotron, which is able to image many 1000s of roots/h. A simple experiment is presented, treating the plants with differing chemical conditions to produce different phenotypes. The automated imaging setup and the new software tool was used to measure proxy traits in each well. A correlation matrix was calculated across automated and manual measures, as a validation. Some particular proxy measures are very highly correlated with the manual measures (e.g. proxy length to manual length, r2 > 0.9). This suggests that while the automated measures are not directly equivalent to classic manual measures, they can be used to indicate phenotypic differences (hence the term, proxy). In addition, the raw discriminative power of the new proxy traits was examined. Principal component analysis was calculated across all proxy measures over two phenotypically-different groups of plants. Many of the proxy traits can be used to separate the data in the two conditions. Conclusion: The new proxy traits proposed tend to correlate well with equivalent manual measures, where these exist. Additionally, the new measures display strong discriminative power. It is suggested that for particular phenotypic differences, different traits will be relevant, and not all will have meaningful manual equivalent measures. However, approaches such as PCA can be used to interrogate the resulting data to identify differences between datasets. Select images can then be carefully manually inspected if the nature of the precise differences is required. We suggest such flexible measurement approaches are necessary for fully automated, high throughput systems such as the Microphenotron

Citizen crowds and experts: observer variability in image-based plant phenotyping

Background:Image-based plant phenotyping has become a powerful tool in unravelling genotype–environment interactions. The utilization of image analysis and machine learning have become paramount in extracting data stemming from phenotyping experiments. Yet we rely on observer (a human expert) input to perform the phenotyping process. We assume such input to be a ‘gold-standard’ and use it to evaluate software and algorithms and to train learning-based algorithms. However, we should consider whether any variability among experienced and non-experienced (including plain citizens) observers exists. Here we design a study that measures such variability in an annotation task of an integer-quantifiable phenotype: the leaf count.Results:We compare several experienced and non-experienced observers in annotating leaf counts in images of Arabidopsis Thaliana to measure intra- and inter-observer variability in a controlled study using specially designed annotation tools but also citizens using a distributed citizen-powered web-based platform. In the controlled study observers counted leaves by looking at top-view images, which were taken with low and high resolution optics. We assessed whether the utilization of tools specifically designed for this task can help to reduce such variability. We found that the presence of tools helps to reduce intra-observer variability, and that although intra- and inter-observer variability is present it does not have any effect on longitudinal leaf count trend statistical assessments. We compared the variability of citizen provided annotations (from the web-based platform) and found that plain citizens can provide statistically accurate leaf counts. We also compared a recent machine-learning based leaf counting algorithm and found that while close in performance it is still not within inter-observer variability.Conclusions:While expertise of the observer plays a role, if sufficient statistical power is present, a collection of non-experienced users and even citizens can be included in image-based phenotyping annotation tasks as long they are suitably designed. We hope with these findings that we can re-evaluate the expectations that we have from automated algorithms: as long as they perform within observer variability they can be considered a suitable alternative. In addition, we hope to invigorate an interest in introducing suitably designed tasks on citizen powered platforms not only to obtain useful information (for research) but to help engage the public in this societal important problem

Zea mays iRS1563: A Comprehensive Genome-Scale Metabolic Reconstruction of Maize Metabolism

The scope and breadth of genome-scale metabolic reconstructions have continued to expand over the last decade. Herein, we introduce a genome-scale model for a plant with direct applications to food and bioenergy production (i.e., maize). Maize annotation is still underway, which introduces significant challenges in the association of metabolic functions to genes. The developed model is designed to meet rigorous standards on gene-protein-reaction (GPR) associations, elementally and charged balanced reactions and a biomass reaction abstracting the relative contribution of all biomass constituents. The metabolic network contains 1,563 genes and 1,825 metabolites involved in 1,985 reactions from primary and secondary maize metabolism. For approximately 42% of the reactions direct literature evidence for the participation of the reaction in maize was found. As many as 445 reactions and 369 metabolites are unique to the maize model compared to the AraGEM model for A. thaliana. 674 metabolites and 893 reactions are present in Zea mays iRS1563 that are not accounted for in maize C4GEM. All reactions are elementally and charged balanced and localized into six different compartments (i.e., cytoplasm, mitochondrion, plastid, peroxisome, vacuole and extracellular). GPR associations are also established based on the functional annotation information and homology prediction accounting for monofunctional, multifunctional and multimeric proteins, isozymes and protein complexes. We describe results from performing flux balance analysis under different physiological conditions, (i.e., photosynthesis, photorespiration and respiration) of a C4 plant and also explore model predictions against experimental observations for two naturally occurring mutants (i.e., bm1 and bm3). The developed model corresponds to the largest and more complete to-date effort at cataloguing metabolism for a plant species

Detecting Myocardial Ischemia at Rest with Cardiac Phase-Resolved BOLD CMR

Background—Fast, noninvasive identification of ischemic territories at rest (prior to tissue-specific changes) and assessment of functional status can be valuable in the management of severe coronary artery disease. This study investigated the utility of cardiac phase-resolved Blood-Oxygen-Level-Dependent (CP-BOLD) CMR in detecting myocardial ischemia at rest secondary to severe coronary artery stenosis. Methods and Results—CP-BOLD, standard-cine, and T2-weighted images were acquired in canines (n=11) at baseline and within 20 minutes of ischemia induction (severe LAD stenosis) at rest. Following 3-hours of ischemia, LAD stenosis was removed and T2-weighted and late-gadolinium-enhancement (LGE) images were acquired. From standard-cine and CP-BOLD images, End-Systolic (ES) and End-Diastolic (ED) myocardium were segmented. Affected and remote sections of the myocardium were identified from post-reperfusion LGE images. S/D, quotient of mean ES and ED signal intensities (on CP-BOLD and standard-cine), was computed for affected and remote segments at baseline and ischemia. Ejection fraction (EF) and segmental wall-thickening (sWT) were derived from CP-BOLD images at baseline and ischemia. On CP-BOLD images: S/D was greater than 1 (remote and affected territories) at baseline; S/D was diminished only in affected territories during ischemia and the findings were statistically significant (ANOVA, post-hoc p<0.01). The dependence of S/D on ischemia was not observed in standard-cine images. Computer simulations confirmed the experimental findings. ROC analysis showed that S/D identifies affected regions with similar performance (AUC:0.87) as EF (AUC:0.89) and sWT (AUC:0.75). Conclusions—Preclinical studies and computer simulations showed that CP-BOLD CMR could be useful in detecting myocardial ischemia at rest. Patient studies are needed for clinical translation

Long-short diffeomorphism memory network for weakly-supervised ultrasound landmark tracking

Ultrasound is a promising medical imaging modality benefiting from low-cost and real-time acquisition.  Accurate tracking of an anatomical landmark has been of high inter-est for various clinical workflows such as minimally invasive surgery and ultrasound-guided radiation therapy.  However, tracking an anatomical landmark accurately in ul-trasound video is very challenging, due to landmark deformation, visual ambiguity andpartial observation. In this paper, we propose a long-short diffeomorphism memory net-work (LSDM), which is a multi-task framework with an auxiliary learnable deformationprior to supporting accurate landmark tracking.  Specifically, we design a novel diffeo-morphic representation, which contains both long and short temporal information storedin separate memory banks for delineating motion margins and reducing cumulative er-rors.   We further propose an expectation maximization memory alignment (EMMA)algorithm to iteratively optimize both the long and short deformation memory, updatingthe  memory  queue  for  mitigating  local  anatomical  ambiguity.   The  proposed  multi-task system can be trained in a weakly-supervised manner,  which only requires fewlandmark annotations for tracking and zero annotation for deformation learning.  Weconduct extensive experiments on both public and private ultrasound landmark track-ing datasets.  Experimental results show that LSDM can achieve better or competitivelandmark tracking performance with a strong generalization capability across differentscanner types and different ultrasound modalities, compared with other state-of-the-artmethods.</p

Cardiac aging synthesis from cross-sectional data with conditional generative adversarial networks

Age has important implications for health, and understanding how age manifests in the human body is the first step for a potential intervention. This becomes especially important for cardiac health, since age is the main risk factor for development of cardiovascular disease. Data-driven modeling of age progression has been conducted successfully in diverse applications such as face or brain aging. While longitudinal data is the preferred option for training deep learning models, collecting such a dataset is usually very costly, especially in medical imaging. In this work, a conditional generative adversarial network is proposed to synthesize older and younger versions of a heart scan by using only cross-sectional data. We train our model with more than 14,000 different scans from the UK Biobank. The induced modifications focused mainly on the interventricular septum and the aorta, which is consistent with the existing literature in cardiac aging. We evaluate the results by measuring image quality, the mean absolute error for predicted age using a pre-trained regressor, and demonstrate the application of synthetic data for counter-balancing biased datasets. The results suggest that the proposed approach is able to model realistic changes in the heart using only cross-sectional data and that these data can be used to correct age bias in a dataset

Long-short diffeomorphism memory network for weakly-supervised ultrasound landmark tracking

Ultrasound is a promising medical imaging modality benefiting from low-cost and real-time acquisition.  Accurate tracking of an anatomical landmark has been of high inter-est for various clinical workflows such as minimally invasive surgery and ultrasound-guided radiation therapy.  However, tracking an anatomical landmark accurately in ul-trasound video is very challenging, due to landmark deformation, visual ambiguity andpartial observation. In this paper, we propose a long-short diffeomorphism memory net-work (LSDM), which is a multi-task framework with an auxiliary learnable deformationprior to supporting accurate landmark tracking.  Specifically, we design a novel diffeo-morphic representation, which contains both long and short temporal information storedin separate memory banks for delineating motion margins and reducing cumulative er-rors.   We further propose an expectation maximization memory alignment (EMMA)algorithm to iteratively optimize both the long and short deformation memory, updatingthe  memory  queue  for  mitigating  local  anatomical  ambiguity.   The  proposed  multi-task system can be trained in a weakly-supervised manner,  which only requires fewlandmark annotations for tracking and zero annotation for deformation learning.  Weconduct extensive experiments on both public and private ultrasound landmark track-ing datasets.  Experimental results show that LSDM can achieve better or competitivelandmark tracking performance with a strong generalization capability across differentscanner types and different ultrasound modalities, compared with other state-of-the-artmethods.</p