Self-supervised Representation Learning for Ultrasound Video

Recent advances in deep learning have achieved promising performance for medical image analysis, while in most cases ground-truth annotations from human experts are necessary to train the deep model. In practice, such annotations are expensive to collect and can be scarce for medical imaging applications. Therefore, there is significant interest in learning representations from unlabelled raw data. In this paper, we propose a self-supervised learning approach to learn meaningful and transferable representations from medical imaging video without any type of human annotation. We assume that in order to learn such a representation, the model should identify anatomical structures from the unlabelled data. Therefore we force the model to address anatomy-aware tasks with free supervision from the data itself. Specifically, the model is designed to correct the order of a reshuffled video clip and at the same time predict the geometric transformation applied to the video clip. Experiments on fetal ultrasound video show that the proposed approach can effectively learn meaningful and strong representations, which transfer well to downstream tasks like standard plane detection and saliency prediction.Comment: ISBI 202

Self-Supervised Ultrasound to MRI Fetal Brain Image Synthesis

Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for second-trimester anomaly screening, for which ultrasound (US) is employed. Although expert sonographers are adept at reading US images, MR images which closely resemble anatomical images are much easier for non-experts to interpret. Thus in this paper we propose to generate MR-like images directly from clinical US images. In medical image analysis such a capability is potentially useful as well, for instance for automatic US-MRI registration and fusion. The proposed model is end-to-end trainable and self-supervised without any external annotations. Specifically, based on an assumption that the US and MRI data share a similar anatomical latent space, we first utilise a network to extract the shared latent features, which are then used for MRI synthesis. Since paired data is unavailable for our study (and rare in practice), pixel-level constraints are infeasible to apply. We instead propose to enforce the distributions to be statistically indistinguishable, by adversarial learning in both the image domain and feature space. To regularise the anatomical structures between US and MRI during synthesis, we further propose an adversarial structural constraint. A new cross-modal attention technique is proposed to utilise non-local spatial information, by encouraging multi-modal knowledge fusion and propagation. We extend the approach to consider the case where 3D auxiliary information (e.g., 3D neighbours and a 3D location index) from volumetric data is also available, and show that this improves image synthesis. The proposed approach is evaluated quantitatively and qualitatively with comparison to real fetal MR images and other approaches to synthesis, demonstrating its feasibility of synthesising realistic MR images.Comment: IEEE Transactions on Medical Imaging 202

Automated description and workflow analysis of fetal echocardiography in first-trimester ultrasound video scans

This paper presents a novel, fully-automatic framework for fetal echocardiography analysis of full-length routine firsttrimester fetal ultrasound scan video. In this study, a new deep learning architecture, which considers spatio-temporal information and spatial attention, is designed to temporally partition ultrasound video into semantically meaningful segments. The resulting automated semantic annotation is used to analyse cardiac examination workflow. The proposed 2D+t convolution neural network architecture achieves an A1 accuracy of 96.37%, F1 of 95.61%, and precision of 96.18% with 21.49% fewer parameters than the smallest ResNet-based architecture. Automated deep-learning based semantic annotation of unlabelled video scans (n=250) shows a high correlation with expert cardiac annotations (ρ = 0.96, p = 0.0004), thereby demonstrating the applicability of the proposed annotation model for echocardiography workflow analysis

Clinical impact of the methodological quality of fetal doppler standards in the management of fetal growth restriction

Esta tesis, que lleva por título “Clinical impact of the methodological quality of fetal Doppler standards in the management of fetal growth Restriction”, es un trabajo realizado en la Universidad de Zaragoza con colaboración de la Universidad de Oxford por lo que opta a la mención internacional. Además, está elaborada según la normativa de la Universidad de Zaragoza como tesis por compendio de publicaciones, con 4 artículos publicados en revistas de elevado factor de impacto.El crecimiento intrauterino restringido (CIR) es una de las enfermedades con mayor repercusión médica, social y económica en obstetricia. Estos fetos pueden interrumpir su crecimiento como consecuencia de una insuficiencia placentaria, apareciendo alteraciones en el Doppler fetal, lo que conlleva un riesgo elevado de resultado perinatal adverso. Para que una herramienta como el Doppler fetal sea fiable, los valores obtenidos deben ser adecuados y reproducibles, la medición del Doppler debe estar estandarizada y así, maximizaremos su potencial en la evaluación del CIR en la práctica clínica. Con este objetivo se desarrolló la primera de las publicaciones que propone un sistema de puntuación objetiva para evaluar imágenes Doppler de la arteria cerebral media (ACM), demostrando que los controles de calidad de imágenes son esenciales, así como el uso de sistemas objetivos que hagan que las imágenes sean reproducibles.Por otro lado, la secuencia de progresión del Doppler fetal ha sido descrita claramente y hay evidencia de que los cambios cualitativos en el Doppler de la arterial umbilical (AU), como la presencia, ausencia o inversión del flujo diastólico, indican un mayor riesgo de muerte fetal. Sin embargo, la asociación entre los cambios semi-cuantitativos en el Doppler de AU y ACM (medidos con el índice de pulsatilidad) y los resultados perinatales y a largo plazo no se han establecido claramente. Como consecuencia, se han publicado multitud de valores de referencia del índice de pulsatilidad del Doppler fetal. Esta falta de evidencia podría explicarse, al menos parcialmente, por la calidad metodología utilizada para establecer estos valores, lo que podría tener importantes implicaciones para la práctica clínica. Con esta hipótesis, en el segundo trabajo se realizó una revisión sistemática de todos los estudios publicados con el objetivo de crear curvas de referencia para la AU, ACM e índice cerebroplacentario (ICP). Tras utilizar una lista de verificación ya validada y evaluar 38 estudios, se llegó a la conclusión de que todos los estudios en los que se basan los valores de referencia que se usan en la práctica clínica tienen numerosos sesgos metodológicos, haciendo que las diferencias entre los valores sean importantes. Además, en el tercer estudio, se comparó todos estos valores demostrando su gran variabilidad y se realizó una simulación clínica en la que se observó que el manejo de un feto con crecimiento intrauterino restringido puede variar en dependencia del valor de referencia que se elija, decidiendo finalizar la gestación o no y produciendo en algunos casos prematuridad iatrogénica y en otros, aumento del riesgo de muerte fetal intrauterina.Finalmente, como solución a todos los problemas planteados y a la falta de estudios de alta calidad metodológica en los que basar nuestras actuaciones médicas, se propone el estudio FETHUS, un estudio de cohortes, longitudinal, multicéntrico, internacional y prospectivo, con el objetivo de crear unos valores de referencia basados en un estudio con alta calidad metodológica que sirvan de referencia universal para el Doppler fetal, unificando así el manejo del feto con crecimiento intrauterino restringido.1. Alfirevic Z, Stampalija T, Dowswell T. Fetal and umbilical Doppler ultrasound in high-risk pregnancies. Vol. 2017, Cochrane Database of Systematic Reviews. John Wiley and Sons Ltd; 2017. 2. Alfirevic Z, Stampalija T, Medley N. Fetal and umbilical Doppler ultrasound in normal pregnancy. Vol. 2015, Cochrane Database of Systematic Reviews. John Wiley and Sons Ltd; 2015. 3. Fetal Growth Restriction. Practice Bulletin No. 134. American College of Obstetricians & Gyncologists. Obstet Gynecol. 2013;4. Gordijn SJ, Beune IM, Thilaganathan B, Papageorghiou A, Baschat AA, Baker PN, et al. Consensus definition of fetal growth restriction: a Delphi procedure. Ultrasound Obstet Gynecol [Internet]. 2016 Sep [cited 2019 Aug 3];48(3):333–9. Available from: http://www.ncbi.nlm.nih.gov/pubmed/269096645. RCOG Green-top Guideline, 2nd Edition J 2014. Investigation and Management of the Small for Gestational Age Fetus. R Coll Obstet Gynaecol (RCOG) [Internet]. Available from: http://www.rcog.org.uk/files/rcog-corp/6. Conde-Agudelo A, Villar J, Kennedy SH, Papageorghiou AT. Predictive accuracy of cerebroplacental ratio for adverse perinatal and neurodevelopmental outcomes in suspected fetal growth restriction: systematic review and meta-analysis. Ultrasound Obstet Gynecol. 2018; 7. Oros D, Figueras F, Cruz-Martinez R, Padilla N, Meler E, Hernandez-Andrade E, et al. Middle versus anterior cerebral artery Doppler for the prediction of perinatal outcome and neonatal neurobehavior in term small-for-gestational-age fetuses with normal umbilical artery Doppler. Ultrasound Obstet Gynecol [Internet]. 2010 Apr [cited 2019 Aug 22];35(4):456–61. Available from: http://www.ncbi.nlm.nih.gov/pubmed/201781158. DeVore GR. The importance of the cerebroplacental ratio in the evaluation of fetal well-being in SGA and AGA fetuses. Vol. 213, American Journal of Obstetrics and Gynecology. Mosby Inc.; 2015. p. 5–15. 9. Arduini D, Rizzo G. Normal values of Pulsatility Index from fetal vessels: a cross-sectional study on 1556 healthy fetuses. J Perinat Med [Internet]. 1990 [cited 2019 Aug 3];18(3):165–72. Available from: http://www.ncbi.nlm.nih.gov/pubmed/220086210. Morales-Roselló J, Diaz García-Donato J. Study of fetal femoral and umbilical artery blood flow by Doppler ultrasound throughout pregnancy.11. Figueras F, Gardosi J. Intrauterine growth restriction: New concepts in antenatal surveillance, diagnosis, and management. Vol. 204, American Journal of Obstetrics and Gynecology. Mosby Inc.; 2011. p. 288–300.12. Royston P, Wright EM. How to construct “normal ranges” for fetal variables. Ultrasound Obstet Gynecol. 1998;11(1):30–8.13. Ruiz-Martinez S, Volpe G, Vannuccini S, Cavallaro A, Impey L, Ioannou C. An objective scoring method to evaluate image quality of middle cerebral artery Doppler. J Matern Fetal Neonatal Med [Internet]. 2018 Jun 27 [cited 2019 Sep 12];1–181. Available from: http://www.ncbi.nlm.nih.gov/pubmed/2995015614. Salomon LJ, Bernard JP, Duyme M, Buvat I, Ville Y. The impact of choice of reference charts and equations on the assessment of fetal biometry. Ultrasound Obstet Gynecol. 2005 Jun;25(6):559–65.<br /

Show from Tell:Audio-Visual Modelling in Clinical Settings

Auditory and visual signals usually present together and correlate with each other, not only in natural environments but also in clinical settings. However, the audio-visual modelling in the latter case can be more challenging, due to the different sources of audio/video signals and the noise (both signal-level and semantic-level) in auditory signals -- usually speech. In this paper, we consider audio-visual modelling in a clinical setting, providing a solution to learn medical representations that benefit various clinical tasks, without human expert annotation. A simple yet effective multi-modal self-supervised learning framework is proposed for this purpose. The proposed approach is able to localise anatomical regions of interest during ultrasound imaging, with only speech audio as a reference. Experimental evaluations on a large-scale clinical multi-modal ultrasound video dataset show that the proposed self-supervised method learns good transferable anatomical representations that boost the performance of automated downstream clinical tasks, even outperforming fully-supervised solutions

Discovering Salient Anatomical Landmarks by Predicting Human Gaze

Anatomical landmarks are a crucial prerequisite for many medical imaging tasks. Usually, the set of landmarks for a given task is predefined by experts. The landmark locations for a given image are then annotated manually or via machine learning methods trained on manual annotations. In this paper, in contrast, we present a method to automatically discover and localize anatomical landmarks in medical images. Specifically, we consider landmarks that attract the visual attention of humans, which we term visually salient landmarks. We illustrate the method for fetal neurosonographic images. First, full-length clinical fetal ultrasound scans are recorded with live sonographer gaze-tracking. Next, a convolutional neural network (CNN) is trained to predict the gaze point distribution (saliency map) of the sonographers on scan video frames. The CNN is then used to predict saliency maps of unseen fetal neurosonographic images, and the landmarks are extracted as the local maxima of these saliency maps. Finally, the landmarks are matched across images by clustering the landmark CNN features. We show that the discovered landmarks can be used within affine image registration, with average landmark alignment errors between 4.1% and 10.9% of the fetal head long axis length.Comment: Accepted at IEEE International Symposium on Biomedical Imaging 2020 (ISBI 2020

Show from Tell: Audio-Visual Modelling in Clinical Settings

Auditory and visual signals usually present together and correlate with each other, not only in natural environments but also in clinical settings. However, the audio-visual modelling in the latter case can be more challenging, due to the different sources of audio/video signals and the noise (both signal-level and semantic-level) in auditory signals -- usually speech. In this paper, we consider audio-visual modelling in a clinical setting, providing a solution to learn medical representations that benefit various clinical tasks, without human expert annotation. A simple yet effective multi-modal self-supervised learning framework is proposed for this purpose. The proposed approach is able to localise anatomical regions of interest during ultrasound imaging, with only speech audio as a reference. Experimental evaluations on a large-scale clinical multi-modal ultrasound video dataset show that the proposed self-supervised method learns good transferable anatomical representations that boost the performance of automated downstream clinical tasks, even outperforming fully-supervised solutions

A link between high serum levels of human chorionic gonadotrophin and chorionic expression of its mature functional receptor (LHCGR) in Down's syndrome pregnancies

Human chorionic gonadotrophin (hCG) is released from placental trophoblasts and is involved in establishing pregnancy by maintaining progesterone secretion from the corpus luteum. Serum hCG is detected in the maternal circulation within the first 2–3 wks of gestation and peaks at the end of the first trimester before declining. In Down's syndrome (DS) pregnancies, serum hCG remains significantly high compared to gestation age-matched uncompromised pregnancies. It has been proposed that increased serum hCG levels could be due to transcriptional hyper-activation of the CGB (hCG beta) gene, or an increased half life of glycosylated hCG hormone, or both. Another possibility is that serum hCG levels remain high due to reduced availability of the hormone's cognate receptor, LHCGR, leading to lack of hormone utilization. We have tested this hypothesis by quantifying the expression of the hCG beta (CGB) RNA, LHCGR RNA and LHCGR proteins in chorionic villous samples. We demonstrate that chorionic expression of hCG beta (CGB) mRNA directly correlates with high serum hCG levels. The steady-state synthesis of LHCGR mRNA (exons 1–5) in DS pregnancies was significantly higher than that of controls, but the expression of full-length LHCGR mRNA (exons 1–11) in DS was comparable to that of uncompromised pregnancies. However, the synthesis of high molecular weight mature LHCGR proteins was significantly reduced in DS compared to uncompromised pregnancies, suggesting a lack of utilization of circulating hCG in DS pregnancies

Preeclampsia and COVID-19: results from the INTERCOVID prospective longitudinal study

Coronavirus SARS-CoV-2; COVID-19; 2019-nCoV; Hipertensió gestacional; PreeclampsiaCoronavirus SARS-CoV-2; COVID-19; 2019-nCoV; Hipertension gestacional; PreeclampsiaCoronavirus SARS-CoV-2; COVID-19; 2019-nCoV; Gestational hypertension; PreeclampsiaBackground It is unclear whether the suggested link between COVID-19 during pregnancy and preeclampsia is an independent association or if these are caused by common risk factors. Objective This study aimed to quantify any independent association between COVID-19 during pregnancy and preeclampsia and to determine the effect of these variables on maternal and neonatal morbidity and mortality. Study Design This was a large, longitudinal, prospective, unmatched diagnosed and not-diagnosed observational study assessing the effect of COVID-19 during pregnancy on mothers and neonates. Two consecutive not-diagnosed women were concomitantly enrolled immediately after each diagnosed woman was identified, at any stage during pregnancy or delivery, and at the same level of care to minimize bias. Women and neonates were followed until hospital discharge using the standardized INTERGROWTH-21 st protocols and electronic data management system. A total of 43 institutions in 18 countries contributed to the study sample. The independent association between the 2 entities was quantified with the risk factors known to be associated with preeclampsia analyzed in each group. The outcomes were compared among women with COVID-19 alone, preeclampsia alone, both conditions, and those without either of the 2 conditions. Results We enrolled 2184 pregnant women; of these, 725 (33.2%) were enrolled in the COVID-19 diagnosed and 1459 (66.8%) in the COVID-19 not-diagnosed groups. Of these women, 123 had preeclampsia of which 59 of 725 (8.1%) were in the COVID-19 diagnosed group and 64 of 1459 (4.4%) were in the not-diagnosed group (risk ratio, 1.86; 95% confidence interval, 1.32–2.61). After adjustment for sociodemographic factors and conditions associated with both COVID-19 and preeclampsia, the risk ratio for preeclampsia remained significant among all women (risk ratio, 1.77; 95% confidence interval, 1.25–2.52) and nulliparous women specifically (risk ratio, 1.89; 95% confidence interval, 1.17–3.05). There was a trend but no statistical significance among parous women (risk ratio, 1.64; 95% confidence interval, 0.99–2.73). The risk ratio for preterm birth for all women diagnosed with COVID-19 and preeclampsia was 4.05 (95% confidence interval, 2.99–5.49) and 6.26 (95% confidence interval, 4.35–9.00) for nulliparous women. Compared with women with neither condition diagnosed, the composite adverse perinatal outcome showed a stepwise increase in the risk ratio for COVID-19 without preeclampsia, preeclampsia without COVID-19, and COVID-19 with preeclampsia (risk ratio, 2.16; 95% confidence interval, 1.63–2.86; risk ratio, 2.53; 95% confidence interval, 1.44–4.45; and risk ratio, 2.84; 95% confidence interval, 1.67–4.82, respectively). Similar findings were found for the composite adverse maternal outcome with risk ratios of 1.76 (95% confidence interval, 1.32–2.35), 2.07 (95% confidence interval, 1.20–3.57), and 2.77 (95% confidence interval, 1.66–4.63). The association between COVID-19 and gestational hypertension and the direction of the effects on preterm birth and adverse perinatal and maternal outcomes, were similar to preeclampsia, but confined to nulliparous women with lower risk ratios. Conclusion COVID-19 during pregnancy is strongly associated with preeclampsia, especially among nulliparous women. This association is independent of any risk factors and preexisting conditions. COVID-19 severity does not seem to be a factor in this association. Both conditions are associated independently of and in an additive fashion with preterm birth, severe perinatal morbidity and mortality, and adverse maternal outcomes. Women with preeclampsia should be considered a particularly vulnerable group with regard to the risks posed by COVID-19.The study was supported by the COVID-19 Research Response Fund from the University of Oxford (Ref 0009083). A.T.P. is supported by the Oxford Partnership Comprehensive Biomedical Research Centre with funding from the National Institute for Health Research (NIHR) Biomedical Research Centre funding scheme. The funding organization had no involvement in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript, and decision to submit the manuscript for publication