Why rankings of biomedical image analysis competitions should be interpreted with care

International challenges have become the standard for validation of biomedical image analysis methods. Given their scientific impact, it is surprising that a critical analysis of common practices related to the organization of challenges has not yet been performed. In this paper, we present a comprehensive analysis of biomedical image analysis challenges conducted up to now. We demonstrate the importance of challenges and show that the lack of quality control has critical consequences. First, reproducibility and interpretation of the results is often hampered as only a fraction of relevant information is typically provided. Second, the rank of an algorithm is generally not robust to a number of variables such as the test data used for validation, the ranking scheme applied and the observers that make the reference annotations. To overcome these problems, we recommend best practice guidelines and define open research questions to be addressed in the future

Hypertensive Disorders of Pregnancy and DNA Methylation in Newborns Findings From the Pregnancy and Childhood Epigenetics Consortium

Hypertensive disorders of pregnancy (HDP) are associated with low birth weight, shorter gestational age, and increased risk of maternal and offspring cardiovascular diseases later in life. The mechanisms involved are poorly understood, but epigenetic regulation of gene expression may play a part. We performed meta-analyses in the Pregnancy and Childhood Epigenetics Consortium to test the association between either maternal HDP (10 cohorts; n=5242 [cases=476]) or preeclampsia (3 cohorts; n=2219 [cases=135]) and epigenome-wide DNA methylation in cord blood using the Illumina HumanMethylation450 BeadChip. In models adjusted for confounders, and with Bonferroni correction, HDP and preeclampsia were associated with DNA methylation at 43 and 26 CpG sites, respectively. HDP was associated with higher methylation at 27 (63%) of the 43 sites, and across all 43 sites, the mean absolute difference in methylation was between 0.6% and 2.6%. Epigenome-wide associations of HDP with offspring DNA methylation were modestly consistent with the equivalent epigenome-wide associations of preeclampsia with offspring DNA methylation (R2=0.26). In longitudinal analyses conducted in 1 study (n=108 HDP cases; 550 controls), there were similar changes in DNA methylation in offspring of those with and without HDP up to adolescence. Pathway analysis suggested that genes located at/near HDP-associated sites may be involved in developmental, embryogenesis, or neurological pathways. HDP is associated with offspring DNA methylation with potential relevance to development