Why is the Winner the Best?

International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multicenter study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), image preprocessing (97%), data curation (79%), and post-processing (66%). The “typical” lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work

Why is the winner the best?

International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multicenter study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), image preprocessing (97%), data curation (79%), and post-processing (66%). The 'typical' lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work

Improved constraints on in situ rates and on quantification of complete chloroethene degradation from stable carbon isotope mass balances in groundwater plumes

International audienceSpills of chloroethenes (CEs) at industrial and urban sites can create groundwater plumes in which tetrachloro- and trichloroethene sequentially degrade to dichloroethenes, vinyl chloride (VC) and ethene, or ethane under reducing conditions. For detoxification, degradation must go beyond VC. Assessments based on ethene and ethane, however, `` are difficult because these products are volatile, may stem from alternative sources, can be further transformed and are not always monitored. To alternatively quantify degradation beyond VC, stable carbon isotope mass balances have been proposed where concentration-weighted CE isotope ratios are summed up and compared to the original source isotope ratio. Reported assessments, however, have provided not satisfactorily quantified results entailing greatly differing upper and lower estimates. This work proposes an integrative approach to better constrain the extent of total chloroethene degradation in groundwater samples. It is based on fitting of measured concentration and compound-specific stable carbon isotope data to an analytical reactive transport equation simulating steady-state plumes in two dimensions using an EXCEL spreadsheet. The fitting also yields estimates of degradation rates, of source width and of dispersivities. The approach is validated using two synthetic benchmark cases where the true extent of degradation is well known, and using data from two real field cases from literature. (C) 2015 Elsevier BM. All rights reserved