Why is the winner the best?

International benchmarking competitions have becomefundamental for the comparative performance assessmentof image analysis methods. However, little attention hasbeen given to investigating what can be learnt from thesecompetitions. Do they really generate scientific progress?What are common and successful participation strategies?What makes a solution superior to a competing method?To address this gap in the literature, we performed a multi-center study with all 80 competitions that were conducted inthe scope of IEEE ISBI 2021 and MICCAI 2021. Statisticalanalyses performed based on comprehensive descriptions ofthe submitted algorithms linked to their rank as well as theunderlying participation strategies revealed common char-acteristics of winning solutions. These typically includethe use of multi-task learning (63%) and/or multi-stagepipelines (61%), and a focus on augmentation (100%), im-age preprocessing (97%), data curation (79%), and post-processing (66%). The “typical” lead of a winning teamis a computer scientist with a doctoral degree, five years ofexperience in biomedical image analysis, and four years ofexperience in deep learning. Two core general developmentstrategies stood out for highly-ranked teams: the reflectionof the metrics in the method design and the focus on analyz-ing and handling failure cases. According to the organizers,43% of the winning algorithms exceeded the state of the artbut only 11% completely solved the respective domain prob-lem. The insights of our study could help researchers (1)improve algorithm development strategies when approach-ing new problems, and (2) focus on open research questionsrevealed by this work

The prognostic significance of [18F]FDG PET/CT in multiple myeloma according to novel interpretation criteria (IMPeTUs)

Purpose!#![!##!Methods!#!Forty-seven patients with newly diagnosed MM underwent [!##!Results!#!Median follow-up from baseline and follow-up PET/CT were 85.1 months and 76.7 months, respectively. The number of focal, [!##!Conclusion!#!Several parameters utilized in IMPeTUs predict PFS in MM patients, suggesting the potentially significant role of the new criteria in patient stratification and response assessment. Additional studies are warranted for the further evaluation of IMPeTUs in the direction of establishment of robust cut-off values with a prognostic significance in the disease

Data_Sheet_1_Capturing COPD heterogeneity: anomaly detection and parametric response mapping comparison for phenotyping on chest computed tomography.docx

BackgroundChronic obstructive pulmonary disease (COPD) poses a substantial global health burden, demanding advanced diagnostic tools for early detection and accurate phenotyping. In this line, this study seeks to enhance COPD characterization on chest computed tomography (CT) by comparing the spatial and quantitative relationships between traditional parametric response mapping (PRM) and a novel self-supervised anomaly detection approach, and to unveil potential additional insights into the dynamic transitional stages of COPD.MethodsNon-contrast inspiratory and expiratory CT of 1,310 never-smoker and GOLD 0 individuals and COPD patients (GOLD 1–4) from the COPDGene dataset were retrospectively evaluated. A novel self-supervised anomaly detection approach was applied to quantify lung abnormalities associated with COPD, as regional deviations. These regional anomaly scores were qualitatively and quantitatively compared, per GOLD class, to PRM volumes (emphysema: PRMEmph, functional small-airway disease: PRMfSAD) and to a Principal Component Analysis (PCA) and Clustering, applied on the self-supervised latent space. Its relationships to pulmonary function tests (PFTs) were also evaluated.ResultsInitial t-Distributed Stochastic Neighbor Embedding (t-SNE) visualization of the self-supervised latent space highlighted distinct spatial patterns, revealing clear separations between regions with and without emphysema and air trapping. Four stable clusters were identified among this latent space by the PCA and Cluster Analysis. As the GOLD stage increased, PRMEmph, PRMfSAD, anomaly score, and Cluster 3 volumes exhibited escalating trends, contrasting with a decline in Cluster 2. The patient-wise anomaly scores significantly differed across GOLD stages (p Emph, PRMfSAD, and cluster classes showed fewer significant differences. Pearson correlation coefficients revealed moderate anomaly score correlations to PFTs (0.41–0.68), except for the functional residual capacity and smoking duration. The anomaly score was correlated with PRMEmph (r = 0.66, p fSAD (r = 0.61, p ConclusionOur study highlights the synergistic utility of the anomaly detection approach and traditional PRM in capturing the nuanced heterogeneity of COPD. The observed disparities in spatial patterns, cluster dynamics, and correlations with PFTs underscore the distinct – yet complementary – strengths of these methods. Integrating anomaly detection and PRM offers a promising avenue for understanding of COPD pathophysiology, potentially informing more tailored diagnostic and intervention approaches to improve patient outcomes.</p

Why is the winner the best?

International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multi-center study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), image preprocessing (97%), data curation (79%), and postprocessing (66%). The "typical" lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work

Why is the winner the best?

International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multi-center study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), image preprocessing (97%), data curation (79%), and postprocessing (66%). The "typical" lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work