Current Progress and Challenges in Large-Scale 3D Mitochondria Instance Segmentation

Abstract

© 2023 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/In this paper, we present the results of the MitoEM challenge on mitochondria 3D instance segmentation from electron microscopy images, organized in conjunction with the IEEE-ISBI 2021 conference. Our benchmark dataset consists of two large-scale 3D volumes, one from human and one from rat cortex tissue, which are 1,986 times larger than previously used datasets. At the time of paper submission, 257 participants had registered for the challenge, 14 teams had submitted their results, and six teams participated in the challenge workshop. Here, we present eight top-performing approaches from the challenge participants, along with our own baseline strategies. Posterior to the challenge, annotation errors in the ground truth were corrected without altering the final ranking. Additionally, we present a retrospective evaluation of the scoring system which revealed that: 1) challenge metric was permissive with the false positive predictions; and 2) size-based grouping of instances did not correctly categorize mitochondria of interest. Thus, we propose a new scoring system that better reflects the correctness of the segmentation results. Although several of the top methods are compared favorably to our own baselines, substantial errors remain unsolved for mitochondria with challenging morphologies. Thus, the challenge remains open for submission and automatic evaluation, with all volumes available for download.This work was supported in part by NSF under Award IIS-1835231; in part by NIH under Award 5U54CA225088-03; in part by the University of the Basque Country (UPV/EHU) under Grant GIU19/027; in part by Ministerio de Ciencia, Innovación y Universidades, MCIN/AEI/10.13039/501100011033, under Grant PID2021-126701OB-I00; in part by The Francis Crick Institute which receives its core funding from Cancer Research U.K. under Grant FC001999; in part by the U.K. Medical Research Council under Grant FC001999; in part by the Wellcome Trust under Grant FC001999; and in part by the Frederick National Laboratory for Cancer Research under NIH Contract 75N91019D00024.Peer reviewe