Computational Anatomy for Multi-Organ Analysis in Medical Imaging: A Review

The medical image analysis field has traditionally been focused on the development of organ-, and disease-specific methods. Recently, the interest in the development of more 20 comprehensive computational anatomical models has grown, leading to the creation of multi-organ models. Multi-organ approaches, unlike traditional organ-specific strategies, incorporate inter-organ relations into the model, thus leading to a more accurate representation of the complex human anatomy. Inter-organ relations are not only spatial, but also functional and physiological. Over the years, the strategies 25 proposed to efficiently model multi-organ structures have evolved from the simple global modeling, to more sophisticated approaches such as sequential, hierarchical, or machine learning-based models. In this paper, we present a review of the state of the art on multi-organ analysis and associated computation anatomy methodology. The manuscript follows a methodology-based classification of the different techniques 30 available for the analysis of multi-organs and multi-anatomical structures, from techniques using point distribution models to the most recent deep learning-based approaches. With more than 300 papers included in this review, we reflect on the trends and challenges of the field of computational anatomy, the particularities of each anatomical region, and the potential of multi-organ analysis to increase the impact of 35 medical imaging applications on the future of healthcare.Comment: Paper under revie

Automatic segmentation of the spine by means of a probabilistic atlas with a special focus on ribs suppression

[EN] Purpose: The development of automatic and reliable algorithms for the detection and segmentation of the vertebrae are of great importance prior to any diagnostic task. However, an important problem found to accurately segment the vertebrae is the presence of the ribs in the thoracic region. To overcome this problem, a probabilistic atlas of the spine has been developed dealing with the proximity of other structures, with a special focus on ribs suppression. Methods: The data sets used consist of Computed Tomography images corresponding to 21 patients suffering from spinal metastases. Two methods have been combined to obtain the final result: firstly, an initial segmentation is performed using a fully automatic level-set method; secondly, to refine the initial segmentation, a 3D volume indicating the probability of each voxel of belonging to the spine has been developed. In this way, a probability map is generated and deformed to be adapted to each testing case. Results: To validate the improvement obtained after applying the atlas, the Dice coefficient (DSC), the Hausdorff distance (HD), and the mean surface-to-surface distance (MSD) were used. The results showed up an average of 10 mm of improvement accuracy in terms of HD, obtaining an overall final average of 15.51 2.74 mm. Also, a global value of 91.01 3.18% in terms of DSC and a MSD of 0.66 0.25 mm were obtained. The major improvement using the atlas was achieved in the thoracic region, as ribs were almost perfectly suppressed. Conclusion: The study demonstrated that the atlas is able to detect and appropriately eliminate the ribs while improving the segmentation accuracy.The authors thank the financial support of the Spanish Ministerio de Economia y Competitividad (MINECO) and FEDER funds under Grants TEC2012-33778 and BFU2015-64380-C2-2-R (D.M.) and DPI2013-4572-R (J.D., E.D.)Ruiz-España, S.; Domingo, J.; Díaz-Parra, A.; Dura, E.; D'ocon-Alcaniz, V.; Arana, E.; Moratal, D. (2017). Automatic segmentation of the spine by means of a probabilistic atlas with a special focus on ribs suppression. Medical Physics. 44(9):4695-4707. https://doi.org/10.1002/mp.12431S46954707449Harris, R. I., & Macnab, I. (1954). STRUCTURAL CHANGES IN THE LUMBAR INTERVERTEBRAL DISCS. The Journal of Bone and Joint Surgery. British volume, 36-B(2), 304-322. doi:10.1302/0301-620x.36b2.304Oliveira, M. F. de, Rotta, J. M., & Botelho, R. V. (2015). Survival analysis in patients with metastatic spinal disease: the influence of surgery, histology, clinical and neurologic status. Arquivos de Neuro-Psiquiatria, 73(4), 330-335. doi:10.1590/0004-282x20150003Chou, R. (2011). Diagnostic Imaging for Low Back Pain: Advice for High-Value Health Care From the American College of Physicians. Annals of Internal Medicine, 154(3), 181. doi:10.7326/0003-4819-154-3-201102010-00008Brayda-Bruno, M., Tibiletti, M., Ito, K., Fairbank, J., Galbusera, F., Zerbi, A., … Sivan, S. S. (2013). Advances in the diagnosis of degenerated lumbar discs and their possible clinical application. European Spine Journal, 23(S3), 315-323. doi:10.1007/s00586-013-2960-9Quattrocchi, C. C., Santini, D., Dell’Aia, P., Piciucchi, S., Leoncini, E., Vincenzi, B., … Zobel, B. B. (2007). A prospective analysis of CT density measurements of bone metastases after treatment with zoledronic acid. Skeletal Radiology, 36(12), 1121-1127. doi:10.1007/s00256-007-0388-1Doi, K. (2007). Computer-aided diagnosis in medical imaging: Historical review, current status and future potential. Computerized Medical Imaging and Graphics, 31(4-5), 198-211. doi:10.1016/j.compmedimag.2007.02.002Ruiz-España, S., Arana, E., & Moratal, D. (2015). Semiautomatic computer-aided classification of degenerative lumbar spine disease in magnetic resonance imaging. Computers in Biology and Medicine, 62, 196-205. doi:10.1016/j.compbiomed.2015.04.028Alomari, R. S., Ghosh, S., Koh, J., & Chaudhary, V. (2014). Vertebral Column Localization, Labeling, and Segmentation. Lecture Notes in Computational Vision and Biomechanics, 193-229. doi:10.1007/978-3-319-12508-4_7Hamarneh, G., & Li, X. (2009). Watershed segmentation using prior shape and appearance knowledge. Image and Vision Computing, 27(1-2), 59-68. doi:10.1016/j.imavis.2006.10.009Ghebreab, S., & Smeulders, A. W. (2004). Combining Strings and Necklaces for Interactive Three-Dimensional Segmentation of Spinal Images Using an Integral Deformable Spine Model. IEEE Transactions on Biomedical Engineering, 51(10), 1821-1829. doi:10.1109/tbme.2004.831540Mastmeyer, A., Engelke, K., Fuchs, C., & Kalender, W. A. (2006). A hierarchical 3D segmentation method and the definition of vertebral body coordinate systems for QCT of the lumbar spine. Medical Image Analysis, 10(4), 560-577. doi:10.1016/j.media.2006.05.005Rasoulian, A., Rohling, R., & Abolmaesumi, P. (2013). Lumbar Spine Segmentation Using a Statistical Multi-Vertebrae Anatomical Shape+Pose Model. IEEE Transactions on Medical Imaging, 32(10), 1890-1900. doi:10.1109/tmi.2013.2268424Ma, J., & Lu, L. (2013). Hierarchical segmentation and identification of thoracic vertebra using learning-based edge detection and coarse-to-fine deformable model. Computer Vision and Image Understanding, 117(9), 1072-1083. doi:10.1016/j.cviu.2012.11.016Kim, Y., & Kim, D. (2009). A fully automatic vertebra segmentation method using 3D deformable fences. Computerized Medical Imaging and Graphics, 33(5), 343-352. doi:10.1016/j.compmedimag.2009.02.006Klinder, T., Ostermann, J., Ehm, M., Franz, A., Kneser, R., & Lorenz, C. (2009). Automated model-based vertebra detection, identification, and segmentation in CT images. Medical Image Analysis, 13(3), 471-482. doi:10.1016/j.media.2009.02.004Štern, D., Likar, B., Pernuš, F., & Vrtovec, T. (2011). Parametric modelling and segmentation of vertebral bodies in 3D CT and MR spine images. Physics in Medicine and Biology, 56(23), 7505-7522. doi:10.1088/0031-9155/56/23/011Korez, R., Ibragimov, B., Likar, B., Pernus, F., & Vrtovec, T. (2015). A Framework for Automated Spine and Vertebrae Interpolation-Based Detection and Model-Based Segmentation. IEEE Transactions on Medical Imaging, 34(8), 1649-1662. doi:10.1109/tmi.2015.2389334Castro-Mateos, I., Pozo, J. M., Pereanez, M., Lekadir, K., Lazary, A., & Frangi, A. F. (2015). Statistical Interspace Models (SIMs): Application to Robust 3D Spine Segmentation. IEEE Transactions on Medical Imaging, 34(8), 1663-1675. doi:10.1109/tmi.2015.2443912Pereanez, M., Lekadir, K., Castro-Mateos, I., Pozo, J. M., Lazary, A., & Frangi, A. F. (2015). Accurate Segmentation of Vertebral Bodies and Processes Using Statistical Shape Decomposition and Conditional Models. IEEE Transactions on Medical Imaging, 34(8), 1627-1639. doi:10.1109/tmi.2015.2396774Michael Kelm, B., Wels, M., Kevin Zhou, S., Seifert, S., Suehling, M., Zheng, Y., & Comaniciu, D. (2013). Spine detection in CT and MR using iterated marginal space learning. Medical Image Analysis, 17(8), 1283-1292. doi:10.1016/j.media.2012.09.007Yan Kang, Engelke, K., & Kalender, W. A. (2003). A new accurate and precise 3-D segmentation method for skeletal structures in volumetric CT data. IEEE Transactions on Medical Imaging, 22(5), 586-598. doi:10.1109/tmi.2003.812265Huang, J., Jian, F., Wu, H., & Li, H. (2013). An improved level set method for vertebra CT image segmentation. BioMedical Engineering OnLine, 12(1), 48. doi:10.1186/1475-925x-12-48Lim, P. H., Bagci, U., & Bai, L. (2013). Introducing Willmore Flow Into Level Set Segmentation of Spinal Vertebrae. IEEE Transactions on Biomedical Engineering, 60(1), 115-122. doi:10.1109/tbme.2012.2225833Forsberg, D., Lundström, C., Andersson, M., & Knutsson, H. (2013). Model-based registration for assessment of spinal deformities in idiopathic scoliosis. Physics in Medicine and Biology, 59(2), 311-326. doi:10.1088/0031-9155/59/2/311Yao, J., Burns, J. E., Forsberg, D., Seitel, A., Rasoulian, A., Abolmaesumi, P., … Li, S. (2016). A multi-center milestone study of clinical vertebral CT segmentation. Computerized Medical Imaging and Graphics, 49, 16-28. doi:10.1016/j.compmedimag.2015.12.006Shi, C., Wang, J., & Cheng, Y. (2015). Sparse Representation-Based Deformation Model for Atlas-Based Segmentation of Liver CT Images. Image and Graphics, 410-419. doi:10.1007/978-3-319-21969-1_36Domingo, J., Dura, E., Ayala, G., & Ruiz-España, S. (2015). Means of 2D and 3D Shapes and Their Application in Anatomical Atlas Building. Lecture Notes in Computer Science, 522-533. doi:10.1007/978-3-319-23192-1_44Hyunjin Park, Bland, P. H., & Meyer, C. R. (2003). Construction of an abdominal probabilistic atlas and its application in segmentation. IEEE Transactions on Medical Imaging, 22(4), 483-492. doi:10.1109/tmi.2003.809139Cabezas, M., Oliver, A., Lladó, X., Freixenet, J., & Bach Cuadra, M. (2011). A review of atlas-based segmentation for magnetic resonance brain images. Computer Methods and Programs in Biomedicine, 104(3), e158-e177. doi:10.1016/j.cmpb.2011.07.015Fortunati, V., Verhaart, R. F., van der Lijn, F., Niessen, W. J., Veenland, J. F., Paulides, M. M., & van Walsum, T. (2013). Tissue segmentation of head and neck CT images for treatment planning: A multiatlas approach combined with intensity modeling. Medical Physics, 40(7), 071905. doi:10.1118/1.4810971Zhuang, X., Bai, W., Song, J., Zhan, S., Qian, X., Shi, W., … Rueckert, D. (2015). Multiatlas whole heart segmentation of CT data using conditional entropy for atlas ranking and selection. Medical Physics, 42(7), 3822-3833. doi:10.1118/1.4921366Zhou, J., Yan, Z., Lasio, G., Huang, J., Zhang, B., Sharma, N., … D’Souza, W. (2015). Automated compromised right lung segmentation method using a robust atlas-based active volume model with sparse shape composition prior in CT. Computerized Medical Imaging and Graphics, 46, 47-55. doi:10.1016/j.compmedimag.2015.07.003Linguraru, M. G., Sandberg, J. K., Li, Z., Shah, F., & Summers, R. M. (2010). Automated segmentation and quantification of liver and spleen from CT images using normalized probabilistic atlases and enhancement estimation. Medical Physics, 37(2), 771-783. doi:10.1118/1.3284530Xu, Y., Xu, C., Kuang, X., Wang, H., Chang, E. I.-C., Huang, W., & Fan, Y. (2016). 3D-SIFT-Flow for atlas-based CT liver image segmentation. Medical Physics, 43(5), 2229-2241. doi:10.1118/1.4945021Michopoulou, S. K., Costaridou, L., Panagiotopoulos, E., Speller, R., Panayiotakis, G., & Todd-Pokropek, A. (2009). Atlas-Based Segmentation of Degenerated Lumbar Intervertebral Discs From MR Images of the Spine. IEEE Transactions on Biomedical Engineering, 56(9), 2225-2231. doi:10.1109/tbme.2009.2019765Taso, M., Le Troter, A., Sdika, M., Ranjeva, J.-P., Guye, M., Bernard, M., & Callot, V. (2013). Construction of an in vivo human spinal cord atlas based on high-resolution MR images at cervical and thoracic levels: preliminary results. Magnetic Resonance Materials in Physics, Biology and Medicine, 27(3), 257-267. doi:10.1007/s10334-013-0403-6Lévy, S., Benhamou, M., Naaman, C., Rainville, P., Callot, V., & Cohen-Adad, J. (2015). White matter atlas of the human spinal cord with estimation of partial volume effect. NeuroImage, 119, 262-271. doi:10.1016/j.neuroimage.2015.06.040Hardisty, M., Gordon, L., Agarwal, P., Skrinskas, T., & Whyne, C. (2007). Quantitative characterization of metastatic disease in the spine. Part I. Semiautomated segmentation using atlas-based deformable registration and the level set method. Medical Physics, 34(8), 3127-3134. doi:10.1118/1.2746498Forsberg, D. (2015). Atlas-Based Registration for Accurate Segmentation of Thoracic and Lumbar Vertebrae in CT Data. Lecture Notes in Computational Vision and Biomechanics, 49-59. doi:10.1007/978-3-319-14148-0_5Ibañez MV Schroeder W Cates L Insight software Consortium. The ITK Software Guide 2016 http://www.itk.org/ItkSoftwareGuide.pdfLoader C R package: Local regression, likelihood and density estimation. CRAN repository 2013 2016 http://cran.r-project.org/web/packages/locfitPARK, H., HERO, A., BLAND, P., KESSLER, M., SEO, J., & MEYER, C. (2010). Construction of Abdominal Probabilistic Atlases and Their Value in Segmentation of Normal Organs in Abdominal CT Scans. IEICE Transactions on Information and Systems, E93-D(8), 2291-2301. doi:10.1587/transinf.e93.d.2291Pohl, K. M., Fisher, J., Bouix, S., Shenton, M., McCarley, R. W., Grimson, W. E. L., … Wells, W. M. (2007). Using the logarithm of odds to define a vector space on probabilistic atlases. Medical Image Analysis, 11(5), 465-477. doi:10.1016/j.media.2007.06.003Baddeley, A., & Molchanov, I. (1998). Journal of Mathematical Imaging and Vision, 8(1), 79-92. doi:10.1023/a:1008214317492De Bruijne, M., van Ginneken, B., Viergever, M. A., & Niessen, W. J. (2003). Adapting Active Shape Models for 3D Segmentation of Tubular Structures in Medical Images. Information Processing in Medical Imaging, 136-147. doi:10.1007/978-3-540-45087-0_12Zhang, K., Zhang, L., Song, H., & Zhou, W. (2010). Active contours with selective local or global segmentation: A new formulation and level set method. Image and Vision Computing, 28(4), 668-676. doi:10.1016/j.imavis.2009.10.009Kalpathy-Cramer, J., Awan, M., Bedrick, S., Rasch, C. R. N., Rosenthal, D. I., & Fuller, C. D. (2013). Development of a Software for Quantitative Evaluation Radiotherapy Target and Organ-at-Risk Segmentation Comparison. Journal of Digital Imaging, 27(1), 108-119. doi:10.1007/s10278-013-9633-4Huttenlocher, D. P., Klanderman, G. A., & Rucklidge, W. J. (1993). Comparing images using the Hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(9), 850-863. doi:10.1109/34.232073Aspert, N., Santa-Cruz, D., & Ebrahimi, T. (s. f.). MESH: measuring errors between surfaces using the Hausdorff distance. Proceedings. IEEE International Conference on Multimedia and Expo. doi:10.1109/icme.2002.103587

3D FCN Feature Driven Regression Forest-Based Pancreas Localization and Segmentation

This paper presents a fully automated atlas-based pancreas segmentation method from CT volumes utilizing 3D fully convolutional network (FCN) feature-based pancreas localization. Segmentation of the pancreas is difficult because it has larger inter-patient spatial variations than other organs. Previous pancreas segmentation methods failed to deal with such variations. We propose a fully automated pancreas segmentation method that contains novel localization and segmentation. Since the pancreas neighbors many other organs, its position and size are strongly related to the positions of the surrounding organs. We estimate the position and the size of the pancreas (localized) from global features by regression forests. As global features, we use intensity differences and 3D FCN deep learned features, which include automatically extracted essential features for segmentation. We chose 3D FCN features from a trained 3D U-Net, which is trained to perform multi-organ segmentation. The global features include both the pancreas and surrounding organ information. After localization, a patient-specific probabilistic atlas-based pancreas segmentation is performed. In evaluation results with 146 CT volumes, we achieved 60.6% of the Jaccard index and 73.9% of the Dice overlap.Comment: Presented in MICCAI 2017 workshop, DLMIA 2017 (Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support

The Liver Tumor Segmentation Benchmark (LiTS)

In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LITS) organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2016 and International Conference On Medical Image Computing Computer Assisted Intervention (MICCAI) 2017. Twenty four valid state-of-the-art liver and liver tumor segmentation algorithms were applied to a set of 131 computed tomography (CT) volumes with different types of tumor contrast levels (hyper-/hypo-intense), abnormalities in tissues (metastasectomie) size and varying amount of lesions. The submitted algorithms have been tested on 70 undisclosed volumes. The dataset is created in collaboration with seven hospitals and research institutions and manually reviewed by independent three radiologists. We found that not a single algorithm performed best for liver and tumors. The best liver segmentation algorithm achieved a Dice score of 0.96(MICCAI) whereas for tumor segmentation the best algorithm evaluated at 0.67(ISBI) and 0.70(MICCAI). The LITS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.Comment: conferenc

Automatic Multi-organ Segmentation on Abdominal CT with Dense V-networks

Automatic segmentation of abdominal anatomy on computed tomography (CT) images can support diagnosis, treatment planning and treatment delivery workflows. Segmentation methods using statistical models and multi-atlas label fusion (MALF) require inter-subject image registrations which are challenging for abdominal images, but alternative methods without registration have not yet achieved higher accuracy for most abdominal organs. We present a registration-free deeplearning- based segmentation algorithm for eight organs that are relevant for navigation in endoscopic pancreatic and biliary procedures, including the pancreas, the GI tract (esophagus, stomach, duodenum) and surrounding organs (liver, spleen, left kidney, gallbladder). We directly compared the segmentation accuracy of the proposed method to existing deep learning and MALF methods in a cross-validation on a multi-centre data set with 90 subjects. The proposed method yielded significantly higher Dice scores for all organs and lower mean absolute distances for most organs, including Dice scores of 0.78 vs. 0.71, 0.74 and 0.74 for the pancreas, 0.90 vs 0.85, 0.87 and 0.83 for the stomach and 0.76 vs 0.68, 0.69 and 0.66 for the esophagus. We conclude that deep-learning-based segmentation represents a registration-free method for multi-organ abdominal CT segmentation whose accuracy can surpass current methods, potentially supporting image-guided navigation in gastrointestinal endoscopy procedures