Robust Resolution-Enhanced Prostate Segmentation in Magnetic Resonance and Ultrasound Images through Convolutional Neural Networks

[EN] Prostate segmentations are required for an ever-increasing number of medical applications, such as image-based lesion detection, fusion-guided biopsy and focal therapies. However, obtaining accurate segmentations is laborious, requires expertise and, even then, the inter-observer variability remains high. In this paper, a robust, accurate and generalizable model for Magnetic Resonance (MR) and three-dimensional (3D) Ultrasound (US) prostate image segmentation is proposed. It uses a densenet-resnet-based Convolutional Neural Network (CNN) combined with techniques such as deep supervision, checkpoint ensembling and Neural Resolution Enhancement. The MR prostate segmentation model was trained with five challenging and heterogeneous MR prostate datasets (and two US datasets), with segmentations from many different experts with varying segmentation criteria. The model achieves a consistently strong performance in all datasets independently (mean Dice Similarity Coefficient -DSC- above 0.91 for all datasets except for one), outperforming the inter-expert variability significantly in MR (mean DSC of 0.9099 vs. 0.8794). When evaluated on the publicly available Promise12 challenge dataset, it attains a similar performance to the best entries. In summary, the model has the potential of having a significant impact on current prostate procedures, undercutting, and even eliminating, the need of manual segmentations through improvements in terms of robustness, generalizability and output resolutionThis work has been partially supported by a doctoral grant of the Spanish Ministry of Innovation and Science, with reference FPU17/01993Pellicer-Valero, OJ.; González-Pérez, V.; Casanova Ramón-Borja, JL.; Martín García, I.; Barrios Benito, M.; Pelechano Gómez, P.; Rubio-Briones, J.... (2021). Robust Resolution-Enhanced Prostate Segmentation in Magnetic Resonance and Ultrasound Images through Convolutional Neural Networks. Applied Sciences. 11(2):1-17. https://doi.org/10.3390/app11020844S117112Marra, G., Ploussard, G., Futterer, J., & Valerio, M. (2019). Controversies in MR targeted biopsy: alone or combined, cognitive versus software-based fusion, transrectal versus transperineal approach? World Journal of Urology, 37(2), 277-287. doi:10.1007/s00345-018-02622-5Ahdoot, M., Lebastchi, A. H., Turkbey, B., Wood, B., & Pinto, P. A. (2019). Contemporary treatments in prostate cancer focal therapy. Current Opinion in Oncology, 31(3), 200-206. doi:10.1097/cco.0000000000000515Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90. doi:10.1145/3065386Allen, P. D., Graham, J., Williamson, D. C., & Hutchinson, C. E. (s. f.). Differential Segmentation of the Prostate in MR Images Using Combined 3D Shape Modelling and Voxel Classification. 3rd IEEE International Symposium on Biomedical Imaging: Macro to Nano, 2006. doi:10.1109/isbi.2006.1624940Freedman, D., Radke, R. J., Tao Zhang, Yongwon Jeong, Lovelock, D. M., & Chen, G. T. Y. (2005). Model-based segmentation of medical imagery by matching distributions. IEEE Transactions on Medical Imaging, 24(3), 281-292. doi:10.1109/tmi.2004.841228Klein, S., van der Heide, U. A., Lips, I. M., van Vulpen, M., Staring, M., & Pluim, J. P. W. (2008). Automatic segmentation of the prostate in 3D MR images by atlas matching using localized mutual information. Medical Physics, 35(4), 1407-1417. doi:10.1118/1.2842076Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, 234-241. doi:10.1007/978-3-319-24574-4_28He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017). Mask R-CNN. 2017 IEEE International Conference on Computer Vision (ICCV). doi:10.1109/iccv.2017.322Shelhamer, E., Long, J., & Darrell, T. (2017). Fully Convolutional Networks for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 640-651. doi:10.1109/tpami.2016.2572683He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr.2016.90Milletari, F., Navab, N., & Ahmadi, S.-A. (2016). V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. 2016 Fourth International Conference on 3D Vision (3DV). doi:10.1109/3dv.2016.79Zhu, Q., Du, B., Turkbey, B., Choyke, P. L., & Yan, P. (2017). Deeply-supervised CNN for prostate segmentation. 2017 International Joint Conference on Neural Networks (IJCNN). doi:10.1109/ijcnn.2017.7965852To, M. N. N., Vu, D. Q., Turkbey, B., Choyke, P. L., & Kwak, J. T. (2018). Deep dense multi-path neural network for prostate segmentation in magnetic resonance imaging. International Journal of Computer Assisted Radiology and Surgery, 13(11), 1687-1696. doi:10.1007/s11548-018-1841-4Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). Densely Connected Convolutional Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr.2017.243Zhu, Y., Wei, R., Gao, G., Ding, L., Zhang, X., Wang, X., & Zhang, J. (2018). Fully automatic segmentation on prostate MR images based on cascaded fully convolution network. Journal of Magnetic Resonance Imaging, 49(4), 1149-1156. doi:10.1002/jmri.26337Wang, Y., Ni, D., Dou, H., Hu, X., Zhu, L., Yang, X., … Wang, T. (2019). Deep Attentive Features for Prostate Segmentation in 3D Transrectal Ultrasound. IEEE Transactions on Medical Imaging, 38(12), 2768-2778. doi:10.1109/tmi.2019.2913184Lemaître, G., Martí, R., Freixenet, J., Vilanova, J. C., Walker, P. M., & Meriaudeau, F. (2015). Computer-Aided Detection and diagnosis for prostate cancer based on mono and multi-parametric MRI: A review. Computers in Biology and Medicine, 60, 8-31. doi:10.1016/j.compbiomed.2015.02.009Litjens, G., Toth, R., van de Ven, W., Hoeks, C., Kerkstra, S., van Ginneken, B., … Madabhushi, A. (2014). Evaluation of prostate segmentation algorithms for MRI: The PROMISE12 challenge. Medical Image Analysis, 18(2), 359-373. doi:10.1016/j.media.2013.12.002Zhu, Q., Du, B., & Yan, P. (2020). Boundary-Weighted Domain Adaptive Neural Network for Prostate MR Image Segmentation. IEEE Transactions on Medical Imaging, 39(3), 753-763. doi:10.1109/tmi.2019.2935018He, K., Zhang, X., Ren, S., & Sun, J. (2015). Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. 2015 IEEE International Conference on Computer Vision (ICCV). doi:10.1109/iccv.2015.123Pan, S. J., & Yang, Q. (2010). A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345-1359. doi:10.1109/tkde.2009.191Smith, L. N. (2017). Cyclical Learning Rates for Training Neural Networks. 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). doi:10.1109/wacv.2017.58Abraham, N., & Khan, N. M. (2019). A Novel Focal Tversky Loss Function With Improved Attention U-Net for Lesion Segmentation. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). doi:10.1109/isbi.2019.8759329Lei, Y., Tian, S., He, X., Wang, T., Wang, B., Patel, P., … Yang, X. (2019). Ultrasound prostate segmentation based on multidirectional deeply supervised V‐Net. Medical Physics, 46(7), 3194-3206. doi:10.1002/mp.13577Orlando, N., Gillies, D. J., Gyacskov, I., Romagnoli, C., D’Souza, D., & Fenster, A. (2020). Automatic prostate segmentation using deep learning on clinically diverse 3D transrectal ultrasound images. Medical Physics, 47(6), 2413-2426. doi:10.1002/mp.14134Karimi, D., Zeng, Q., Mathur, P., Avinash, A., Mahdavi, S., Spadinger, I., … Salcudean, S. E. (2019). Accurate and robust deep learning-based segmentation of the prostate clinical target volume in ultrasound images. Medical Image Analysis, 57, 186-196. doi:10.1016/j.media.2019.07.005PROMISE12 Resultshttps://promise12.grand-challenge.org/Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J., & Maier-Hein, K. H. (2020). nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods, 18(2), 203-211. doi:10.1038/s41592-020-01008-

Experimental Assessment of Time Reversal for In-Body to In-Body UWB Communications

[EN] The standard of in-body communications is limited to the use of narrowband systems. These systems are far from the high data rate connections achieved by other wireless telecommunication services today in force. The UWB frequency band has been proposed as a possible candidate for future in-body networks. However, the attenuation of body tissues at gigahertz frequencies could be a serious drawback. Experimental measurements for channel modeling are not easy to carry out, while the use of humans is practically forbidden. Sophisticated simulation tools could provide inaccurate results since they are not able to reproduce all the in-body channel conditions. Chemical solutions known as phantoms could provide a fair approximation of body tissues¿ behavior. In this work, the Time Reversal technique is assessed to increase the channel performance of in-body communications. For this task, a large volume of experimental measurements is performed at the low part of UWB spectrum (3.1-5.1 GHz) by using a highly accurate phantom-based measurement setup. This experimental setup emulates an in-body to in-body scenario, where all the nodes are implanted inside the body. Moreover, the in-body channel characteristics such as the path loss, the correlation in transmission and reception, and the reciprocity of the channel are assessed and discussed.This work was supported by the Programa de Ayudas de Investigacion y Desarrollo (PAID-01-16) from Universitat Politecnica de Valencia and by the Ministerio de Economia y Competitividad, Spain (TEC2014-60258-C2-1-R), by the European FEDER funds.Andreu-Estellés, C.; Garcia-Pardo, C.; Castelló-Palacios, S.; Cardona Marcet, N. (2018). Experimental Assessment of Time Reversal for In-Body to In-Body UWB Communications. Wireless Communications and Mobile Computing (Online). (8927107):1-12. https://doi.org/10.1155/2018/8927107S1128927107Fireman, Z. (2003). Diagnosing small bowel Crohn’s disease with wireless capsule endoscopy. Gut, 52(3), 390-392. doi:10.1136/gut.52.3.390Burri, H., & Senouf, D. (2009). Remote monitoring and follow-up of pacemakers and implantable cardioverter defibrillators. Europace, 11(6), 701-709. doi:10.1093/europace/eup110Scanlon, W. G., Burns, B., & Evans, N. E. (2000). Radiowave propagation from a tissue-implanted source at 418 MHz and 916.5 MHz. IEEE Transactions on Biomedical Engineering, 47(4), 527-534. doi:10.1109/10.828152Chavez-Santiago, R., Garcia-Pardo, C., Fornes-Leal, A., Valles-Lluch, A., Vermeeren, G., Joseph, W., … Cardona, N. (2015). Experimental Path Loss Models for In-Body Communications within 2.36-2.5 GHz. IEEE Journal of Biomedical and Health Informatics, 1-1. doi:10.1109/jbhi.2015.2418757Khaleghi, A., Chávez-Santiago, R., & Balasingham, I. (2010). Ultra-wideband pulse-based data communications for medical implants. IET Communications, 4(15), 1889. doi:10.1049/iet-com.2009.0692Khaleghi, A., Chávez-Santiago, R., & Balasingham, I. (2011). Ultra-wideband statistical propagation channel model for implant sensors in the human chest. IET Microwaves, Antennas & Propagation, 5(15), 1805. doi:10.1049/iet-map.2010.0537Kurup, D., Scarpello, M., Vermeeren, G., Joseph, W., Dhaenens, K., Axisa, F., … Vanfleteren, J. (2011). In-body path loss models for implants in heterogeneous human tissues using implantable slot dipole conformal flexible antennas. EURASIP Journal on Wireless Communications and Networking, 2011(1). doi:10.1186/1687-1499-2011-51Floor, P. A., Chavez-Santiago, R., Brovoll, S., Aardal, O., Bergsland, J., Grymyr, O.-J. H. N., … Balasingham, I. (2015). In-Body to On-Body Ultrawideband Propagation Model Derived From Measurements in Living Animals. IEEE Journal of Biomedical and Health Informatics, 19(3), 938-948. doi:10.1109/jbhi.2015.2417805Shimizu, Y., Anzai, D., Chavez-Santiago, R., Floor, P. A., Balasingham, I., & Wang, J. (2017). Performance Evaluation of an Ultra-Wideband Transmit Diversity in a Living Animal Experiment. IEEE Transactions on Microwave Theory and Techniques, 65(7), 2596-2606. doi:10.1109/tmtt.2017.2669039Anzai, D., Katsu, K., Chavez-Santiago, R., Wang, Q., Plettemeier, D., Wang, J., & Balasingham, I. (2014). Experimental Evaluation of Implant UWB-IR Transmission With Living Animal for Body Area Networks. IEEE Transactions on Microwave Theory and Techniques, 62(1), 183-192. doi:10.1109/tmtt.2013.2291542Chou, C.-K., Chen, G.-W., Guy, A. W., & Luk, K. H. (1984). Formulas for preparing phantom muscle tissue at various radiofrequencies. Bioelectromagnetics, 5(4), 435-441. doi:10.1002/bem.2250050408Cheung, A. Y., & Koopman, D. W. (1976). Experimental Development of Simulated Biomaterials for Dosimetry Studies of Hazardous Microwave Radiation (Short Papers). IEEE Transactions on Microwave Theory and Techniques, 24(10), 669-673. doi:10.1109/tmtt.1976.1128936YAMAMOTO, H., ZHOU, J., & KOBAYASHI, T. (2008). Ultra Wideband Electromagnetic Phantoms for Antennas and Propagation Studies. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, E91-A(11), 3173-3182. doi:10.1093/ietfec/e91-a.11.3173Lazebnik, M., Madsen, E. L., Frank, G. R., & Hagness, S. C. (2005). Tissue-mimicking phantom materials for narrowband and ultrawideband microwave applications. Physics in Medicine and Biology, 50(18), 4245-4258. doi:10.1088/0031-9155/50/18/001Yilmaz, T., Foster, R., & Hao, Y. (2014). Broadband Tissue Mimicking Phantoms and a Patch Resonator for Evaluating Noninvasive Monitoring of Blood Glucose Levels. IEEE Transactions on Antennas and Propagation, 62(6), 3064-3075. doi:10.1109/tap.2014.2313139Gezici, S., Zhi Tian, Giannakis, G. B., Kobayashi, H., Molisch, A. F., Poor, H. V., & Sahinoglu, Z. (2005). Localization via ultra-wideband radios: a look at positioning aspects for future sensor networks. IEEE Signal Processing Magazine, 22(4), 70-84. doi:10.1109/msp.2005.1458289Marinova, M., Thielens, A., Tanghe, E., Vallozzi, L., Vermeeren, G., Joseph, W., … Martens, L. (2015). Diversity Performance of Off-Body MB-OFDM UWB-MIMO. IEEE Transactions on Antennas and Propagation, 63(7), 3187-3197. doi:10.1109/tap.2015.2422353SHI, J., ANZAI, D., & WANG, J. (2012). Channel Modeling and Performance Analysis of Diversity Reception for Implant UWB Wireless Link. IEICE Transactions on Communications, E95.B(10), 3197-3205. doi:10.1587/transcom.e95.b.3197Pajusco, P., & Pagani, P. (2009). On the Use of Uniform Circular Arrays for Characterizing UWB Time Reversal. IEEE Transactions on Antennas and Propagation, 57(1), 102-109. doi:10.1109/tap.2008.2009715Chavez-Santiago, R., Sayrafian-Pour, K., Khaleghi, A., Takizawa, K., Wang, J., Balasingham, I., & Li, H.-B. (2013). Propagation models for IEEE 802.15.6 standardization of implant communication in body area networks. IEEE Communications Magazine, 51(8), 80-87. doi:10.1109/mcom.2013.6576343Andreu, C., Castello-Palacios, S., Garcia-Pardo, C., Fornes-Leal, A., Valles-Lluch, A., & Cardona, N. (2016). Spatial In-Body Channel Characterization Using an Accurate UWB Phantom. IEEE Transactions on Microwave Theory and Techniques, 64(11), 3995-4002. doi:10.1109/tmtt.2016.2609409Pahlavan, K., & Levesque, A. H. (2005). Wireless Information Networks. doi:10.1002/0471738646Qiu, R. C., Zhou, C., Guo, N., & Zhang, J. Q. (2006). Time Reversal With MISO for Ultrawideband Communications: Experimental Results. IEEE Antennas and Wireless Propagation Letters, 5, 269-273. doi:10.1109/lawp.2006.875888Ando, H., Takizawa, K., Yoshida, T., Matsushita, K., Hirata, M., & Suzuki, T. (2016). Wireless Multichannel Neural Recording With a 128-Mbps UWB Transmitter for an Implantable Brain-Machine Interfaces. IEEE Transactions on Biomedical Circuits and Systems, 10(6), 1068-1078. doi:10.1109/tbcas.2016.251452

Deep Burst Denoising

Noise is an inherent issue of low-light image capture, one which is exacerbated on mobile devices due to their narrow apertures and small sensors. One strategy for mitigating noise in a low-light situation is to increase the shutter time of the camera, thus allowing each photosite to integrate more light and decrease noise variance. However, there are two downsides of long exposures: (a) bright regions can exceed the sensor range, and (b) camera and scene motion will result in blurred images. Another way of gathering more light is to capture multiple short (thus noisy) frames in a "burst" and intelligently integrate the content, thus avoiding the above downsides. In this paper, we use the burst-capture strategy and implement the intelligent integration via a recurrent fully convolutional deep neural net (CNN). We build our novel, multiframe architecture to be a simple addition to any single frame denoising model, and design to handle an arbitrary number of noisy input frames. We show that it achieves state of the art denoising results on our burst dataset, improving on the best published multi-frame techniques, such as VBM4D and FlexISP. Finally, we explore other applications of image enhancement by integrating content from multiple frames and demonstrate that our DNN architecture generalizes well to image super-resolution

Incremental construction of LSTM recurrent neural network

Long Short--Term Memory (LSTM) is a recurrent neural network that uses structures called memory blocks to allow the net remember significant events distant in the past input sequence in order to solve long time lag tasks, where other RNN approaches fail. Throughout this work we have performed experiments using LSTM networks extended with growing abilities, which we call GLSTM. Four methods of training growing LSTM has been compared. These methods include cascade and fully connected hidden layers as well as two different levels of freezing previous weights in the cascade case. GLSTM has been applied to a forecasting problem in a biomedical domain, where the input/output behavior of five controllers of the Central Nervous System control has to be modelled. We have compared growing LSTM results against other neural networks approaches, and our work applying conventional LSTM to the task at hand.Postprint (published version

Deep Learning in Cardiology

The medical field is creating large amount of data that physicians are unable to decipher and use efficiently. Moreover, rule-based expert systems are inefficient in solving complicated medical tasks or for creating insights using big data. Deep learning has emerged as a more accurate and effective technology in a wide range of medical problems such as diagnosis, prediction and intervention. Deep learning is a representation learning method that consists of layers that transform the data non-linearly, thus, revealing hierarchical relationships and structures. In this review we survey deep learning application papers that use structured data, signal and imaging modalities from cardiology. We discuss the advantages and limitations of applying deep learning in cardiology that also apply in medicine in general, while proposing certain directions as the most viable for clinical use.Comment: 27 pages, 2 figures, 10 table