Why rankings of biomedical image analysis competitions should be interpreted with care

International challenges have become the standard for validation of biomedical image analysis methods. Given their scientific impact, it is surprising that a critical analysis of common practices related to the organization of challenges has not yet been performed. In this paper, we present a comprehensive analysis of biomedical image analysis challenges conducted up to now. We demonstrate the importance of challenges and show that the lack of quality control has critical consequences. First, reproducibility and interpretation of the results is often hampered as only a fraction of relevant information is typically provided. Second, the rank of an algorithm is generally not robust to a number of variables such as the test data used for validation, the ranking scheme applied and the observers that make the reference annotations. To overcome these problems, we recommend best practice guidelines and define open research questions to be addressed in the future

Machine Learning and Artificial Intelligence in Cardiovascular Imaging

Artificial intelligence (AI) has captured the minds of science fiction writers and the general public for quite some time. As advancements have been made in computer science and engineering research, much improved computational power and the creation of newer, more efficient algorithms such as machine learning (ML) and deep learning (DL) have enabled the feasibility of big data analysis. AI has moved from the realm of science fiction to applications used in everyday life, such as Tesla’s self-driving cars, Facebook’s facial recognition, Amazon’s product recommendations, mobile check deposits, language translation software, and more. As AI continues to improve, ML algorithms can now master tasks that were previously thought to be too complex for machines and are now even capable of detecting patterns that are beyond human perception. This has led to a renewed and increased interest in ML as a useful tool in medical practice, particularly in the field of medical imaging. Indeed, now more than ever, medicine has become a big data science, with the introduction of electronic medical records (EMR) leading to a substantial amount of patient information being recorded. This available information will only increase in the future through the use of bidirectional patient portals. Moreover, in the era of evidence-based medicine, thousands of new evidence and data are being published daily. Going through such large volumes of data to determine what is clinically relevant and actionable can be overwhelming, resulting in important information being missed by physicians. However, AI machines can now consis- tently perform repetitive tasks at maximum capacity, sometimes producing results faster and more efficiently than humans. Medicine is thereby a perfect testing ground for the application of ML, as these systems can augment the ability of physicians to identify key information required for patient management while presenting it in an understandable manner. In particular, because radiology directly involves extracting data consisting of specific features seen on images and interpreting them through the knowledge base acquired by the radiologist, the medical imag- ing field serves as an attractive arena for the incorporation of ML systems. As advanced AI and ML systems transition from fiction to reality and steadily approach their implementation into med- ical and radiology practices, understanding the general meth- ods, capabilities, and limitations of machine learning is of fundamental importance to physicians and radiologists for the effective use of these systems. This chapter will introduce some of the basic concepts of machine learning techniques, provide a basic framework for their use, and highlight current and future applications in medicine and radiology with a special focus on cardiovascular imaging