Short-segment heart sound classification using an ensemble of deep convolutional neural networks

This paper proposes a framework based on deep convolutional neural networks (CNNs) for automatic heart sound classification using short-segments of individual heart beats. We design a 1D-CNN that directly learns features from raw heart-sound signals, and a 2D-CNN that takes inputs of two- dimensional time-frequency feature maps based on Mel-frequency cepstral coefficients (MFCC). We further develop a time-frequency CNN ensemble (TF-ECNN) combining the 1D-CNN and 2D-CNN based on score-level fusion of the class probabilities. On the large PhysioNet CinC challenge 2016 database, the proposed CNN models outperformed traditional classifiers based on support vector machine and hidden Markov models with various hand-crafted time- and frequency-domain features. Best classification scores with 89.22% accuracy and 89.94% sensitivity were achieved by the ECNN, and 91.55% specificity and 88.82% modified accuracy by the 2D-CNN alone on the test set.Comment: 8 pages, 1 figure, conferenc

Deep Learning in Cardiology

The medical field is creating large amount of data that physicians are unable to decipher and use efficiently. Moreover, rule-based expert systems are inefficient in solving complicated medical tasks or for creating insights using big data. Deep learning has emerged as a more accurate and effective technology in a wide range of medical problems such as diagnosis, prediction and intervention. Deep learning is a representation learning method that consists of layers that transform the data non-linearly, thus, revealing hierarchical relationships and structures. In this review we survey deep learning application papers that use structured data, signal and imaging modalities from cardiology. We discuss the advantages and limitations of applying deep learning in cardiology that also apply in medicine in general, while proposing certain directions as the most viable for clinical use.Comment: 27 pages, 2 figures, 10 table

An Ensemble of Transfer, Semi-supervised and Supervised Learning Methods for Pathological Heart Sound Classification

In this work, we propose an ensemble of classifiers to distinguish between various degrees of abnormalities of the heart using Phonocardiogram (PCG) signals acquired using digital stethoscopes in a clinical setting, for the INTERSPEECH 2018 Computational Paralinguistics (ComParE) Heart Beats SubChallenge. Our primary classification framework constitutes a convolutional neural network with 1D-CNN time-convolution (tConv) layers, which uses features transferred from a model trained on the 2016 Physionet Heart Sound Database. We also employ a Representation Learning (RL) approach to generate features in an unsupervised manner using Deep Recurrent Autoencoders and use Support Vector Machine (SVM) and Linear Discriminant Analysis (LDA) classifiers. Finally, we utilize an SVM classifier on a high-dimensional segment-level feature extracted using various functionals on short-term acoustic features, i.e., Low-Level Descriptors (LLD). An ensemble of the three different approaches provides a relative improvement of 11.13% compared to our best single sub-system in terms of the Unweighted Average Recall (UAR) performance metric on the evaluation dataset.Comment: 5 pages, 5 figures, Interspeech 2018 accepted manuscrip

A Comprehensive Survey on Heart Sound Analysis in the Deep Learning Era

Heart sound auscultation has been demonstrated to be beneficial in clinical usage for early screening of cardiovascular diseases. Due to the high requirement of well-trained professionals for auscultation, automatic auscultation benefiting from signal processing and machine learning can help auxiliary diagnosis and reduce the burdens of training professional clinicians. Nevertheless, classic machine learning is limited to performance improvement in the era of big data. Deep learning has achieved better performance than classic machine learning in many research fields, as it employs more complex model architectures with stronger capability of extracting effective representations. Deep learning has been successfully applied to heart sound analysis in the past years. As most review works about heart sound analysis were given before 2017, the present survey is the first to work on a comprehensive overview to summarise papers on heart sound analysis with deep learning in the past six years 2017--2022. We introduce both classic machine learning and deep learning for comparison, and further offer insights about the advances and future research directions in deep learning for heart sound analysis

A Robust Interpretable Deep Learning Classifier for Heart Anomaly Detection Without Segmentation

Traditionally, abnormal heart sound classification is framed as a three-stage process. The first stage involves segmenting the phonocardiogram to detect fundamental heart sounds; after which features are extracted and classification is performed. Some researchers in the field argue the segmentation step is an unwanted computational burden, whereas others embrace it as a prior step to feature extraction. When comparing accuracies achieved by studies that have segmented heart sounds before analysis with those who have overlooked that step, the question of whether to segment heart sounds before feature extraction is still open. In this study, we explicitly examine the importance of heart sound segmentation as a prior step for heart sound classification, and then seek to apply the obtained insights to propose a robust classifier for abnormal heart sound detection. Furthermore, recognizing the pressing need for explainable Artificial Intelligence (AI) models in the medical domain, we also unveil hidden representations learned by the classifier using model interpretation techniques. Experimental results demonstrate that the segmentation plays an essential role in abnormal heart sound classification. Our new classifier is also shown to be robust, stable and most importantly, explainable, with an accuracy of almost 100% on the widely used PhysioNet dataset

Recent advances in heart sound analysis

"This is an author-created, un-copyedited versíon of an article published in Physiological Measurement. IOP Publishing Ltd is not responsíble for any errors or omissíons in this versíon of the manuscript or any versíon derived from it. The Versíon of Record is available online at https://doi.org/10.1088/1361-6579/aa7ec8".[EN] Objective: Auscultation of heart sound recordings or the phonocardiogram (PCG) has been shown to be valuable for the detection of disease and pathologies (Leatham 1975, Raghu et al 2015). The automated classification of pathology in heart sounds has been studied for over 50 years. Typical methods can be grouped into: artificial neural network-based approaches (Uguz 2012), support vector machines (Ari et al 2010), hidden Markov model-based approaches (Saracoglu 2012) and clustering-based approaches (Quiceno-Manrique et al 2010). However, accurate automated classification still remains a significant challenge due to the lack of highquality, rigorously validated, and standardized open databases of heart sound recordings. Approach: The 2016 PhysioNet/Computing in Cardiology (CinC) Challenge sought to create a large database to facilitate this, by assembling recordings from multiple research groups across the world, acquired in different real-world clinical and nonclinical environments (such as in-home visits), to encourage the development of algorithms to accurately identify, from a single short recording (10-60s), as normal, abnormal or poor signal quality, and thus to further identify whether the subject of the recording should be referred on for an expert diagnosis (Liu et al 2016). Until this Challenge, no significant open-access heart sound database was available for researchers to train and evaluate the automated diagnostics algorithms upon (Clifford et al 2016). Moreover, no open source heart sound segmentation and classification algorithms were available. The Challenge changed this situation significantly. Main results and Significance: This editorial reviews the follow-up research generated as a result of the Challenge, published in the concurrent special issue of Physiological Measurement. Additionally we make some recommendations for promising research avenues in the field of heart sound signal processing and classification as a result of the Challenge.This work was funded in part by the National Institutes of Health, grant R01-GM104987, the International Postdoctoral Exchange Programme of the National Postdoctoral Management Committee of China and Emory University. We are also grateful to Mathworks for providing free software licenses and sponsoring the Challenge prize money, and Computing in Cardiology for sponsoring the Challenge prize money and providing a forum to present the Challenge results. We would also like to thank the database contributors, and data annotators for their invaluable assistance. Finally, we would like to thank all the competitors and researchers themselves, without whom there would be no Challenge or special issue.Clifford, GD.; Liu, C.; Moody, B.; Millet Roig, J.; Schmidt, S.; Li, Q.; Silva, I.... (2017). Recent advances in heart sound analysis. Physiological Measurement. 38(8):10-25. https://doi.org/10.1088/1361-6579/aa7ec8S1025388Abdollahpur, M., Ghaffari, A., Ghiasi, S., & Mollakazemi, M. J. (2017). Detection of pathological heart sounds. Physiological Measurement, 38(8), 1616-1630. doi:10.1088/1361-6579/aa7840Ari, S., Hembram, K., & Saha, G. (2010). Detection of cardiac abnormality from PCG signal using LMS based least square SVM classifier. Expert Systems with Applications, 37(12), 8019-8026. doi:10.1016/j.eswa.2010.05.088Chauhan, S., Wang, P., Sing Lim, C., & Anantharaman, V. (2008). A computer-aided MFCC-based HMM system for automatic auscultation. Computers in Biology and Medicine, 38(2), 221-233. doi:10.1016/j.compbiomed.2007.10.006Nabhan Homsi, M., & Warrick, P. (2017). Ensemble methods with outliers for phonocardiogram classification. Physiological Measurement, 38(8), 1631-1644. doi:10.1088/1361-6579/aa7982Kay, E., & Agarwal, A. (2017). DropConnected neural networks trained on time-frequency and inter-beat features for classifying heart sounds. Physiological Measurement, 38(8), 1645-1657. doi:10.1088/1361-6579/aa6a3dLangley, P., & Murray, A. (2017). Heart sound classification from unsegmented phonocardiograms. Physiological Measurement, 38(8), 1658-1670. doi:10.1088/1361-6579/aa724cLiu, C., Springer, D., Li, Q., Moody, B., Juan, R. A., Chorro, F. J., … Clifford, G. D. (2016). An open access database for the evaluation of heart sound algorithms. Physiological Measurement, 37(12), 2181-2213. doi:10.1088/0967-3334/37/12/2181Maknickas, V., & Maknickas, A. (2017). Recognition of normal–abnormal phonocardiographic signals using deep convolutional neural networks and mel-frequency spectral coefficients. Physiological Measurement, 38(8), 1671-1684. doi:10.1088/1361-6579/aa7841Plesinger, F., Viscor, I., Halamek, J., Jurco, J., & Jurak, P. (2017). Heart sounds analysis using probability assessment. Physiological Measurement, 38(8), 1685-1700. doi:10.1088/1361-6579/aa7620Da Poian, G., Liu, C., Bernardini, R., Rinaldo, R., & Clifford, G. D. (2017). Atrial fibrillation detection on compressed sensed ECG. Physiological Measurement, 38(7), 1405-1425. doi:10.1088/1361-6579/aa7652Quiceno-Manrique, A. F., Godino-Llorente, J. I., Blanco-Velasco, M., & Castellanos-Dominguez, G. (2009). Selection of Dynamic Features Based on Time–Frequency Representations for Heart Murmur Detection from Phonocardiographic Signals. Annals of Biomedical Engineering, 38(1), 118-137. doi:10.1007/s10439-009-9838-3Jull, J., Giles, A., Boyer, Y., & Stacey, D. (2015). Cultural adaptation of a shared decision making tool with Aboriginal women: a qualitative study. BMC Medical Informatics and Decision Making, 15(1). doi:10.1186/s12911-015-0129-7Saraçoğlu, R. (2012). Hidden Markov model-based classification of heart valve disease with PCA for dimension reduction. Engineering Applications of Artificial Intelligence, 25(7), 1523-1528. doi:10.1016/j.engappai.2012.07.005Schmidt, S. E., Holst-Hansen, C., Graff, C., Toft, E., & Struijk, J. J. (2010). Segmentation of heart sound recordings by a duration-dependent hidden Markov model. Physiological Measurement, 31(4), 513-529. doi:10.1088/0967-3334/31/4/004Springer, D. B., Brennan, T., Ntusi, N., Abdelrahman, H. Y., Zühlke, L. J., Mayosi, B. M., … Clifford, G. D. (2016). Automated signal quality assessment of mobile phone-recorded heart sound signals. Journal of Medical Engineering & Technology, 40(7-8), 342-355. doi:10.1080/03091902.2016.1213902Springer, D., Tarassenko, L., & Clifford, G. (2015). Logistic Regression-HSMM-based Heart Sound Segmentation. IEEE Transactions on Biomedical Engineering, 1-1. doi:10.1109/tbme.2015.2475278Uğuz, H. (2010). A Biomedical System Based on Artificial Neural Network and Principal Component Analysis for Diagnosis of the Heart Valve Diseases. Journal of Medical Systems, 36(1), 61-72. doi:10.1007/s10916-010-9446-7Whitaker, B. M., Suresha, P. B., Liu, C., Clifford, G. D., & Anderson, D. V. (2017). Combining sparse coding and time-domain features for heart sound classification. Physiological Measurement, 38(8), 1701-1713. doi:10.1088/1361-6579/aa7623Zhu, T., Dunkley, N., Behar, J., Clifton, D. A., & Clifford, G. D. (2015). Fusing Continuous-Valued Medical Labels Using a Bayesian Model. Annals of Biomedical Engineering, 43(12), 2892-2902. doi:10.1007/s10439-015-1344-1Zhu, T., Johnson, A. E. W., Behar, J., & Clifford, G. D. (2013). Crowd-Sourced Annotation of ECG Signals Using Contextual Information. Annals of Biomedical Engineering, 42(4), 871-884. doi:10.1007/s10439-013-0964-