Patient clustering for vital organ failure using ICD code with graph attention

Objective: Heart failure, respiratory failure and kidney failure are three severe organ failures (OF) that have high mortalities and are most prevalent in intensive care units. The objective of this work is to offer insights into OF clustering from the aspects of graph neural networks and diagnosis history. Methods: This paper proposes a neural network-based pipeline to cluster three types of organ failure patients by incorporating embedding pre-train using an ontology graph of the International Classification of Diseases (ICD) codes. We employ an autoencoder-based deep clustering architecture jointly trained with a K-means loss, and a non-linear dimension reduction is performed to obtain patient clusters on the MIMIC-III dataset. Results: The clustering pipeline shows superior performance on a public-domain image dataset. On the MIMIC-III dataset, it discovers two distinct clusters that exhibit different comorbidity spectra which can be related to the severity of diseases. The proposed pipeline is compared with several other clustering models and shows superiority. Conclusion: Our proposed pipeline gives stable clusters, however, they do not correspond to the type of OF which indicates these OF share significant hidden characteristics in diagnosis. These clusters can be used to signal possible complications and severity of illness and aid personalised treatment. Significance: We are the first to apply an unsupervised approach to offer insights from a biomedical engineering perspective on these three types of organ failure, and publish the pre-trained embeddings for future transfer learning

Gradient-Based Competitive Learning: Theory

Deep learning has been recently used to extract the relevant features for representing input data also in the unsupervised setting. However, state-of-the-art techniques focus mostly on algorithmic efficiency and accuracy rather than mimicking the input manifold. On the contrary, competitive learning is a powerful tool for replicating the input distribution topology. It is cognitive/biologically inspired as it is founded on Hebbian learning, a neuropsychological theory claiming that neurons can increase their specialization by competing for the right to respond to/represent a subset of the input data. This paper introduces a novel perspective by combining these two techniques: unsupervised gradient-based and competitive learning. The theory is based on the intuition that neural networks can learn topological structures by working directly on the transpose of the input matrix. At this purpose, the vanilla competitive layer and its dual are presented. The former is representative of a standard competitive layer for deep clustering, while the latter is trained on the transposed matrix. The equivalence of the layers is extensively proven both theoretically and experimentally. The dual competitive layer has better properties. Unlike the vanilla layer, it directly outputs the prototypes of the data inputs, while still allowing learning by backpropagation. More importantly, this paper proves theoretically that the dual layer is better suited for handling high-dimensional data (e.g., for biological applications), because the estimation of the weights is driven by a constraining subspace which does not depend on the input dimensionality, but only on the dataset cardinality. This paper has introduced a novel approach for unsupervised gradient-based competitive learning. This approach is very promising both in the case of small datasets of high-dimensional data and for better exploiting the advantages of a deep architecture: the dual layer perfectly integrates with the deep layers. A theoretical justification is also given by using the analysis of the gradient flow for both vanilla and dual layers