Urinary peptide-based classifier CKD273: towards clinical application in chronic kidney disease

Capillary electrophoresis coupled with mass spectrometry (CE-MS) has been used as a platform for discovery and validation of urinary peptides associated with chronic kidney disease (CKD). CKD affects ∼ 10% of the population, with high associated costs for treatments. A urinary proteome-based classifier (CKD273) has been discovered and validated in cross-sectional and longitudinal studies to assess and predict the progression of CKD. It has been implemented in studies employing cohorts of > 1000 patients. CKD273 is commercially available as an in vitro diagnostic test for early detection of CKD and is currently being used for patient stratification in a multicentre randomized clinical trial (PRIORITY). The validity of the CKD273 classifier has recently been evaluated applying the Oxford Evidence-Based Medicine and Southampton Oxford Retrieval Team guidelines and a letter of support for CKD273 was issued by the US Food and Drug Administration. In this article we review the current evidence published on CKD273 and the challenges associated with implementation. Definition of a possible surrogate early endpoint combined with CKD273 as a biomarker for patient stratification currently appears as the most promising strategy to enable the development of effective drugs to be used at an early time point when intervention can still be effective

Mining health knowledge graph for health risk prediction

Nowadays classification models have been widely adopted in healthcare, aiming at supporting practitioners for disease diagnosis and human error reduction. The challenge is utilising effective methods to mine real-world data in the medical domain, as many different models have been proposed with varying results. A large number of researchers focus on the diversity problem of real-time data sets in classification models. Some previous works developed methods comprising of homogeneous graphs for knowledge representation and then knowledge discovery. However, such approaches are weak in discovering different relationships among elements. In this paper, we propose an innovative classification model for knowledge discovery from patients’ personal health repositories. The model discovers medical domain knowledge from the massive data in the National Health and Nutrition Examination Survey (NHANES). The knowledge is conceptualised in a heterogeneous knowledge graph. On the basis of the model, an innovative method is developed to help uncover potential diseases suffered by people and, furthermore, to classify patients’ health risk. The proposed model is evaluated by comparison to a baseline model also built on the NHANES data set in an empirical experiment. The performance of proposed model is promising. The paper makes significant contributions to the advancement of knowledge in data mining with an innovative classification model specifically crafted for domain-based data. In addition, by accessing the patterns of various observations, the research contributes to the work of practitioners by providing a multifaceted understanding of individual and public health

Threshold-free Evaluation of Medical Tests for Classification and Prediction: Average Precision versus Area Under the ROC Curve

When evaluating medical tests or biomarkers for disease classification, the area under the receiver-operating characteristic (ROC) curve is a widely used performance metric that does not require us to commit to a specific decision threshold. For the same type of evaluations, a different metric known as the average precision (AP) is used much more widely in the information retrieval literature. We study both metrics in some depths in order to elucidate their difference and relationship. More specifically, we explain mathematically why the AP may be more appropriate if the earlier part of the ROC curve is of interest. We also address practical matters, deriving an expression for the asymptotic variance of the AP, as well as providing real-world examples concerning the evaluation of protein biomarkers for prostate cancer and the assessment of digital versus film mammography for breast cancer screening.Comment: The first two authors contributed equally to this paper, and should be regarded as co-first author

Doctor of Philosophy

dissertationDisease-specific ontologies, designed to structure and represent the medical knowledge about disease etiology, diagnosis, treatment, and prognosis, are essential for many advanced applications, such as predictive modeling, cohort identification, and clinical decision support. However, manually building disease-specific ontologies is very labor-intensive, especially in the process of knowledge acquisition. On the other hand, medical knowledge has been documented in a variety of biomedical knowledge resources, such as textbook, clinical guidelines, research articles, and clinical data repositories, which offers a great opportunity for an automated knowledge acquisition. In this dissertation, we aim to facilitate the large-scale development of disease-specific ontologies through automated extraction of disease-specific vocabularies from existing biomedical knowledge resources. Three separate studies presented in this dissertation explored both manual and automated vocabulary extraction. The first study addresses the question of whether disease-specific reference vocabularies derived from manual concept acquisition can achieve a near-saturated coverage (or near the greatest possible amount of disease-pertinent concepts) by using a small number of literature sources. Using a general-purpose, manual acquisition approach we developed, this study concludes that a small number of expert-curated biomedical literature resources can prove sufficient for acquiring near-saturated disease-specific vocabularies. The second and third studies introduce automated techniques for extracting disease-specific vocabularies from both MEDLINE citations (title and abstract) and a clinical data repository. In the second study, we developed and assessed a pipeline-based system which extracts disease-specific treatments from PubMed citations. The system has achieved a mean precision of 0.8 for the top 100 extracted treatment concepts. In the third study, we applied classification models to reduce irrelevant disease-concepts associations extracted from MEDLINE citations and electronic medical records. This study suggested the combination of measures of relevance from disparate sources to improve the identification of true-relevant concepts through classification and also demonstrated the generalizability of the studied classification model to new diseases. With the studies, we concluded that existing biomedical knowledge resources are valuable sources for extracting disease-concept associations, from which classification based on statistical measures of relevance could assist a semi-automated generation of disease-specific vocabularies

Evaluation, Validation & Implementation of a Computerized Diagnostic Decision Support System in Primary Practice

Background: Medical diagnosis may be the most complex task attempted by humans. Studies estimate that 95% of diagnoses in outpatient care are accurate, implying that the annual rate of inaccurate diagnoses is 12 million in the US alone, with the potential for patient harm in about half. A well-researched differential might reduce inaccurate diagnoses by offering alternatives matching the patient’s symptoms. This study searched the literature for articles evaluating the diagnostic performance of commercially available computerized diagnostic decision support systems. This search led to selecting Isabel Pro, developed by Isabel Healthcare, Ltd. of Haslemere, UK. Evaluation and Validation: A computerized diagnostic decision support system should respond adequately to four questions: What is the “diagnostic retrieval accuracy”? Does it perform as well as clinicians? When provided with the differential, do clinicians improve diagnostic accuracy? Is it easily incorporated into routine practice? The project validated the diagnostic retrieval accuracy of Isabel Pro using 46 cases with a previously confirmed diagnosis. The confirmed diagnosis appeared in Isabel Pro’s differential in 24 cases (52.2%), outperforming even internal medicine faculty (47%). Using those 24 cases and the differentials produced, the author conducted a diagnostic challenge that involved 120 McGovern Medical School residents. The residents produced 406 diagnoses, of which 105 (25.9%) were correct without the differentials, and 37 were correct post-consultation, a 9.1% absolute improvement. In responses, 75.1% of the participants agreed the differentials would be helpful in routine practice, and 64.1% agreed they would consult the differentials if available. Implementation: The project successfully proposed Isabel Pro as a solution to UT practice leadership on September 16, 2021, and incorporated the system into the Epic EHR as a menu line link on November 30, 2021. This system-wide integration also included a QR code for downloading Isabel Pro to a mobile device. Usage of Isabel Pro in the practices of UTPhysicians began on December 8, 2021. Results: The project concluded data collection after 86 days on March 4, 2022, with usage showing a steady increase in the final three weeks. The project produced 73 unique users (37 faculty and 36 residents). The user survey responses showed 83.3% agreeing they would consult the differential generated by Isabel Pro if available at every patient encounter (+19.2% compared to the challenge survey) and 77.8% agreeing that the suggestions would be helpful in routine practice (+2.7% compared to the challenge survey). More than one-third (36.8%) responded that they changed their diagnosis in response to the differential. Limitations: Only usage statistics were analyzed; the system records no reason for the clinician discontinuing a diagnostic session. Only 20 participants responded out of 73 (27.4%), so even though the respondents represented a spread of experience levels, the results may not represent the total number of potential users. The project covered a limited period of 86 days. Conclusions: Diagnostic inaccuracy is a significant patient safety concern. Studies show that computerized diagnostic decision support systems improve diagnostic accuracy, but they are not wide implementation lags despite these findings. This project demonstrated the feasibility of implementing such a well-known system in academic medical practice. The responses to the surveys demonstrate favorable opinions about the system’s perceived usefulness. Active communication and dissemination programs may be essential to improve and sustain use