Improved Estimation of Optimal Cut-Off Point Associated with Youden Index Using Ranked Set Sampling

A diagnostic cut-off point of a biomarker measurement is needed for classifying a random subject to be either diseased or healthy. However, the cut-off point is usually unknown and needs to be estimated by some optimization criteria. One important criterion is the Youden index, which has been widely adopted in practice. The Youden index, which is defined as the maximum of (sensitivity + specificity −1), directly measures the largest total diagnostic accuracy a biomarker can achieve. Therefore, it is desirable to estimate the optimal cut-off point associated with the Youden index. Sometimes, taking the actual measurements of a biomarker is very difficult and expensive, while ranking them without the actual measurement can be relatively easy. In such cases, ranked set sampling can give more precise estimation than simple random sampling, as ranked set samples are more likely to span the full range of the population. In this study, kernel density estimation is utilized to numerically solve for an estimate of the optimal cut-off point. The asymptotic distributions of the kernel estimators based on two sampling schemes are derived analytically and we prove that the estimators based on ranked set sampling are relatively more efficient than that of simple random sampling and both estimators are asymptotically unbiased. Furthermore, the asymptotic confidence intervals are derived. Intensive simulations are carried out to compare the proposed method using ranked set sampling with simple random sampling, with the proposed method outperforming simple random sampling in all cases. A real data set is analyzed for illustrating the proposed method

Meta-analysis of diagnostic accuracy studies with multiple thresholds – comparison of different approaches

Methods for standard meta-analysis of diagnostic test accuracy studies are well established and understood. For the more complex case in which studies report test accuracy across multiple thresholds, several approaches have recently been proposed. These are based on similar ideas, but make different assumptions. In this article, we apply four different approaches to data from a recent systematic review in the area of nephrology and compare the results. The four approaches use: a linear mixed effects model, a Bayesian multinomial random effects model, a time-to-event model and a nonparametric model, respectively. In the case study data, the accuracy of neutrophil gelatinase-associated lipocalin for the diagnosis of acute kidney injury was assessed in different scenarios, with sensitivity and specificity estimates available for three thresholds in each primary study. All approaches led to plausible and mostly similar summary results. However, we found considerable differences in results for some scenarios, for example, differences in the area under the receiver operating characteristic curve (AUC) of up to 0.13. The Bayesian approach tended to lead to the highest values of the AUC, and the nonparametric approach tended to produce the lowest values across the different scenarios. Though we recommend using these approaches, our findings motivate the need for a simulation study to explore optimal choice of method in various scenarios

Using the ROC Curve to Measure Association and Evaluate Prediction Accuracy for a Binary Outcome

This review article addresses the ROC curve and its advantage over the odds ratio to measure the association between a continuous variable and a binary outcome. A simple parametric model under the normality assumption and the method of Box-Cox transformation for non-normal data are discussed. Applications of the binormal model and the Box-Cox transformation under both univariate and multivariate inference are illustrated by a comprehensive data analysis tutorial. Finally, a summary and recommendations are given as to the usage of the binormal ROC curve

Social Media Management Strategies for Organizational Impression Management and their Effect on Public Perception

With the growing importance of social media, companies increasingly rely on social media management tools to analyze social media activities and to professionalize their social media engagement. In this study, we evaluate how social media management tools, as part of an overarching social media strategy, help companies to positively influence the public perception among social media users. A mixed methods approach is applied, where we quantitatively analyze 15 million user-generated Twitter messages containing information about 45 large global companies highly active on Twitter, as well as almost 160 thousand corresponding messages sent from these companies via their corporate Twitter accounts. Additionally, we conducted interviews with six social media experts to gain complementary insights. By these means, we are able to identify significant differences between different social media management strategies and measure the corresponding effects on the public perception. (C) 2015 Elsevier B.V. All rights reserved

Multi-parametric MRI zone-specific diagnostic model performance compared with experienced radiologists for detection of prostate cancer

OBJECTIVES: Compare the performance of zone-specific multi-parametric-MRI (mp-MRI) diagnostic models in prostate cancer detection with experienced radiologists. METHODS: A single-centre, IRB approved, prospective STARD compliant 3 T MRI test dataset of 203 patients was generated to test validity and generalisability of previously reported 1.5 T mp-MRI diagnostic models. All patients included within the test dataset underwent 3 T mp-MRI, comprising T2, diffusion-weighted and dynamic contrast-enhanced imaging followed by transperineal template ± targeted index lesion biopsy. Separate diagnostic models (transition zone (TZ) and peripheral zone (PZ)) were applied to respective zones. Sensitivity/specificity and the area under the receiver operating characteristic curve (ROC-AUC) were calculated for the two zone-specific models. Two radiologists (A and B) independently Likert scored test 3 T mp-MRI dataset, allowing ROC analysis for each radiologist for each prostate zone. RESULTS: Diagnostic models applied to the test dataset demonstrated a ROC-AUC = 0.74 (95% CI 0.67–0.81) in the PZ and 0.68 (95% CI 0.61–0.75) in the TZ. Radiologist A/B had a ROC-AUC = 0.78/0.74 in the PZ and 0.69/0.69 in the TZ. Radiologists A and B each scored 51 patients in the PZ and 41 and 45 patients respectively in the TZ as Likert 3. The PZ model demonstrated a ROC-AUC = 0.65/0.67 for the patients Likert scored as indeterminate by radiologist A/B respectively, whereas the TZ model demonstrated a ROC-AUC = 0.74/0.69. CONCLUSION: Zone-specific mp-MRI diagnostic models demonstrate generalisability between 1.5 and 3 T mp-MRI protocols and show similar classification performance to experienced radiologists for prostate cancer detection. Results also indicate the ability of diagnostic models to classify cases with an indeterminate radiologist score. KEY POINTS: • MRI diagnostic models had similar performance to experienced radiologists for classification of prostate cancer. • MRI diagnostic models may help radiologists classify tumour in patients with indeterminate Likert 3 scores

A Bayesian model to estimate the cutoff and the clinical utility of a biomarker assay

To enable targeted therapies and enhance medical decision-making, biomarkers are increasingly used as screening and diagnostic tests. When using quantitative biomarkers for classification purposes, this often implies that an appropriate cutoff for the biomarker has to be determined and its clinical utility must be assessed. In the context of drug development, it is of interest how the probability of response changes with increasing values of the biomarker. Unlike sensitivity and specificity, predictive values are functions of the accuracy of the test, depend on the prevalence of the disease and therefore are a useful tool in this setting. In this paper, we propose a Bayesian method to not only estimate the cutoff value using the negative and positive predictive values, but also estimate the uncertainty around this estimate. Using Bayesian inference allows us to incorporate prior information, and obtain posterior estimates and credible intervals for the cut-off and associated predictive values. The performance of the Bayesian approach is compared with alternative methods via simulation studies of bias, interval coverage and width and illustrations on real data with binary and time-to-event outcomes are provided

Calcolo del cut-off nel caso di test diagnostici continui

Calcolare il punto di cut-off di un test diagnostico continu

A stepwise algorithm for linearly combining biomakers under Youden Index maximisation

Combining multiple biomarkers to provide predictive models with a greater discriminatory ability is a discipline that has received attention in recent years. Choosing the probability threshold that corresponds to the highest combined marker accuracy is key in disease diagnosis. The Youden index is a statistical metric that provides an appropriate synthetic index for diagnostic accuracy and a good criterion for choosing a cut-off point to dichotomize a biomarker. In this study, we present a new stepwise algorithm for linearly combining continuous biomarkers to maximize the Youden index. To investigate the performance of our algorithm, we analyzed a wide range of simulated scenarios and compared its performance with that of five other linear combination methods in the literature (a stepwise approach introduced by Yin and Tian, the min-max approach, logistic regression, a parametric approach under multivariate normality and a non-parametric kernel smoothing approach). The obtained results show that our proposed stepwise approach showed similar results to other algorithms in normal simulated scenarios and outperforms all other algorithms in non-normal simulated scenarios. In scenarios of biomarkers with the same means and a different covariance matrix for the diseased and non-diseased population, the min-max approach outperforms the rest. The methods were also applied on two real datasets (to discriminate Duchenne muscular dystrophy and prostate cancer), whose results also showed a higher predictive ability in our algorithm in the prostate cancer databas