Cancer journals included in the PRE-study and in the POST-study.

<p>Cancer journals included in the PRE-study and in the POST-study.</p

Did the reporting of prognostic studies of tumour markers improve since the introduction of REMARK guideline? A comparison of reporting in published articles

<div><p>Although biomarkers are perceived as highly relevant for future clinical practice, few biomarkers reach clinical utility for several reasons. Among them, poor reporting of studies is one of the major problems. To aid improvement, reporting guidelines like REMARK for tumour marker prognostic (TMP) studies were introduced several years ago. The aims of this project were to assess whether reporting quality of TMP-studies improved in comparison to a previously conducted study assessing reporting quality of TMP-studies (PRE-study) and to assess whether articles citing REMARK (citing group) are better reported, in comparison to articles not citing REMARK (not-citing group).</p><p>For the POST-study, recent articles citing and not citing REMARK (53 each) were identified in selected journals through systematic literature search and evaluated in same way as in the PRE-study. Ten of the 20 items of the REMARK checklist were evaluated and used to define an overall score of reporting quality.</p><p>The observed overall scores were 53.4% (range: 10%-90%) for the PRE-study, 57.7% (range: 20%-100%) for the not-citing group and 58.1% (range: 30%-100%) for the citing group of the POST-study. While there is no difference between the two groups of the POST-study, the POST-study shows a slight but not relevant improvement in reporting relative to the PRE-study. Not all the articles of the citing group, cited REMARK appropriately. Irrespective of whether REMARK was cited, the overall score was slightly higher for articles published in journals requesting adherence to REMARK than for those published in journals not requesting it: 59.9% versus 51.9%, respectively.</p><p>Several years after the introduction of REMARK, many key items of TMP-studies are still very poorly reported. A combined effort is needed from authors, editors, reviewers and methodologists to improve the current situation. Good reporting is not just nice to have but is essential for any research to be useful.</p></div

Percentages of articles adequately reporting information for 10 selected items of the REMARK checklist.

<p>The items are sorted by percentages obtained in the PRE-study [<a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0178531#pone.0178531.ref030" target="_blank">30</a>]. See <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0178531#pone.0178531.t001" target="_blank">Table 1</a> or <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0178531#pone.0178531.s007" target="_blank">S2 Table</a> for explanation of abbreviations used for different checklist items.</p

Overview of the 10 assessed items of the REMARK checklist.

<p>Overview of the 10 assessed items of the REMARK checklist.</p

Literature search–flow chart.

<p>Literature search–flow chart.</p

Distribution of confidence scores for patients with and without polyps.

<p>Each histogram shows the distribution of confidence score values using CAD CT colonography for an individual reader separately for patients with (brown) and without polyps (yellow) based on the reference standard. Five readers are shown in plots labelled 1 to 5.</p

Distribution of published confidence scores.

<p>Published confidence scores assigned by individual readers (Reader 1 and Reader 2) reported for patients without disease (DN) and patients with disease (DP) from two studies. In Hussain et al. <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0107633#pone.0107633-Hussain1" target="_blank">[17]</a>, classification of MR imaging of cirrohotic liver used categories: 1 definitely benign; 2 probably benign; 3 possibly malignant; 4 definitely malignant. In Westphalen et al. <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0107633#pone.0107633-Westphalen1" target="_blank">[18]</a>, classification of MR imaging of peripheral zone tissue from patients with prostate cancer used categories: 1 likely benign; 2 possibly benign; 3 equivocal or indeterminate; 4 possibly malignant; 5 likely malignant.</p

Steps in calculation of ROC AUC.

<p>Steps in calculation of ROC AUC.</p

Impact of few false positives.

<p>ROC curve for reader 4 using CT colonography with CAD. The data from the original read (orange curve) includes two patients where false positive polyps have been indicated with confidence scores of 40 and 50. Perturbing these two patient scores to values of 70 (yellow curve) and 20 and 70 (brown curve) demonstrate changes in ROC curves. Sensitivity and specificity are expressed as percentages.</p

ROC AUC using different methods and different ROC curves.

<p>ROC AUC and standard errors calculated for five readers using CT colonography without CAD.</p><p>The Wilcoxon method is equivalent to the Wilcoxon statistic, an empirical method not requiring a ROC curve to be fitted. DBM and PROPROC methods sometimes give different ROC AUC because different ROC curves are fitted as seen in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0107633#pone-0107633-g005" target="_blank">figure 5</a> for Readers 2 and 5. NC = could not be calculated with DBM MRMC software.</p><p>ROC AUC using different methods and different ROC curves.</p