Use of weighted multivariate estimates in trials of multi-serotype vaccines to simplify interpretation of treatment differences

<div><p>Background</p><p>Many vaccines contain multiple components. Licensed pneumococcal conjugate vaccines (PCV) contain polysaccharides from 7, 10, or 13 different serotypes of <i>Streptococcus pneumoniae</i>. The main outcomes in randomised trials of pneumococcal vaccines are serotype-specific antibody measures. Comparisons are made between groups for each serotype, resulting in multiple separate comparisons of treatment effects which can be complicated to interpret. We investigated methods for computing the overall difference between vaccine groups across all serotypes.</p><p>Methods</p><p>Pneumococcal antibody concentrations were obtained from a randomised controlled trial of ten-valent pneumococcal vaccine, conducted in Kathmandu, Nepal. Infants received either 2 priming doses of vaccine at 6 and 14 weeks of age followed by a booster (2+1), or 3 priming doses at 6, 10, and 14 weeks of age with no booster (3+0). The overall difference between vaccine schedules across all serotypes was computed at each visit using a multivariate linear model with equal weights for each serotype. Alternative weights were derived from invasive pneumococcal disease cases in Nepal, Bangladesh and Pakistan, and from estimates of the relative invasiveness of each serotype and used in sensitivity analyses.</p><p>Results</p><p>When 10 separate estimates of treatment differences were computed the ratio of antibody responses for each serotype in the 2+1 group compared with the 3+0 group at 10 months of age varied greatly, with serotype-specific GMRs ranging from 2.80 for serotype 14, to 9.14 for serotype 18C. Using equal weights for each serotype, the overall geometric mean ratio (GMR) was 5.02 (95% CI 4.06−6.22) at 10 months of age, and 1.46 (95% CI 1.14−1.88) at 3 years of age. Using weights based on disease incidence gave GMRs ranging from 5.15 to 6.63 at 10 months of age, and 1.47 to 1.78 at 3 years of age. Using weights based on relative invasiveness gave estimates of 6.81 and 1.59, at 10 months and 3 years respectively.</p><p>Conclusion</p><p>PCV clinical trial data have a multivariate structure with correlated outcomes for different serotypes. When analysing each serotype separately, the multiple estimates of the treatment effect can complicate the interpretation of trial results. Reporting a single overall estimate which accounts for the correlation between outcomes can simplify such interpretation. Treatment effects can be weighted equally or alternative weights derived from independent data can be used.</p><p>Many modern vaccines have multiple components, such as quadrivalent meningococcal group ACWY vaccine or four-component group B meningococcal vaccine, thus these methods are widely applicable.</p></div

Weights derived from estimates of the invasive potential of pneumococcal PCV10 serotypes.

<p>Weights derived from estimates of the invasive potential of pneumococcal PCV10 serotypes.</p

Correlation coefficients for correlations between treatment differences at 10 months of age (post-booster).

<p>Correlation coefficients for correlations between treatment differences at 10 months of age (post-booster).</p

Multi-Reader Multi-Case Studies Using the Area under the Receiver Operator Characteristic Curve as a Measure of Diagnostic Accuracy: Systematic Review with a Focus on Quality of Data Reporting

<div><p>Introduction</p><p>We examined the design, analysis and reporting in multi-reader multi-case (MRMC) research studies using the area under the receiver-operating curve (ROC AUC) as a measure of diagnostic performance.</p><p>Methods</p><p>We performed a systematic literature review from 2005 to 2013 inclusive to identify a minimum 50 studies. Articles of diagnostic test accuracy in humans were identified via their citation of key methodological articles dealing with MRMC ROC AUC. Two researchers in consensus then extracted information from primary articles relating to study characteristics and design, methods for reporting study outcomes, model fitting, model assumptions, presentation of results, and interpretation of findings. Results were summarized and presented with a descriptive analysis.</p><p>Results</p><p>Sixty-four full papers were retrieved from 475 identified citations and ultimately 49 articles describing 51 studies were reviewed and extracted. Radiological imaging was the index test in all. Most studies focused on lesion detection vs. characterization and used less than 10 readers. Only 6 (12%) studies trained readers in advance to use the confidence scale used to build the ROC curve. Overall, description of confidence scores, the ROC curve and its analysis was often incomplete. For example, 21 (41%) studies presented no ROC curve and only 3 (6%) described the distribution of confidence scores. Of 30 studies presenting curves, only 4 (13%) presented the data points underlying the curve, thereby allowing assessment of extrapolation. The mean change in AUC was 0.05 (−0.05 to 0.28). Non-significant change in AUC was attributed to underpowering rather than the diagnostic test failing to improve diagnostic accuracy.</p><p>Conclusions</p><p>Data reporting in MRMC studies using ROC AUC as an outcome measure is frequently incomplete, hampering understanding of methods and the reliability of results and study conclusions. Authors using this analysis should be encouraged to provide a full description of their methods and results.</p></div

Serotype-specific geometric mean ratios (2+1 schedule relative to 3+0 schedule) with overall estimates derived using different weighting structures.

<p>Serotype-specific geometric mean ratios (2+1 schedule relative to 3+0 schedule) with overall estimates derived using different weighting structures.</p

South Asian estimates of the proportions of vaccine serotype-specific invasive pneumococcal disease due to PCV10 serotypes in unvaccinated populations.

<p>South Asian estimates of the proportions of vaccine serotype-specific invasive pneumococcal disease due to PCV10 serotypes in unvaccinated populations.</p

PRISMA flow diagram [8] for the systematic review.

<p>PRISMA flow diagram <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0116018#pone.0116018-Moher1" target="_blank">[8]</a> for the systematic review.</p

Citations for the 49 papers (contributing 51 studies) included in the systematic review.

<p>Details are also provided for the 15 articles excluded from the systematic review after reading the full-text, along with primary reasons for their exclusion (multiple reasons for exclusion were possible).</p><p>Citations for the 49 papers (contributing 51 studies) included in the systematic review.</p

Bar chart showing data extracted by the systematic review relating to the presentation of individual study results.

<p>Bar chart showing data extracted by the systematic review relating to the presentation of individual study results.</p