Statistical evaluation of surrogate outcomes: methodological extensions to ordinal outcomes with applications in acute stroke

Background Surrogate outcomes are measures of treatment effect that can be used to predict treatment effect on the true outcome of interest. Surrogates are valued as they can be used in place of true outcomes to reduce the length, size, or intrusiveness of a clinical trial. However, validation of surrogacy is a conceptually complicated area and much theoretical and practical statistical development has been conducted in recent years. Methods A systematic review was conducted to identify which surrogate evaluation approach was best suited to be extended to ordinal outcomes. I extended a foremost approach to the case where the surrogate, the true clinical outcome, or both are ordinal outcomes. This extension investigated surrogacy at both the trial and individual levels; trial level surrogacy was based on a two stage method. The extension was developed through large simulation studies and used to investigate whether deep venous thromboembolism (DVT) was a surrogate for the ongoing measure of death and disability the Oxford Handicap Scale (OHS), using data from the stroke trial CLOTS3. CLOTS3 was a large multi-centre randomised clinical trial which investigated whether intermittent pneumatic compression (IPC) applied to the legs reduced the occurrence of deep venous thromboembolism (DVT) in stroke clinical trial patients. Results The systematic review identified the information theory approach as the most intuitively and practically worthwhile approach to surrogacy evaluation. I extended this approach to: a binary surrogate and ordinal true outcome (the binary-ordinal setting); the ordinal-binary and the ordinal-ordinal settings. The simulation studies showed that the approach worked well in most scenarios tested. However, trial level surrogacy was impacted by loss of efficiency due to the use of the two stage method. Bias imposed at the trial level by separation of discrete outcomes was effectively dealt with using a penalised likelihood method. The information theory approach for ordinal outcomes identified no surrogate that would predict treatment effect of IPC on the true outcome OHS measured at six months in the stroke trial CLOTS3

Post stroke intervention trial in fatigue (POSITIF):Randomised multicentre feasibility trial

OBJECTIVE: To test the feasibility of a telephone delivered intervention, informed by cognitive behavioural principles, for post-stroke fatigue, and estimated its effect on fatigue and other outcomes. DESIGN: Randomised controlled parallel group trial. SETTING: Three Scottish stroke services. SUBJECTS: Stroke survivors with fatigue three months to two years post-stroke onset. INTERVENTIONS: Seven telephone calls (fortnightly then a ‘booster session’ at 16 weeks) of a manualised intervention, plus information about fatigue, versus information only. MAIN MEASURES: Feasibility of trial methods, and collected outcome measures (fatigue, mood, anxiety, social participation, quality of life, return to work) just before randomisation, at the end of treatment (four months after randomisation) and at six months after randomisation. RESULTS: Between October 2018 and January 2020, we invited 886 stroke survivors to participate in postal screening: 188/886 (21%) returned questionnaires and consented, of whom 76/188 (40%) were eligible and returned baseline forms; 64/76 (84%) returned six month follow-up questionnaires. Of the 39 allocated the intervention, 23 (59%) attended at least four sessions. At six months, there were no significant differences between the groups (adjusted mean differences in Fatigue Assessment Scale −0.619 (95% CI −4.9631, 3.694; p = 0.768), the Generalised Anxiety Disorder 7 −0.178 (95% CI −3.823, 3.467, p = 0.92), and the Patient Health Questionnaire −0.247 (95% CI −2.935, 2.442, p = 0.851). There were no between-group differences in quality of life, social participation or return to work. CONCLUSION: Patients can be recruited to a trial of this design. These data will inform the design of further trials in post-stroke fatigue

Physiological deterioration in the Emergency Department: the SNAP40-ED study

Continuous novel ambulatory monitoring may detect deterioration in Emergency Department (ED) patients more rapidly, prompting treatment and preventing adverse events. Single-centre, open-label, prospective, observational cohort study recruiting high/medium acuity (Manchester triage category 2 and 3) participants, aged over 16 years, presenting to ED. Participants were fitted with a novel wearable monitoring device alongside standard clinical care (wired monitoring and/or manual clinical staff vital sign recording) and observed for up to 4 hours in the ED. Primary outcome was time to detection of deterioration. Two-hundred and fifty (250) patients were enrolled. In 82 patients (32.8%) with standard monitoring (wired monitoring and/or manual clinical staff vital sign recording), deterioration in at least one vital sign was noted during their four-hour ED stay. Overall, the novel device detected deterioration a median of 34 minutes earlier than wired monitoring (Q1, Q3 67,194; n=73, mean difference 39.48, p<0.0001). The novel device detected deterioration a median of 24 minutes (Q1, Q3 2,43; n=42) earlier than wired monitoring and 65 minutes (Q1, Q3 28,114; n=31) earlier than manual vital signs. Deterioration in physiology was common in ED patients. ED staff spent a significant amount of time performing observations and responding to alarms, with many not escalated. The novel device detected deterioration significantly earlier than standard care

Pathogenic outcome following experimental infection of sheep with Chlamydia abortus variant strains LLG and POS

This study investigated the pathogenesis of two variant strains (LLG and POS) of Chlamydia abortus, in comparison to a typical wild-type strain (S26/3) which is known to be responsible for late term abortion in small ruminants. Challenge with the three strains at mid-gestation resulted in similar pregnancy outcomes, with abortion occurring in approximately 50±60% of ewes with the mean gestational lengths also being similar. However, differences were observed in the severity of placental pathology, with infection appearing milder for strain LLG, which was reflected in the lower number of organisms shed in vaginal swabs post-partum and less gross pathology and organisms present in placental smears. Results for strain POS were somewhat different than LLG with a more focal restriction of infection observed. Post-abortion antibody responses revealed prominent differences in seropositivity to the major outer membrane protein (MOMP) present in elementary body (EB) preparations under denaturing conditions, most notably with anti-LLG and anti-POS convalescent sera where there was no or reduced detection of MOMP present in EBs derived from the three strains. These results and additional analysis of whole EB and chlamydial outer membrane complex preparations suggest that there are conformational differences in MOMP for the three strains. Overall, the results suggest that gross placental pathology and clinical outcome is not indicative of bacterial colonization and the severity of infection. The results also highlight potential conformational differences in MOMP epitopes that perhaps impact on disease diagnosis and the development of new vaccines

A smartphone-based test for the assessment of attention deficits in delirium: A case-control diagnostic test accuracy study in older hospitalised patients.

BACKGROUND: Delirium is a common and serious acute neuropsychiatric syndrome which is often missed in routine clinical care. Inattention is the core cognitive feature. Diagnostic test accuracy (including cut-points) of a smartphone Delirium App (DelApp) for assessing attention deficits was assessed in older hospital inpatients. METHODS: This was a case-control study of hospitalised patients aged ≥65 years with delirium (with or without pre-existing cognitive impairment), who were compared to patients with dementia without delirium, and patients without cognitive impairment. Reference standard delirium assessment, which included a neuropsychological test battery, was based on Diagnostic and Statistical Manual of Mental Disorders-5 criteria. A separate blinded assessor administered the DelApp arousal assessment (score 0-4) and attention task (0-6) yielding an overall score of 0 to 10 (lower scores indicate poorer performance). Analyses included receiver operating characteristic curves and sensitivity and specificity. Optimal cut-points for delirium detection were determined using Youden's index. RESULTS: A total of 187 patients were recruited, mean age 83.8 (range 67-98) years, 152 (81%) women; n = 61 with delirium; n = 61 with dementia without delirium; and n = 65 without cognitive impairment. Patients with delirium performed poorly on the DelApp (median score = 4/10; inter-quartile range 3.0, 5.5) compared to patients with dementia (9.0; 5.5, 10.0) and those without cognitive impairment (10.0; 10.0, 10.0). Area under the curve for detecting delirium was 0.89 (95% Confidence Interval 0.84, 0.94). At an optimal cut-point of ≤8, sensitivity was 91.7% (84.7%, 98.7%) and specificity 74.2% (66.5%, 81.9%) for discriminating delirium from the other groups. Specificity was 68.3% (56.6%, 80.1%) for discriminating delirium from dementia (cut-point ≤6). CONCLUSION: Patients with delirium (with or without pre-existing cognitive impairment) perform poorly on the DelApp compared to patients with dementia and those without cognitive impairment. A cut-point of ≤8/10 is suggested as having optimal sensitivity and specificity. The DelApp is a promising tool for assessment of attention deficits associated with delirium in older hospitalised adults, many of whom have prior cognitive impairment, and should be further validated in representative patient cohorts

Erratum to: Methods for evaluating medical tests and biomarkers

[This corrects the article DOI: 10.1186/s41512-016-0001-y.]