Factors affecting the labelling accuracy of brain MRI studies relevant for deep learning abnormality detection

Unlocking the vast potential of deep learning-based computer vision classification systems necessitates large data sets for model training. Natural Language Processing (NLP)—involving automation of dataset labelling—represents a potential avenue to achieve this. However, many aspects of NLP for dataset labelling remain unvalidated. Expert radiologists manually labelled over 5,000 MRI head reports in order to develop a deep learning-based neuroradiology NLP report classifier. Our results demonstrate that binary labels (normal vs. abnormal) showed high rates of accuracy, even when only two MRI sequences (T2-weighted and those based on diffusion weighted imaging) were employed as opposed to all sequences in an examination. Meanwhile, the accuracy of more specific labelling for multiple disease categories was variable and dependent on the category. Finally, resultant model performance was shown to be dependent on the expertise of the original labeller, with worse performance seen with non-expert vs. expert labellers

Labelling imaging datasets on the basis of neuroradiology reports: a validation study

Natural language processing (NLP) shows promise as a means to automate the labelling of hospital-scale neuroradiology magnetic resonance imaging (MRI) datasets for computer vision applications. To date, however, there has been no thorough investigation into the validity of this approach, including determining the accuracy of report labels compared to image labels as well as examining the performance of non-specialist labellers. In this work, we draw on the experience of a team of neuroradiologists who labelled over 5000 MRI neuroradiology reports as part of a project to build a dedicated deep learning-based neuroradiology report classifier. We show that, in our experience, assigning binary labels (i.e. normal vs abnormal) to images from reports alone is highly accurate. In contrast to the binary labels, however, the accuracy of more granular labelling is dependent on the category, and we highlight reasons for this discrepancy. We also show that downstream model performance is reduced when labelling of training reports is performed by a non-specialist. To allow other researchers to accelerate their research, we make our refined abnormality definitions and labelling rules available, as well as our easy-to-use radiology report labelling app which helps streamline this process

Deep learning to automate the labelling of head MRI datasets for computer vision applications

Objectives: The purpose of this study was to build a deep learning model to derive labels from neuroradiology reports and assign these to the corresponding examinations, overcoming a bottleneck to computer vision model development. Methods: Reference-standard labels were generated by a team of neuroradiologists for model training and evaluation. Three thousand examinations were labelled for the presence or absence of any abnormality by manually scrutinising the corresponding radiology reports (‘reference-standard report labels’); a subset of these examinations (n = 250) were assigned ‘reference-standard image labels’ by interrogating the actual images. Separately, 2000 reports were labelled for the presence or absence of 7 specialised categories of abnormality (acute stroke, mass, atrophy, vascular abnormality, small vessel disease, white matter inflammation, encephalomalacia), with a subset of these examinations (n = 700) also assigned reference-standard image labels. A deep learning model was trained using labelled reports and validated in two ways: comparing predicted labels to (i) reference-standard report labels and (ii) reference-standard image labels. The area under the receiver operating characteristic curve (AUC-ROC) was used to quantify model performance. Accuracy, sensitivity, specificity, and F1 score were also calculated. Results: Accurate classification (AUC-ROC &gt; 0.95) was achieved for all categories when tested against reference-standard report labels. A drop in performance (ΔAUC-ROC &gt; 0.02) was seen for three categories (atrophy, encephalomalacia, vascular) when tested against reference-standard image labels, highlighting discrepancies in the original reports. Once trained, the model assigned labels to 121,556 examinations in under 30 min. Conclusions: Our model accurately classifies head MRI examinations, enabling automated dataset labelling for downstream computer vision applications. Key Points: • Deep learning is poised to revolutionise image recognition tasks in radiology; however, a barrier to clinical adoption is the difficulty of obtaining large labelled datasets for model training. • We demonstrate a deep learning model which can derive labels from neuroradiology reports and assign these to the corresponding examinations at scale, facilitating the development of downstream computer vision models. • We rigorously tested our model by comparing labels predicted on the basis of neuroradiology reports with two sets of reference-standard labels: (1) labels derived by manually scrutinising each radiology report and (2) labels derived by interrogating the actual images.</p

Effects of antiplatelet therapy on stroke risk by brain imaging features of intracerebral haemorrhage and cerebral small vessel diseases: subgroup analyses of the RESTART randomised, open-label trial

Background Findings from the RESTART trial suggest that starting antiplatelet therapy might reduce the risk of recurrent symptomatic intracerebral haemorrhage compared with avoiding antiplatelet therapy. Brain imaging features of intracerebral haemorrhage and cerebral small vessel diseases (such as cerebral microbleeds) are associated with greater risks of recurrent intracerebral haemorrhage. We did subgroup analyses of the RESTART trial to explore whether these brain imaging features modify the effects of antiplatelet therapy

Global, regional, and national burden of traumatic brain injury and spinal cord injury, 1990–2016: A systematic analysis for the Global Burden of Disease Study 2016

© 2019 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license Background: Traumatic brain injury (TBI) and spinal cord injury (SCI) are increasingly recognised as global health priorities in view of the preventability of most injuries and the complex and expensive medical care they necessitate. We aimed to measure the incidence, prevalence, and years of life lived with disability (YLDs) for TBI and SCI from all causes of injury in every country, to describe how these measures have changed between 1990 and 2016, and to estimate the proportion of TBI and SCI cases caused by different types of injury. Methods: We used results from the Global Burden of Diseases, Injuries, and Risk Factors (GBD) Study 2016 to measure the global, regional, and national burden of TBI and SCI by age and sex. We measured the incidence and prevalence of all causes of injury requiring medical care in inpatient and outpatient records, literature studies, and survey data. By use of clinical record data, we estimated the proportion of each cause of injury that required medical care that would result in TBI or SCI being considered as the nature of injury. We used literature studies to establish standardised mortality ratios and applied differential equations to convert incidence to prevalence of long-term disability. Finally, we applied GBD disability weights to calculate YLDs. We used a Bayesian meta-regression tool for epidemiological modelling, used cause-specific mortality rates for non-fatal estimation, and adjusted our results for disability experienced with comorbid conditions. We also analysed results on the basis of the Socio-demographic Index, a compound measure of income per capita, education, and fertility. Findings: In 2016, there were 27·08 million (95% uncertainty interval [UI] 24·30–30·30 million) new cases of TBI and 0·93 million (0·78–1·16 million) new cases of SCI, with age-standardised incidence rates of 369 (331–412) per 100 000 population for TBI and 13 (11–16) per 100 000 for SCI. In 2016, the number of prevalent cases of TBI was 55·50 million (53·40–57·62 million) and of SCI was 27·04 million (24·98–30·15 million). From 1990 to 2016, the age-standardised prevalence of TBI increased by 8·4% (95% UI 7·7 to 9·2), whereas that of SCI did not change significantly (−0·2% [–2·1 to 2·7]). Age-standardised incidence rates increased by 3·6% (1·8 to 5·5) for TBI, but did not change significantly for SCI (−3·6% [–7·4 to 4·0]). TBI caused 8·1 million (95% UI 6·0–10·4 million) YLDs and SCI caused 9·5 million (6·7–12·4 million) YLDs in 2016, corresponding to age-standardised rates of 111 (82–141) per 100 000 for TBI and 130 (90–170) per 100 000 for SCI. Falls and road injuries were the leading causes of new cases of TBI and SCI in most regions. Interpretation: TBI and SCI constitute a considerable portion of the global injury burden and are caused primarily by falls and road injuries. The increase in incidence of TBI over time might continue in view of increases in population density, population ageing, and increasing use of motor vehicles, motorcycles, and bicycles. The number of individuals living with SCI is expected to increase in view of population growth, which is concerning because of the specialised care that people with SCI can require. Our study was limited by data sparsity in some regions, and it will be important to invest greater resources in collection of data for TBI and SCI to improve the accuracy of future assessments. Funding: Bill & Melinda Gates Foundation

Effects of antiplatelet therapy after stroke due to intracerebral haemorrhage (RESTART): a randomised, open-label trial

Background: Antiplatelet therapy reduces the risk of major vascular events for people with occlusive vascular disease, although it might increase the risk of intracranial haemorrhage. Patients surviving the commonest subtype of intracranial haemorrhage, intracerebral haemorrhage, are at risk of both haemorrhagic and occlusive vascular events, but whether antiplatelet therapy can be used safely is unclear. We aimed to estimate the relative and absolute effects of antiplatelet therapy on recurrent intracerebral haemorrhage and whether this risk might exceed any reduction of occlusive vascular events. Methods: The REstart or STop Antithrombotics Randomised Trial (RESTART) was a prospective, randomised, open-label, blinded endpoint, parallel-group trial at 122 hospitals in the UK. We recruited adults (≥18 years) who were taking antithrombotic (antiplatelet or anticoagulant) therapy for the prevention of occlusive vascular disease when they developed intracerebral haemorrhage, discontinued antithrombotic therapy, and survived for 24 h. Computerised randomisation incorporating minimisation allocated participants (1:1) to start or avoid antiplatelet therapy. We followed participants for the primary outcome (recurrent symptomatic intracerebral haemorrhage) for up to 5 years. We analysed data from all randomised participants using Cox proportional hazards regression, adjusted for minimisation covariates. This trial is registered with ISRCTN (number ISRCTN71907627). Findings: Between May 22, 2013, and May 31, 2018, 537 participants were recruited a median of 76 days (IQR 29–146) after intracerebral haemorrhage onset: 268 were assigned to start and 269 (one withdrew) to avoid antiplatelet therapy. Participants were followed for a median of 2·0 years (IQR [1·0– 3·0]; completeness 99·3%). 12 (4%) of 268 participants allocated to antiplatelet therapy had recurrence of intracerebral haemorrhage compared with 23 (9%) of 268 participants allocated to avoid antiplatelet therapy (adjusted hazard ratio 0·51 [95% CI 0·25–1·03]; p=0·060). 18 (7%) participants allocated to antiplatelet therapy experienced major haemorrhagic events compared with 25 (9%) participants allocated to avoid antiplatelet therapy (0·71 [0·39–1·30]; p=0·27), and 39 [15%] participants allocated to antiplatelet therapy had major occlusive vascular events compared with 38 [14%] allocated to avoid antiplatelet therapy (1·02 [0·65–1·60]; p=0·92). Interpretation: These results exclude all but a very modest increase in the risk of recurrent intracerebral haemorrhage with antiplatelet therapy for patients on antithrombotic therapy for the prevention of occlusive vascular disease when they developed intracerebral haemorrhage. The risk of recurrent intracerebral haemorrhage is probably too small to exceed the established benefits of antiplatelet therapy for secondary prevention

Effects of antiplatelet therapy after stroke due to intracerebral haemorrhage (RESTART): a randomised, open-label trial

Background: Antiplatelet therapy reduces the risk of major vascular events for people with occlusive vascular disease, although it might increase the risk of intracranial haemorrhage. Patients surviving the commonest subtype of intracranial haemorrhage, intracerebral haemorrhage, are at risk of both haemorrhagic and occlusive vascular events, but whether antiplatelet therapy can be used safely is unclear. We aimed to estimate the relative and absolute effects of antiplatelet therapy on recurrent intracerebral haemorrhage and whether this risk might exceed any reduction of occlusive vascular events. Methods: The REstart or STop Antithrombotics Randomised Trial (RESTART) was a prospective, randomised, open-label, blinded endpoint, parallel-group trial at 122 hospitals in the UK. We recruited adults (≥18 years) who were taking antithrombotic (antiplatelet or anticoagulant) therapy for the prevention of occlusive vascular disease when they developed intracerebral haemorrhage, discontinued antithrombotic therapy, and survived for 24 h. Computerised randomisation incorporating minimisation allocated participants (1:1) to start or avoid antiplatelet therapy. We followed participants for the primary outcome (recurrent symptomatic intracerebral haemorrhage) for up to 5 years. We analysed data from all randomised participants using Cox proportional hazards regression, adjusted for minimisation covariates. This trial is registered with ISRCTN (number ISRCTN71907627). Findings: Between May 22, 2013, and May 31, 2018, 537 participants were recruited a median of 76 days (IQR 29–146) after intracerebral haemorrhage onset: 268 were assigned to start and 269 (one withdrew) to avoid antiplatelet therapy. Participants were followed for a median of 2·0 years (IQR [1·0– 3·0]; completeness 99·3%). 12 (4%) of 268 participants allocated to antiplatelet therapy had recurrence of intracerebral haemorrhage compared with 23 (9%) of 268 participants allocated to avoid antiplatelet therapy (adjusted hazard ratio 0·51 [95% CI 0·25–1·03]; p=0·060). 18 (7%) participants allocated to antiplatelet therapy experienced major haemorrhagic events compared with 25 (9%) participants allocated to avoid antiplatelet therapy (0·71 [0·39–1·30]; p=0·27), and 39 [15%] participants allocated to antiplatelet therapy had major occlusive vascular events compared with 38 [14%] allocated to avoid antiplatelet therapy (1·02 [0·65–1·60]; p=0·92). Interpretation: These results exclude all but a very modest increase in the risk of recurrent intracerebral haemorrhage with antiplatelet therapy for patients on antithrombotic therapy for the prevention of occlusive vascular disease when they developed intracerebral haemorrhage. The risk of recurrent intracerebral haemorrhage is probably too small to exceed the established benefits of antiplatelet therapy for secondary prevention

Effects of antiplatelet therapy after stroke due to intracerebral haemorrhage (RESTART): a randomised, open-label trial

BACKGROUND: Antiplatelet therapy reduces the risk of major vascular events for people with occlusive vascular disease, although it might increase the risk of intracranial haemorrhage. Patients surviving the commonest subtype of intracranial haemorrhage, intracerebral haemorrhage, are at risk of both haemorrhagic and occlusive vascular events, but whether antiplatelet therapy can be used safely is unclear. We aimed to estimate the relative and absolute effects of antiplatelet therapy on recurrent intracerebral haemorrhage and whether this risk might exceed any reduction of occlusive vascular events. METHODS: The REstart or STop Antithrombotics Randomised Trial (RESTART) was a prospective, randomised, open-label, blinded endpoint, parallel-group trial at 122 hospitals in the UK. We recruited adults (≥18 years) who were taking antithrombotic (antiplatelet or anticoagulant) therapy for the prevention of occlusive vascular disease when they developed intracerebral haemorrhage, discontinued antithrombotic therapy, and survived for 24 h. Computerised randomisation incorporating minimisation allocated participants (1:1) to start or avoid antiplatelet therapy. We followed participants for the primary outcome (recurrent symptomatic intracerebral haemorrhage) for up to 5 years. We analysed data from all randomised participants using Cox proportional hazards regression, adjusted for minimisation covariates. This trial is registered with ISRCTN (number ISRCTN71907627). FINDINGS: Between May 22, 2013, and May 31, 2018, 537 participants were recruited a median of 76 days (IQR 29-146) after intracerebral haemorrhage onset: 268 were assigned to start and 269 (one withdrew) to avoid antiplatelet therapy. Participants were followed for a median of 2·0 years (IQR [1·0- 3·0]; completeness 99·3%). 12 (4%) of 268 participants allocated to antiplatelet therapy had recurrence of intracerebral haemorrhage compared with 23 (9%) of 268 participants allocated to avoid antiplatelet therapy (adjusted hazard ratio 0·51 [95% CI 0·25-1·03]; p=0·060). 18 (7%) participants allocated to antiplatelet therapy experienced major haemorrhagic events compared with 25 (9%) participants allocated to avoid antiplatelet therapy (0·71 [0·39-1·30]; p=0·27), and 39 [15%] participants allocated to antiplatelet therapy had major occlusive vascular events compared with 38 [14%] allocated to avoid antiplatelet therapy (1·02 [0·65-1·60]; p=0·92). INTERPRETATION: These results exclude all but a very modest increase in the risk of recurrent intracerebral haemorrhage with antiplatelet therapy for patients on antithrombotic therapy for the prevention of occlusive vascular disease when they developed intracerebral haemorrhage. The risk of recurrent intracerebral haemorrhage is probably too small to exceed the established benefits of antiplatelet therapy for secondary prevention. FUNDING: British Heart Foundation

Acute ischaemic brain lesions in intracerebral haemorrhage : multicentre cross-sectional magnetic resonance imaging study.

Subclinical acute ischaemic lesions on brain magnetic resonance imaging have recently been described in spontaneous intracerebral haemorrhage, and may be important to understand pathophysiology and guide treatment. The underlying mechanisms are uncertain. We tested the hypothesis that ischaemic lesions are related to magnetic resonance imaging markers of the severity and type of small-vessel disease (hypertensive arteriopathy or cerebral amyloid angiopathy) in a multicentre, cross-sectional study. We studied consecutive patients with intracerebral haemorrhage from four specialist stroke centres, and age-matched stroke service referrals without intracerebral haemorrhage. Acute ischaemic lesions were assessed on magnetic resonance imaging (<3 months after intracerebral haemorrhage) using diffusion-weighted imaging. White matter changes and cerebral microbleeds were rated with validated scales. We investigated associations between diffusion-weighted imaging lesions, clinical and radiological characteristics. We included 114 patients with intracerebral haemorrhage (39 with clinically probable cerebral amyloid angiopathy) and 47 age-matched controls. The prevalence of diffusion-weighted imaging lesions was 9/39 (23%) in probable cerebral amyloid angiopathy-related intracerebral haemorrhage versus 6/75 (8%) in the remaining patients with intracerebral haemorrhage (P = 0.024); no diffusion-weighted imaging lesions were found in controls. Diffusion-weighted imaging lesions were mainly cortical and were associated with mean white matter change score (odds ratio 1.14 per unit increase, 95% confidence interval 1.02-1.28, P = 0.024) and the presence of strictly lobar cerebral microbleeds (odds ratio 3.85, 95% confidence interval 1.15-12.93, P = 0.029). Acute, subclinical ischaemic brain lesions are frequent but previously underestimated after intracerebral haemorrhage, and are three times more common in cerebral amyloid angiopathy-related intracerebral haemorrhage than in other intracerebral haemorrhage types. Ischaemic brain lesions are associated with white matter changes and cerebral microbleeds, suggesting that they result from an occlusive small-vessel arteriopathy. Diffusion-weighted imaging lesions contribute to the overall burden of vascular-related brain damage in intracerebral haemorrhage, and may be a useful surrogate marker of ongoing ischaemic injury from small-vessel damage