16 research outputs found
Inter- and intra-observer reproducibility of CT-Leaman score by an independent core lab
To assess the reproducibility of CT-based Leaman score (CT-LeSc). CT-LeSc can non-invasively quantify total coronary atherosclerotic burden and is an independent long-term predictor of cardiac events. Its calculation however relies on the subjective assessment of lesions using coronary computed tomography angiography and therefore is subject to intra- and inter-observer variability. Inter-observer reproducibility was assessed by calculating the CT-LeSc in 50 patients randomly selected from the SYNTAX III REVOLUTION and ABSORB trials by two separate teams, each made up of two cardiologists, who reported results by consensus. For intra-observer reproducibility, the CT-LeSc was calculated in same 50 patients on two occasions eight weeks apart, by the same team of two cardiologists. The level of agreement was measured by the weighted kappa statistic, with intra- and inter-observer variability used to evaluate the CT-LeSc’s reproducibility. The variables evaluated by weighted kappa statistics were total number of lesions; number of calcified lesions; number of non-calcified lesions; number of mixed lesions; number of obstructive lesions; number of non-obstructive lesions; and the total CT-LeSc in increments of ten and five. During assessment of inter-observer variability the mean ± standard deviation (SD) CT-LeSc calculated by the first and second team was 15.36 ± 5.57 versus 15.24 ± 5.16. The mean of the differences (precision) was 0.97, with a SD (accuracy) 1.17. The inter-observer variability was lowest for Leaman score in increments of five (weighted kappa 0.93), and highest for the total number of calcified lesions (weighted kappa 0.66). During assessment of intra-observer variability, the mean ± SD CT-LeSc were 16.61 ± 5.28 versus 16.82 ± 5.55. The mean ± SD of the differences was 1.28 ± 1.02. The intra-observer variability was the lowest for Leaman score in increments of five (weighted kappa 0.93), and the highest for the total number of lesions and calcified lesions (weighted kappa 0.65). CT-LeSc has substantial to near-perfect agreement for reproducibility. Cardiolog
The CHEMDNER corpus of chemicals and drugs and its annotation principles
The automatic extraction of chemical information from text requires the recognition of chemical entity mentions as one
of its key steps. When developing supervised named entity recognition (NER) systems, the availability of a large,
manually annotated text corpus is desirable. Furthermore, large corpora permit the robust evaluation and comparison
of different approaches that detect chemicals in documents. We present the CHEMDNER corpus, a collection of 10,000
PubMed abstracts that contain a total of 84,355 chemical entity mentions labeled manually by expert chemistry
literature curators, following annotation guidelines specifically defined for this task. The abstracts of the CHEMDNER
corpus were selected to be representative for all major chemical disciplines. Each of the chemical entity mentions was
manually labeled according to its structure-associated chemical entity mention (SACEM) class: abbreviation, family,
formula, identifier, multiple, systematic and trivial. The difficulty and consistency of tagging chemicals in text was
measured using an agreement study between annotators, obtaining a percentage agreement of 91. For a subset of the
CHEMDNER corpus (the test set of 3,000 abstracts) we provide not only the Gold Standard manual annotations, but also
mentions automatically detected by the 26 teams that participated in the BioCreative IV CHEMDNER chemical mention
recognition task. In addition, we release the CHEMDNER silver standard corpus of automatically extracted mentions
from 17,000 randomly selected PubMed abstracts. A version of the CHEMDNER corpus in the BioC format has been
generated as well. We propose a standard for required minimum information about entity annotations for the
construction of domain specific corpora on chemical and drug entities. The CHEMDNER corpus and annotation
guidelines are available at: http://www.biocreative.org/resources/biocreative-iv/chemdner-corpus