Development and evaluation of a quality score for abstracts

BACKGROUND: The evaluation of abstracts for scientific meetings has been shown to suffer from poor inter observer reliability. A measure was developed to assess the formal quality of abstract submissions in a standardized way. METHODS: Item selection was based on scoring systems for full reports, taking into account published guidelines for structured abstracts. Interrater agreement was examined using a random sample of submissions to the American Gastroenterological Association, stratified for research type (n = 100, 1992–1995). For construct validity, the association of formal quality with acceptance for presentation was examined. A questionnaire to expert reviewers evaluated sensibility items, such as ease of use and comprehensiveness. RESULTS: The index comprised 19 items. The summary quality scores showed good interrater agreement (intra class coefficient 0.60 – 0.81). Good abstract quality was associated with abstract acceptance for presentation at the meeting. The instrument was found to be acceptable by expert reviewers. CONCLUSION: A quality index was developed for the evaluation of scientific meeting abstracts which was shown to be reliable, valid and useful

Reviewer agreement trends from four years of electronic submissions of conference abstract

BACKGROUND: The purpose of this study was to determine the inter-rater agreement between reviewers on the quality of abstract submissions to an annual national scientific meeting (Canadian Association of Emergency Physicians; CAEP) to identify factors associated with low agreement. METHODS: All abstracts were submitted using an on-line system and assessed by three volunteer CAEP reviewers blinded to the abstracts' source. Reviewers used an on-line form specific for each type of study design to score abstracts based on nine criteria, each contributing from two to six points toward the total (maximum 24). The final score was determined to be the mean of the three reviewers' scores using Intraclass Correlation Coefficient (ICC). RESULTS: 495 Abstracts were received electronically during the four-year period, 2001 – 2004, increasing from 94 abstracts in 2001 to 165 in 2004. The mean score for submitted abstracts over the four years was 14.4 (95% CI: 14.1–14.6). While there was no significant difference between mean total scores over the four years (p = 0.23), the ICC increased from fair (0.36; 95% CI: 0.24–0.49) to moderate (0.59; 95% CI: 0.50–0.68). Reviewers agreed less on individual criteria than on the total score in general, and less on subjective than objective criteria. CONCLUSION: The correlation between reviewers' total scores suggests general recognition of "high quality" and "low quality" abstracts. Criteria based on the presence/absence of objective methodological parameters (i.e., blinding in a controlled clinical trial) resulted in higher inter-rater agreement than the more subjective and opinion-based criteria. In future abstract competitions, defining criteria more objectively so that reviewers can base their responses on empirical evidence may lead to increased consistency of scoring and, presumably, increased fairness to submitters

Environmental Factors in the Relapse and Recurrence of Inflammatory Bowel Disease:A Review of the Literature

The causes of relapse in patients with Crohn's disease (CD) and ulcerative colitis (UC) are largely unknown. This paper reviews the epidemiological and clinical data on how medications (non-steroidal anti-inflammatory drugs, estrogens and antibiotics), lifestyle factors (smoking, psychological stress, diet and air pollution) may precipitate clinical relapses and recurrence. Potential biological mechanisms include: increasing thrombotic tendency, imbalances in prostaglandin synthesis, alterations in the composition of gut microbiota, and mucosal damage causing increased permeability

A phase I dose-escalation study to evaluate safety and tolerability of sorafenib combined with sirolimus in patients with advanced solid cancer

Contains fulltext : 87630.pdf (publisher's version ) (Closed access)BACKGROUND: The combination of sorafenib (vascular endothelial growth factor receptor 2 inhibitor) and sirolimus (mammalian target of rapamycin inhibitor) might work synergistically. METHODS: A phase I dose-escalation study with sorafenib twice a day (b.i.d.) and sirolimus once daily (q.d.) was performed to determine the recommended dose of the combination in patients with solid tumours. Secondary objectives were to determine the safety profile and maximum tolerated dose (MTD), and to evaluate the pharmacokinetics (PK) of the combination. RESULTS: Dose-limiting toxicities were transaminitis and cutaneous toxicity. The most frequently reported adverse events were elevated transaminases, hypophosphatemia, fatigue, anorexia, diarrhoea, nausea, rash and palmar-plantar erythrodysaesthesia. Sirolimus did not change the PK of sorafenib; in contrast, sorafenib reduced the AUC(0-96) and C(max) of sirolimus. No objective responses were observed; eight patients showed stable disease for a median of 16.3 weeks (range 8-24). The MTD of the combination was sorafenib 200 mg b.i.d. with sirolimus 1 mg q.d. CONCLUSION: The combination of sorafenib and sirolimus showed enhanced toxicity, which could not be explained by the PK of both drugs. The relative low doses at the MTD, in combination with the PK results, do not warrant further development of this combination

More insight into the fate of biomedical meeting abstracts: a systematic review

BACKGROUND: It has been estimated that about 45% of abstracts that are accepted for presentation at biomedical meetings will subsequently be published in full. The acceptance of abstracts at meetings and their fate after initial rejection are less well understood. We set out to estimate the proportion of abstracts submitted to meetings that are eventually published as full reports, and to explore factors that are associated with meeting acceptance and successful publication. METHODS: Studies analysing acceptance of abstracts at biomedical meetings or their subsequent full publication were searched in MEDLINE, OLDMEDLINE, EMBASE, Cochrane Library, CINAHL, BIOSIS, Science Citation Index Expanded, and by hand searching of bibliographies and proceedings. We estimated rates of abstract acceptance and of subsequent full publication, and identified abstract and meeting characteristics associated with acceptance and publication, using logistic regression analysis, survival-type analysis, and meta-analysis. RESULTS: Analysed meetings were held between 1957 and 1999. Of 14945 abstracts that were submitted to 43 meetings, 46% were accepted. The rate of full publication was studied with 19123 abstracts that were presented at 234 meetings. Using survival-type analysis, we estimated that 27% were published after two, 41% after four, and 44% after six years. Of 2412 abstracts that were rejected at 24 meetings, 27% were published despite rejection. Factors associated with both abstract acceptance and subsequent publication were basic science and positive study outcome. Large meetings and those held outside the US were more likely to accept abstracts. Abstracts were more likely to be published subsequently if presented either orally, at small meetings, or at a US meeting. Abstract acceptance itself was strongly associated with full publication. CONCLUSIONS: About one third of abstracts submitted to biomedical meetings were published as full reports. Acceptance at meetings and publication were associated with specific characteristics of abstracts and meetings

Consequences of Cold-Ischemia Time on Primary Nonfunction and Patient and Graft Survival in Liver Transplantation: A Meta-Analysis

Introduction: The ability to preserve organs prior to transplant is essential to the organ allocation process. Objective: The purpose of this study is to describe the functional relationship between cold-ischemia time (CIT) and primary nonfunction (PNF), patient and graft survival in liver transplant. Methods: To identify relevant articles Medline, EMBASE and the Cochrane database, including the non-English literature identified in these databases, was searched from 1966 to April 2008. Two independent reviewers screened and extracted the data. CIT was analyzed both as a continuous variable and stratified by clinically relevant intervals. Nondichotomous variables were weighted by sample size. Percent variables were weighted by the inverse of the binomial variance. Results: Twenty-six studies met criteria. Functionally, PNF%=-6.678281+0.9134701*CIT Mean+0.1250879*(CIT Mean-9.89535) 2 - 0.0067663*(CIT Mean-9.89535) 3, r2=.625, p<.0001. Mean patient survival: 93 % (1 month), 88 % (3 months), 83 % (6 months) and 83 % (12 months). Mean graft survival: 85.9 % (1 month), 80.5 % (3 months), 78.1 % (6 months) and 76.8 % (12 months). Maximum patient and graft survival occurred with CITs between 7.5-12.5 hrs at each survival interval. PNF was also significantly correlated with ICU time, % first time grafts and % immunologic mismatches. Conclusion: The results of this work imply that CIT may be the most important pre-transplant information needed in the decision to accept an organ. © 2008 Stahl et al

A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants

Background: This paper presents the first meta-analysis for the inter-rater reliability (IRR) of journal peer reviews. IRR is defined as the extent to which two or more independent reviews of the same scientific document agree. Methodology/Principal Findings: Altogether, 70 reliability coefficients (Cohen’s Kappa, intra-class correlation [ICC], and Pearson product-moment correlation [r]) from 48 studies were taken into account in the meta-analysis. The studies were based on a total of 19,443 manuscripts; on average, each study had a sample size of 311 manuscripts (minimum: 28, maximum: 1983). The results of the meta-analysis confirmed the findings of the narrative literature reviews published to date: The level of IRR (mean ICC/r 2 =.34, mean Cohen’s Kappa =.17) was low. To explain the study-to-study variation of the IRR coefficients, meta-regression analyses were calculated using seven covariates. Two covariates that emerged in the metaregression analyses as statistically significant to gain an approximate homogeneity of the intra-class correlations indicated that, firstly, the more manuscripts that a study is based on, the smaller the reported IRR coefficients are. Secondly, if the information of the rating system for reviewers was reported in a study, then this was associated with a smaller IRR coefficient than if the information was not conveyed. Conclusions/Significance: Studies that report a high level of IRR are to be considered less credible than those with a low level o

Neural Circuits Underlying Rodent Sociality: A Comparative Approach

All mammals begin life in social groups, but for some species, social relationships persist and develop throughout the course of an individual’s life. Research in multiple rodent species provides evidence of relatively conserved circuitry underlying social behaviors and processes such as social recognition and memory, social reward, and social approach/avoidance. Species exhibiting different complex social behaviors and social systems (such as social monogamy or familiarity preferences) can be characterized in part by when and how they display specific social behaviors. Prairie and meadow voles are closely related species that exhibit similarly selective peer preferences but different mating systems, aiding direct comparison of the mechanisms underlying affiliative behavior. This chapter draws on research in voles as well as other rodents to explore the mechanisms involved in individual social behavior processes, as well as specific complex social patterns. Contrasts between vole species exemplify how the laboratory study of diverse species improves our understanding of the mechanisms underlying social behavior. We identify several additional rodent species whose interesting social structures and available ecological and behavioral field data make them good candidates for study. New techniques and integration across laboratory and field settings will provide exciting opportunities for future mechanistic work in non-model species