Development and validation of the ACE tool: Assessing medical trainees' competency in evidence based medicine

BACKGROUND: While a variety of instruments have been developed to assess knowledge and skills in evidence based medicine (EBM), few assess all aspects of EBM - including knowledge, skills attitudes and behaviour - or have been psychometrically evaluated. The aim of this study was to develop and validate an instrument that evaluates medical trainees’ competency in EBM across knowledge, skills and attitude. METHODS: The ‘Assessing Competency in EBM’ (ACE) tool was developed by the authors, with content and face validity assessed by expert opinion. A cross-sectional sample of 342 medical trainees representing ‘novice’, ‘intermediate’ and ‘advanced’ EBM trainees were recruited to complete the ACE tool. Construct validity, item difficulty, internal reliability and item discrimination were analysed. RESULTS: We recruited 98 EBM-novice, 108 EBM-intermediate and 136 EBM-advanced participants. A statistically significant difference in the total ACE score was observed and corresponded to the level of training: on a 0-15-point test, the mean ACE scores were 8.6 for EBM-novice; 9.5 for EBM-intermediate; and 10.4 for EBM-advanced (p < 0.0001). Individual item discrimination was excellent (Item Discrimination Index ranging from 0.37 to 0.84), with internal reliability consistent across all but three items (Item Total Correlations were all positive ranging from 0.14 to 0.20). CONCLUSION: The 15-item ACE tool is a reliable and valid instrument to assess medical trainees’ competency in EBM. The ACE tool provides a novel assessment that measures user performance across the four main steps of EBM. To provide a complete suite of instruments to assess EBM competency across various patient scenarios, future refinement of the ACE instrument should include further scenarios across harm, diagnosis and prognosis

Evidence-based practice educational intervention studies: A systematic review of what is taught and how it is measured

Abstract Background Despite the established interest in evidence-based practice (EBP) as a core competence for clinicians, evidence for how best to teach and evaluate EBP remains weak. We sought to systematically assess coverage of the five EBP steps, review the outcome domains measured, and assess the properties of the instruments used in studies evaluating EBP educational interventions. Methods We conducted a systematic review of controlled studies (i.e. studies with a separate control group) which had investigated the effect of EBP educational interventions. We used citation analysis technique and tracked the forward and backward citations of the index articles (i.e. the systematic reviews and primary studies included in an overview of the effect of EBP teaching) using Web of Science until May 2017. We extracted information on intervention content (grouped into the five EBP steps), and the outcome domains assessed. We also searched the literature for published reliability and validity data of the EBP instruments used. Results Of 1831 records identified, 302 full-text articles were screened, and 85 included. Of these, 46 (54%) studies were randomised trials, 51 (60%) included postgraduate level participants, and 63 (75%) taught medical professionals. EBP Step 3 (critical appraisal) was the most frequently taught step (63 studies; 74%). Only 10 (12%) of the studies taught content which addressed all five EBP steps. Of the 85 studies, 52 (61%) evaluated EBP skills, 39 (46%) knowledge, 35 (41%) attitudes, 19 (22%) behaviours, 15 (18%) self-efficacy, and 7 (8%) measured reactions to EBP teaching delivery. Of the 24 instruments used in the included studies, 6 were high-quality (achieved ≥3 types of established validity evidence) and these were used in 14 (29%) of the 52 studies that measured EBP skills; 14 (41%) of the 39 studies that measured EBP knowledge; and 8 (26%) of the 35 studies that measured EBP attitude. Conclusions Most EBP educational interventions which have been evaluated in controlled studies focus on teaching only some of the EBP steps (predominantly critically appraisal of evidence) and did not use high-quality instruments to measure outcomes. Educational packages and instruments which address all EBP steps are needed to improve EBP teaching

Projected t-SNE for batch correction

Motivation: Low-dimensional representations of high-dimensional data are routinely employed in biomedical research to visualize, interpret and communicate results from different pipelines. In this article, we propose a novel procedure to directly estimate t-SNE embeddings that are not driven by batch effects. Without correction, interesting structure in the data can be obscured by batch effects. The proposed algorithm can therefore significantly aid visualization of high-dimensional data. Results: The proposed methods are based on linear algebra and constrained optimization, leading to efficient algorithms and fast computation in many high-dimensional settings. Results on artificial single-cell transcription profiling data show that the proposed procedure successfully removes multiple batch effects from t-SNE embeddings, while retaining fundamental information on cell types. When applied to single-cell gene expression data to investigate mouse medulloblastoma, the proposed method successfully removes batches related with mice identifiers and the date of the experiment, while preserving clusters of oligodendrocytes, astrocytes, and endothelial cells and microglia, which are expected to lie in the stroma within or adjacent to the tumours. Contact: [email protected]

In vitro techniques for the assessment of neurotoxicity.

Risk assessment is a process often divided into the following steps: a) hazard identification, b) dose-response assessment, c) exposure assessment, and d) risk characterization. Regulatory toxicity studies usually are aimed at providing data for the first two steps. Human case reports, environmental research, and in vitro studies may also be used to identify or to further characterize a toxic hazard. In this report the strengths and limitations of in vitro techniques are discussed in light of their usefulness to identify neurotoxic hazards, as well as for the subsequent dose-response assessment. Because of the complexity of the nervous system, multiple functions of individual cells, and our limited knowledge of biochemical processes involved in neurotoxicity, it is not known how well any in vitro system would recapitulate the in vivo system. Thus, it would be difficult to design an in vitro test battery to replace in vivo test systems. In vitro systems are well suited to the study of biological processes in a more isolated context and have been most successfully used to elucidate mechanisms of toxicity, identify target cells of neurotoxicity, and delineate the development and intricate cellular changes induced by neurotoxicants. Both biochemical and morphological end points can be used, but many of the end points used can be altered by pharmacological actions as well as toxicity. Therefore, for many of these end points it is difficult or impossible to set a criterion that allows one to differentiate between a pharmacological and a neurotoxic effect. For the process of risk assessment such a discrimination is central. Therefore, end points used to determine potential neurotoxicity of a compound have to be carefully selected and evaluated with respect to their potential to discriminate between an adverse neurotoxic effect and a pharmacologic effect. It is obvious that for in vitro neurotoxicity studies the primary end points that can be used are those affected through specific mechanisms of neurotoxicity. For example, in vitro systems may be useful for certain structurally defined compounds and mechanisms of toxicity, such as organophosphorus compounds and delayed neuropathy, for which target cells and the biochemical processes involved in the neurotoxicity are well known. For other compounds and the different types of neurotoxicity, a mechanism of toxicity needs to be identified first. Once identified, by either in vivo or in vitro methods, a system can be developed to detect and to evaluate predictive ability for the type of in vivo neurotoxicity produced. Therefore, in vitro tests have their greatest potential in providing information on basic mechanistic processes in order to refine specific experimental questions to be addressed in the whole animal

Technology-assisted stroke rehabilitation in Mexico: a pilot randomized trial comparing traditional therapy to circuit training in a Robot/technology-assisted therapy gym

Background Stroke rehabilitation in low- and middle-income countries, such as Mexico, is often hampered by lack of clinical resources and funding. To provide a cost-effective solution for comprehensive post-stroke rehabilitation that can alleviate the need for one-on-one physical or occupational therapy, in lower and upper extremities, we proposed and implemented a technology-assisted rehabilitation gymnasium in Chihuahua, Mexico. The Gymnasium for Robotic Rehabilitation (Robot Gym) consisted of low- and high-tech systems for upper and lower limb rehabilitation. Our hypothesis is that the Robot Gym can provide a cost- and labor-efficient alternative for post-stroke rehabilitation, while being more or as effective as traditional physical and occupational therapy approaches. Methods A typical group of stroke patients was randomly allocated to an intervention (n = 10) or a control group (n = 10). The intervention group received rehabilitation using the devices in the Robot Gym, whereas the control group (n = 10) received time-matched standard care. All of the study subjects were subjected to 24 two-hour therapy sessions over a period of 6 to 8 weeks. Several clinical assessments tests for upper and lower extremities were used to evaluate motor function pre- and post-intervention. A cost analysis was done to compare the cost effectiveness for both therapies. Results No significant differences were observed when comparing the results of the pre-intervention Mini-mental, Brunnstrom Test, and Geriatric Depression Scale Test, showing that both groups were functionally similar prior to the intervention. Although, both training groups were functionally equivalent, they had a significant age difference. The results of all of the upper extremity tests showed an improvement in function in both groups with no statistically significant differences between the groups. The Fugl-Meyer and the 10 Meters Walk lower extremity tests showed greater improvement in the intervention group compared to the control group. On the Time Up and Go Test, no statistically significant differences were observed pre- and post-intervention when comparing the control and the intervention groups. For the 6 Minute Walk Test, both groups presented a statistically significant difference pre- and post-intervention, showing progress in their performance. The robot gym therapy was more cost-effective than the traditional one-to-one therapy used during this study in that it enabled therapist to train up to 1.5 to 6 times more patients for the approximately same cost in the long term. Conclusions The results of this study showed that the patients that received therapy using the Robot Gym had enhanced functionality in the upper extremity tests similar to patients in the control group. In the lower extremity tests, the intervention patients showed more improvement than those subjected to traditional therapy. These results support that the Robot Gym can be as effective as traditional therapy for stroke patients, presenting a more cost- and labor-efficient option for countries with scarce clinical resources and funding. Trial registration ISRCTN98578807