Systematic review and stratified meta-analysis of the efficacy of RhoA and Rho kinase inhibitors in animal models of ischaemic stroke

There is currently only one clinically approved drug, tissue plasminogen activator (tPA), for the treatment of acute ischaemic stroke. The RhoA pathway, including RhoA and its downstream effector Rho kinase (ROCK), has been identified as a possible therapeutic target. Our aim was to assess the impact of study design characteristics and study quality on reported measures of efficacy and to assess for the presence and impact of publication bias. We conducted a systematic review and meta-analysis on publications describing the efficacy of RhoA and ROCK inhibitors in animal models of focal cerebral ischaemia where outcome was assessed as a change in lesion size or neurobehavioural score, or both. We identified 25 published papers which met our inclusion criteria. RhoA and ROCK inhibitors reduced lesion size by 37.3% in models of focal cerebral ischaemia (95% CI, 28.6% to 46.0%, 41 comparisons), and reduced neurobehavioural data by 40.5% (33.4% to 47.7%, 30 comparisons). Overall study quality was low (median=4, interquartile range 3-5) and measures to reduce bias were seldom reported. Publication bias was prevalent and associated with a substantial overstatement of efficacy for lesion size. RhoA and ROCK inhibitors appear to be effective in animal models of stroke. However the low quality score, publication bias and limited number of studies are areas which need attention prior to conducting clinical trials

A protocol for the systematic review and meta-analysis of thigmotactic behaviour in the open field test in rodent models associated with persistent pain

ObjectiveThigmotaxis is an innate predator avoidance behaviour of rodents and is enhanced when animals are under stress. It is characterised by the preference of a rodent to seek shelter, rather than expose itself to the aversive open area. The behaviour has been proposed to be a measurable construct that can address the impact of pain on rodent behaviour. This systematic review will assess whether thigmotaxis can be influenced by experimental persistent pain and attenuated by pharmacological interventions in rodents.Search strategyWe will conduct search on three electronic databases to identify studies in which thigmotaxis was used as an outcome measure contextualised to a rodent model associated with persistent pain. All studies published until the date of the search will be considered.Screening and annotationTwo independent reviewers will screen studies based on the order of (1) titles and abstracts, and (2) full texts.Data management and reportingFor meta-analysis, we will extract thigmotactic behavioural data and calculate effect sizes. Effect sizes will be combined using a random-effects model. We will assess heterogeneity and identify sources of heterogeneity. A risk-of-bias assessment will be conducted to evaluate study quality. Publication bias will be assessed using funnel plots, Egger’s regression and trim-and-fill analysis. We will also extract stimulus-evoked limb withdrawal data to assess its correlation with thigmotaxis in the same animals. The evidence obtained will provide a comprehensive understanding of the strengths and limitations of using thigmotactic outcome measure in animal pain research so that future experimental designs can be optimised. We will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guidelines and disseminate the review findings through publication and conference presentation.</jats:sec

Reprint: Good laboratory practice: preventing introduction of bias at the bench

As a research community, we have failed to show that drugs, which show substantial efficacy in animal models of cerebral ischemia, can also improve outcome in human stroke. Accumulating evidence suggests this may be due, at least in part, to problems in the design, conduct, and reporting of animal experiments which create a systematic bias resulting in the overstatement of neuroprotective efficacy. Here, we set out a series of measures to reduce bias in the design, conduct and reporting of animal experiments modeling human stroke

Two Years Later: Journals Are Not Yet Enforcing the ARRIVE Guidelines on Reporting Standards for Pre-Clinical Animal Studies

There is growing concern that poor experimental design and lack of transparent reporting contribute to the frequent failure of pre-clinical animal studies to translate into treatments for human disease. In 2010, the Animal Research: Reporting of In Vivo Experiments (ARRIVE) guidelines were introduced to help improve reporting standards. They were published in PLOS Biology and endorsed by funding agencies and publishers and their journals, including PLOS, Nature research journals, and other top-tier journals. Yet our analysis of papers published in PLOS and Nature journals indicates that there has been very little improvement in reporting standards since then. This suggests that authors, referees, and editors generally are ignoring guidelines, and the editorial endorsement is yet to be effectively implemented

Retrospective harm benefit analysis of pre-clinical animal research for six treatment interventions

The harm benefit analysis (HBA) is the cornerstone of animal research regulation and is considered to be a key ethical safeguard for animals. The HBA involves weighing the anticipated benefits of animal research against its predicted harms to animals but there are doubts about how objective and accountable this process is.i. To explore the harms to animals involved in pre-clinical animal studies and to assess these against the benefits for humans accruing from these studies; ii. To test the feasibility of conducting this type of retrospective HBA.Data on harms were systematically extracted from a sample of pre-clinical animal studies whose clinical relevance had already been investigated by comparing systematic reviews of the animal studies with systematic reviews of human studies for the same interventions (antifibrinolytics for haemorrhage, bisphosphonates for osteoporosis, corticosteroids for brain injury, Tirilazad for stroke, antenatal corticosteroids for neonatal respiratory distress and thrombolytics for stroke). Clinical relevance was also explored in terms of current clinical practice. Harms were categorised for severity using an expert panel. The quality of the research and its impact were considered. Bateson's Cube was used to conduct the HBA.The most common assessment of animal harms by the expert panel was 'severe'. Reported use of analgesia was rare and some animals (including most neonates) endured significant procedures with no, or only light, anaesthesia reported. Some animals suffered iatrogenic harms. Many were kept alive for long periods post-experimentally but only 1% of studies reported post-operative care. A third of studies reported that some animals died prior to endpoints. All the studies were of poor quality. Having weighed the actual harms to animals against the actual clinical benefits accruing from these studies, and taking into account the quality of the research and its impact, less than 7% of the studies were permissible according to Bateson's Cube: only the moderate bisphosphonate studies appeared to minimise harms to animals whilst being associated with benefit for humans.This is the first time the accountability of the HBA has been systematically explored across a range of pre-clinical animal studies. The regulatory systems in place when these studies were conducted failed to safeguard animals from severe suffering or to ensure that only beneficial, scientifically rigorous research was conducted. Our findings indicate a pressing need to: i. review regulations, particularly those that permit animals to suffer severe harms; ii. reform the processes of prospectively assessing pre-clinical animal studies to make them fit for purpose; and iii. systematically evaluate the benefits of pre-clinical animal research to permit a more realistic assessment of its likely future benefits

Reproducibility of preclinical animal research improves with heterogeneity of study samples

Single-laboratory studies conducted under highly standardized conditions are the gold standard in preclinical animal research. Using simulations based on 440 preclinical studies across 13 different interventions in animal models of stroke, myocardial infarction, and breast cancer, we compared the accuracy of effect size estimates between single-laboratory and multi-laboratory study designs. Single-laboratory studies generally failed to predict effect size accurately, and larger sample sizes rendered effect size estimates even less accurate. By contrast, multi-laboratory designs including as few as 2 to 4 laboratories increased coverage probability by up to 42 percentage points without a need for larger sample sizes. These findings demonstrate that within-study standardization is a major cause of poor reproducibility. More representative study samples are required to improve the external validity and reproducibility of preclinical animal research and to prevent wasting animals and resources for inconclusive research

Resolving the Sources of Plasma Glucose Excursions following a Glucose Tolerance Test in the Rat with Deuterated Water and [U-13C]Glucose

Sources of plasma glucose excursions (PGE) following a glucose tolerance test enriched with [U-13C]glucose and deuterated water were directly resolved by 13C and 2H Nuclear Magnetic Resonance spectroscopy analysis of plasma glucose and water enrichments in rat. Plasma water 2H-enrichment attained isotopic steady-state within 2–4 minutes following the load. The fraction of PGE derived from endogenous sources was determined from the ratio of plasma glucose position 2 and plasma water 2H-enrichments. The fractional gluconeogenic contributions to PGE were obtained from plasma glucose positions 2 and 5 2H-positional enrichment ratios and load contributions were estimated from plasma [U-13C]glucose enrichments. At 15 minutes, the load contributed 26±5% of PGE while 14±2% originated from gluconeogenesis in healthy control rats. Between 15 and 120 minutes, the load contribution fell whereas the gluconeogenic contribution remained constant. High-fat fed animals had significant higher 120-minute blood glucose (173±6 mg/dL vs. 139±10 mg/dL, p<0.05) and gluconeogenic contributions to PGE (59±5 mg/dL vs. 38±3 mg/dL, p<0.01) relative to standard chow-fed controls. In summary, the endogenous and load components of PGE can be resolved during a glucose tolerance test and these measurements revealed that plasma glucose synthesis via gluconeogenesis remained active during the period immediately following a glucose load. In rats that were placed on high-fat diet, the development of glucose intolerance was associated with a significantly higher gluconeogenic contribution to plasma glucose levels after the load

Machine learning algorithms for systematic review: reducing workload in a preclinical review of animal studies and reducing human screening error

BACKGROUND: Here, we outline a method of applying existing machine learning (ML) approaches to aid citation screening in an on-going broad and shallow systematic review of preclinical animal studies. The aim is to achieve a high-performing algorithm comparable to human screening that can reduce human resources required for carrying out this step of a systematic review. METHODS: We applied ML approaches to a broad systematic review of animal models of depression at the citation screening stage. We tested two independently developed ML approaches which used different classification models and feature sets. We recorded the performance of the ML approaches on an unseen validation set of papers using sensitivity, specificity and accuracy. We aimed to achieve 95% sensitivity and to maximise specificity. The classification model providing the most accurate predictions was applied to the remaining unseen records in the dataset and will be used in the next stage of the preclinical biomedical sciences systematic review. We used a cross-validation technique to assign ML inclusion likelihood scores to the human screened records, to identify potential errors made during the human screening process (error analysis). RESULTS: ML approaches reached 98.7% sensitivity based on learning from a training set of 5749 records, with an inclusion prevalence of 13.2%. The highest level of specificity reached was 86%. Performance was assessed on an independent validation dataset. Human errors in the training and validation sets were successfully identified using the assigned inclusion likelihood from the ML model to highlight discrepancies. Training the ML algorithm on the corrected dataset improved the specificity of the algorithm without compromising sensitivity. Error analysis correction leads to a 3% improvement in sensitivity and specificity, which increases precision and accuracy of the ML algorithm. CONCLUSIONS: This work has confirmed the performance and application of ML algorithms for screening in systematic reviews of preclinical animal studies. It has highlighted the novel use of ML algorithms to identify human error. This needs to be confirmed in other reviews with different inclusion prevalence levels, but represents a promising approach to integrating human decisions and automation in systematic review methodology

Evaluation of Excess Significance Bias in Animal Studies of Neurological Diseases

Animal studies generate valuable hypotheses that lead to the conduct of preventive or therapeutic clinical trials. We assessed whether there is evidence for excess statistical significance in results of animal studies on neurological disorders, suggesting biases. We used data from meta-analyses of interventions deposited in Collaborative Approach to Meta-Analysis and Review of Animal Data in Experimental Studies (CAMARADES). The number of observed studies with statistically significant results (O) was compared with the expected number (E), based on the statistical power of each study under different assumptions for the plausible effect size. We assessed 4,445 datasets synthesized in 160 meta-analyses on Alzheimer disease (n = 2), experimental autoimmune encephalomyelitis (n = 34), focal ischemia (n = 16), intracerebral hemorrhage (n = 61), Parkinson disease (n = 45), and spinal cord injury (n = 2). 112 meta-analyses (70%) found nominally (p≤0.05) statistically significant summary fixed effects. Assuming the effect size in the most precise study to be a plausible effect, 919 out of 4,445 nominally significant results were expected versus 1,719 observed (p<10-9). Excess significance was present across all neurological disorders, in all subgroups defined by methodological characteristics, and also according to alternative plausible effects. Asymmetry tests also showed evidence of small-study effects in 74 (46%) meta-analyses. Significantly effective interventions with more than 500 animals, and no hints of bias were seen in eight (5%) meta-analyses. Overall, there are too many animal studies with statistically significant results in the literature of neurological disorders. This observation suggests strong biases, with selective analysis and outcome reporting biases being plausible explanations, and provides novel evidence on how these biases might influence the whole research domain of neurological animal literature. © 2013 Tsilidis et al