NIPS - Not Even Wrong? A Systematic Review of Empirically Complete Demonstrations of Algorithmic Effectiveness in the Machine Learning and Artificial Intelligence Literature

Objective: To determine the completeness of argumentative steps necessary to conclude effectiveness of an algorithm in a sample of current ML/AI supervised learning literature. Data Sources: Papers published in the Neural Information Processing Systems (NeurIPS, n\'ee NIPS) journal where the official record showed a 2017 year of publication. Eligibility Criteria: Studies reporting a (semi-)supervised model, or pre-processing fused with (semi-)supervised models for tabular data. Study Appraisal: Three reviewers applied the assessment criteria to determine argumentative completeness. The criteria were split into three groups, including: experiments (e.g real and/or synthetic data), baselines (e.g uninformed and/or state-of-art) and quantitative comparison (e.g. performance quantifiers with confidence intervals and formal comparison of the algorithm against baselines). Results: Of the 121 eligible manuscripts (from the sample of 679 abstracts), 99\% used real-world data and 29\% used synthetic data. 91\% of manuscripts did not report an uninformed baseline and 55\% reported a state-of-art baseline. 32\% reported confidence intervals for performance but none provided references or exposition for how these were calculated. 3\% reported formal comparisons. Limitations: The use of one journal as the primary information source may not be representative of all ML/AI literature. However, the NeurIPS conference is recognised to be amongst the top tier concerning ML/AI studies, so it is reasonable to consider its corpus to be representative of high-quality research. Conclusion: Using the 2017 sample of the NeurIPS supervised learning corpus as an indicator for the quality and trustworthiness of current ML/AI research, it appears that complete argumentative chains in demonstrations of algorithmic effectiveness are rare

Machine Learning in Falls Prediction; A cognition-based predictor of falls for the acute neurological in-patient population

Background Information: Falls are associated with high direct and indirect costs, and significant morbidity and mortality for patients. Pathological falls are usually a result of a compromised motor system, and/or cognition. Very little research has been conducted on predicting falls based on this premise. Aims: To demonstrate that cognitive and motor tests can be used to create a robust predictive tool for falls. Methods: Three tests of attention and executive function (Stroop, Trail Making, and Semantic Fluency), a measure of physical function (Walk-12), a series of questions (concerning recent falls, surgery and physical function) and demographic information were collected from a cohort of 323 patients at a tertiary neurological center. The principal outcome was a fall during the in-patient stay (n = 54). Data-driven, predictive modelling was employed to identify the statistical modelling strategies which are most accurate in predicting falls, and which yield the most parsimonious models of clinical relevance. Results: The Trail test was identified as the best predictor of falls. Moreover, addition of any others variables, to the results of the Trail test did not improve the prediction (Wilcoxon signed-rank p < .001). The best statistical strategy for predicting falls was the random forest (Wilcoxon signed-rank p < .001), based solely on results of the Trail test. Tuning of the model results in the following optimized values: 68% (+- 7.7) sensitivity, 90% (+- 2.3) specificity, with a positive predictive value of 60%, when the relevant data is available. Conclusion: Predictive modelling has identified a simple yet powerful machine learning prediction strategy based on a single clinical test, the Trail test. Predictive evaluation shows this strategy to be robust, suggesting predictive modelling and machine learning as the standard for future predictive tools

A Methodological Framework for Assessing the Benefit of SARS-CoV-2 Vaccination following Previous Infection: Case Study of Five- to Eleven-Year-Olds

Vaccination rates against SARS-CoV-2 in children aged five to eleven years remain low in many countries. The current benefit of vaccination in this age group has been questioned given that the large majority of children have now experienced at least one SARS-CoV-2 infection. However, protection from infection, vaccination or both wanes over time. National decisions on offering vaccines to this age group have tended to be made without considering time since infection. There is an urgent need to evaluate the additional benefits of vaccination in previously infected children and under what circumstances those benefits accrue. We present a novel methodological framework for estimating the potential benefits of COVID-19 vaccination in previously infected children aged five to eleven, accounting for waning. We apply this framework to the UK context and for two adverse outcomes: hospitalisation related to SARS-CoV-2 infection and Long Covid. We show that the most important drivers of benefit are: the degree of protection provided by previous infection; the protection provided by vaccination; the time since previous infection; and future attack rates. Vaccination can be very beneficial for previously infected children if future attack rates are high and several months have elapsed since the previous major wave in this group. Benefits are generally larger for Long Covid than hospitalisation, because Long Covid is both more common than hospitalisation and previous infection offers less protection against it. Our framework provides a structure for policy makers to explore the additional benefit of vaccination across a range of adverse outcomes and different parameter assumptions. It can be easily updated as new evidence emerges

Developing vocational rehabilitation services for people with long-term neurological conditions : identifying facilitators and barriers to service provision

Purpose: This study aimed to understand existing vocational rehabilitation (VR) service provision in one locality in London (population 3.74 million), identify any gaps and explore reasons for this, to support service development. Method: Using Soft Systems Methodology to guide the research process, semi - structured interviews were completed with nine participants, who were clinicians and managers providing VR within NHS services. Data were analysed thematically to build a ‘rich picture’ and develop a conceptual model of VR service delivery. Findings were then ratified with participants at an engagement event. Results: The findings indicate a spectrum of VR service provision for long - term neurological conditions with differing levels of funding in place. VR often takes place ‘under the radar’ and therefore the true VR needs of this population, and the extent of service provision is not known. There is inconsistency of understanding across the services as to what constitutes VR and outcomes are not routinely measured. Conclusion: For VR services to develop they require appropriate funding, driven by Government policy to commissioners. Clear definitions of VR, collecting and sharing outcome data and effective communication across services are needed at a local level. This is expressed in a conceptual model of VR service delivery

DNA Methylation: Basic Biology and Application to Traumatic Brain Injury.

This article reviews the literature pertinent to epigenetic changes, and in particular, DNA methylation following traumatic brain injury (TBI). TBI is a heterogeneous disease that is a major cause of death and long-term disability. The links between TBI and epigenetics, the process by which environmental factors alter gene expression without changing the underlying DNA sequence, is an expanding area of research that may have profound consequences for understanding the disease, and for clinical care. There are various epigenetic changes that may occur as a direct result of TBI, including DNA methylation, histone modification, and changes in the levels of non-coding RNA. This review focuses on DNA methylation, its potential to alter the degree of injury, and the extent of recovery, including development of post-traumatic neurodegeneration, response to therapies, and the hereditable consequences of injury. The functional consequences of non-coding RNA and histone modifications are well described in the literature; however, the mechanism by which these three mechanisms interact are often overlooked. Here, we briefly describe the interaction of DNA methylation with the two other key epigenetic changes, and highlight key work being performed to understand the functional relevance of those mechanisms. The field of epigenetics is rapidly advancing as a result of the advent of less invasive and more versatile methods for measuring epigenetic proteins and their functional impact on cells; however, the evidence specific to TBI is limited. This review identifies several important outstanding questions that remain from the work already conducted, and highlights directions for the future

Systematic review of health-related work outcome measures and quality criteria-based evaluations of their psychometric properties

Objective To examine the state of psychometric validation in the health-related work outcome literature. Data Sources We searched PubMed, PubMed Central, CINAHL, Embase (plus Embase Classic), and PsycINFO from inception to January 2016 using the following search terms: stroke, multiple sclerosis, epilepsy, spinal cord injury, brain injury, musculoskeletal disease, work, absenteeism, presenteeism, occupation, employment, job, outcome measure, assessment, work capacity evaluation, scale, and questionnaire. Study Selection From the 22,676 retrieved abstracts, 597 outcome measures were identified. Inclusion was based on content analysis. There were 95 health-related work outcome measures retained; of these, 2 were treated as outliers and therefore are discussed separately. All 6 authors individually organized the 93 remaining scales based on their content. Data Extraction A follow-up search using the same sources, and time period, with the name of the outcome measures and the terms psychometric, reliability, validity, and responsiveness, identified 263 unique classical test theory psychometric property datasets for the 93 tools. An assessment criterion for psychometric properties was applied to each article, and where consensus was not achieved, the rating delivered by most of the assessors was reported. Data Synthesis Of the articles reported, 18 reporting psychometric data were not accessible and therefore could not be assessed. There were 39 that scored 80%. The 3 outcome measures associated with the highest scoring datasets were the Sheehan Disability Scale, the Fear Avoidance Beliefs Questionnaire, and the assessment of the Subjective Handicap of Epilepsy. Finally, only 2 psychometric validation datasets reported the complete set of baseline psychometric properties. Conclusions This systematic review highlights the current limitations of the health-related work outcome measure literature, including the limited number of robust tools available

The Trail Making test : a study of its ability to predict falls in the acute neurological in-patient population

Objective: To determine whether tests of cognitive function and patient-reported outcome measures of motor function can be used to create a machine learning-based predictive tool for falls. Design: Prospective cohort study. Setting: Tertiary neurological and neurosurgical center. Subjects: In all, 337 in-patients receiving neurosurgical, neurological, or neurorehabilitation-based care. Main Measures: Binary (Y/N) for falling during the in-patient episode, the Trail Making Test (a measure of attention and executive function) and the Walk-12 (a patient-reported measure of physical function). Results: The principal outcome was a fall during the in-patient stay (n = 54). The Trail test was identified as the best predictor of falls. Moreover, addition of other variables, did not improve the prediction (Wilcoxon signed-rank P < 0.001). Classical linear statistical modeling methods were then compared with more recent machine learning based strategies, for example, random forests, neural networks, support vector machines. The random forest was the best modeling strategy when utilizing just the Trail Making Test data (Wilcoxon signed-rank P < 0.001) with 68% (± 7.7) sensitivity, and 90% (± 2.3) specificity. Conclusion: This study identifies a simple yet powerful machine learning (Random Forest) based predictive model for an in-patient neurological population, utilizing a single neuropsychological test of cognitive function, the Trail Making test

A methodological framework for assessing the benefit of SARS-CoV-2 vaccination following previous infection : case study of five- to eleven-year-olds

Vaccination rates against SARS-CoV-2 in children aged five to eleven years remain low in many countries. The current benefit of vaccination in this age group has been questioned given that the large majority of children have now experienced at least one SARS-CoV-2 infection. However, protection from infection, vaccination or both wanes over time. National decisions on offering vaccines to this age group have tended to be made without considering time since infection. There is an urgent need to evaluate the additional benefits of vaccination in previously infected children and under what circumstances those benefits accrue. We present a novel methodological framework for estimating the potential benefits of COVID-19 vaccination in previously infected children aged five to eleven, accounting for waning. We apply this framework to the UK context and for two adverse outcomes: hospitalisation related to SARS-CoV-2 infection and Long Covid. We show that the most important drivers of benefit are: the degree of protection provided by previous infection; the protection provided by vaccination; the time since previous infection; and future attack rates. Vaccination can be very beneficial for previously infected children if future attack rates are high and several months have elapsed since the previous major wave in this group. Benefits are generally larger for Long Covid than hospitalisation, because Long Covid is both more common than hospitalisation and previous infection offers less protection against it. Our framework provides a structure for policy makers to explore the additional benefit of vaccination across a range of adverse outcomes and different parameter assumptions. It can be easily updated as new evidence emerges