Ordinal Convolutional Neural Networks for Predicting RDoC Positive Valence Psychiatric Symptom Severity Scores

Background—The CEGS N-GRID 2016 Shared Task in Clinical Natural Language Processing (NLP) provided a set of 1000 neuropsychiatric notes to participants as part of a competition to predict psychiatric symptom severity scores. This paper summarizes our methods, results, and experiences based on our participation in the second track of the shared task. Objective—Classical methods of text classification usually fall into one of three problem types: binary, multi-class, and multi-label classification. In this effort, we study ordinal regression problems with text data where misclassifications are penalized differently based on how far apart the ground truth and model predictions are on the ordinal scale. Specifically, we present our entries (methods and results) in the N-GRID shared task in predicting research domain criteria (RDoC) positive valence ordinal symptom severity scores (absent, mild, moderate, and severe) from psychiatric notes. Methods—We propose a novel convolutional neural network (CNN) model designed to handle ordinal regression tasks on psychiatric notes. Broadly speaking, our model combines an ordinal loss function, a CNN, and conventional feature engineering (wide features) into a single model which is learned end-to-end. Given interpretability is an important concern with nonlinear models, we apply a recent approach called locally interpretable model-agnostic explanation (LIME) to identify important words that lead to instance specific predictions. Results—Our best model entered into the shared task placed third among 24 teams and scored a macro mean absolute error (MMAE) based normalized score (100 · (1 − M M AE)) of 83.86. Since the competition, we improved our score (using basic ensembling) to 85.55, comparable with the winning shared task entry. Applying LIME to model predictions, we demonstrate the feasibility of instance specific prediction interpretation by identifying words that led to a particular decision

The IUCN Red List of Ecosystems: motivations, challenges, and applications

Abstract In response to growing demand for ecosystem-level risk assessment in biodiversity conservation, and rapid proliferation of locally tailored protocols, the IUCN recently endorsed new Red List criteria as a global standard for ecosystem risk assessment. Four qualities were sought in the design of the IUCN criteria: generality; precision; realism; and simplicity. Drawing from extensive global consultation, we explore trade-offs among these qualities when dealing with key challenges, including ecosystem classification, measuring ecosystem dynamics, degradation and collapse, and setting decision thresholds to delimit ordinal categories of threat. Experience from countries with national lists of threatened ecosystems demonstrates well-balanced trade-offs in current and potential applications of Red Lists of Ecosystems in legislation, policy, environmental management and education. The IUCN Red List of Ecosystems should be judged by whether it achieves conservation ends and improves natural resource management, whether its limitations are outweighed by its benefits, and whether it performs better than alternative methods. Future development of the Red List of Ecosystems will benefit from the history of the Red List of Threatened Species which was trialed and adjusted iteratively over 50 years from rudimentary beginnings. We anticipate the Red List of Ecosystems will promote policy focus on conservation outcomes in situ across whole landscapes and seascapes

Unravelling black box machine learning methods using biplots

Following the development of new mathematical techniques, the improvement of computer processing power and the increased availability of possible explanatory variables, the financial services industry is moving toward the use of new machine learning methods, such as neural networks, and away from older methods such as generalised linear models. However, their use is currently limited because they are seen as “black box” models, which gives predictions without justifications and which are therefore not understood and cannot be trusted. The goal of this dissertation is to expand on the theory and use of biplots to visualise the impact of the various input factors on the output of the machine learning black box. Biplots are used because they give an optimal two-dimensional representation of the data set on which the machine learning model is based.The biplot allows every point on the biplot plane to be converted back to the original ��-dimensions – in the same format as is used by the machine learning model. This allows the output of the model to be represented by colour coding each point on the biplot plane according to the output of an independently calibrated machine learning model. The interaction of the changing prediction probabilities – represented by the coloured output – in relation to the data points and the variable axes and category level points represented on the biplot, allows the machine learning model to be globally and locally interpreted. By visualing the models and their predictions, this dissertation aims to remove the stigma of calling non-linear models “black box” models and encourage their wider application in the financial services industry

What is a competent 'competence standard'? Tensions between the construct and assessment as a tool for learning

Using the UK's recent disability legislation as a trigger, the paper explores issues in evidencing competence in the current context of using assessment as a tool for learning. In the current context of requiring assessment to be edumetrically sound, the legislation of competence standards is problematic in four respects. Task Validity is now a much more diffuse concept. Scoring Validity has to contend with many possible accommodations. Assessment Generalisability has to consider both the relevance and representativeness of the assessment task. Consequential Validity is essentially concerned with formative assessment; which is considered pedagogically important but, politically, is of less significance than summative assessment. This study offers academics and administrators a framework within which to review the edumetric soundness of their assessment practices and policies. In so doing possible difficulties in equitable assessment can be made explicit. This, in turn, has implications for staff development

Bioinformatics and Medicine in the Era of Deep Learning

Many of the current scientific advances in the life sciences have their origin in the intensive use of data for knowledge discovery. In no area this is so clear as in bioinformatics, led by technological breakthroughs in data acquisition technologies. It has been argued that bioinformatics could quickly become the field of research generating the largest data repositories, beating other data-intensive areas such as high-energy physics or astroinformatics. Over the last decade, deep learning has become a disruptive advance in machine learning, giving new live to the long-standing connectionist paradigm in artificial intelligence. Deep learning methods are ideally suited to large-scale data and, therefore, they should be ideally suited to knowledge discovery in bioinformatics and biomedicine at large. In this brief paper, we review key aspects of the application of deep learning in bioinformatics and medicine, drawing from the themes covered by the contributions to an ESANN 2018 special session devoted to this topic