Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcity

We investigate the problem of determining the predictive confidence (or, conversely, uncertainty) of a neural classifier through the lens of low-resource languages. By training models on sub-sampled datasets in three different languages, we assess the quality of estimates from a wide array of approaches and their dependence on the amount of available data. We find that while approaches based on pre-trained models and ensembles achieve the best results overall, the quality of uncertainty estimates can surprisingly suffer with more data. We also perform a qualitative analysis of uncertainties on sequences, discovering that a model's total uncertainty seems to be influenced to a large degree by its data uncertainty, not model uncertainty. All model implementations are open-sourced in a software package

deep-significance - Easy and Meaningful Statistical Significance Testing in the Age of Neural Networks

A lot of Machine Learning (ML) and Deep Learning (DL) research is of an empirical nature. Nevertheless, statistical significance testing (SST) is still not widely used. This endangers true progress, as seeming improvements over a baseline might be statistical flukes, leading follow-up research astray while wasting human and computational resources. Here, we provide an easy-to-use package containing different significance tests and utility functions specifically tailored towards research needs and usability

Mechanistic distinctions between agrin and laminin-1 induced aggregation of acetylcholine receptors

BACKGROUND: One of the earliest steps in synaptogenesis at the neuromuscular junction is the aggregation of nicotinic acetylcholine receptors at the postsynaptic membrane. This study presents quantitative analyses of receptor and α-Dystroglycan aggregation in response to agrin and laminin-1, alone or in combination. RESULTS: Both laminin and agrin increased overall expression of receptors on the plasma membrane. Following a 24 hour exposure, agrin increased the number of receptor aggregates but did not affect the number of α-Dystroglycan aggregates, while the reverse was true of laminin-1. Laminin also increased receptor concentration within aggregates, while agrin had no such effect. Finally, the spatial distribution of aggregates was indistinguishable from random in the case of laminin, while agrin induced aggregates were closer together than predicted by a random model. CONCLUSIONS: Agrin and laminin-1 both increase acetylcholine receptor aggregate size after 24 hours, but several lines of evidence indicate that this is achieved via different mechanisms. Agrin and laminin had different effects on the number and density of receptor and α-Dystroglycan aggregates. Moreover the random distribution of laminin induced (as opposed to agrin induced) receptor aggregates suggests that the former may influence aggregate size by simple mass action effects due to increased receptor expression

Plan de negocio para la intermediaci?n entre restaurantes y personas que desean realizar reservas de mesa, selecci?n de platos y bebidas y pago de cuenta mediante una aplicaci?n m?vil

El presente plan de negocio busca evaluar la viabilidad comercial, operativa y econ?mica para el desarrollo de una plataforma digital denominada Appetito, la cual brindar? el servicio de intermediaci?n entre restaurantes y comensales, mediante una aplicaci?n m?vil, plataforma web y m?dulo de integraci?n con los sistemas de restaurantes, que facilitar?: el proceso de reserva, el proceso de pedido de platos de comida y bebida y el proceso de pago. Es as? que este startup o emprendimiento basado en un modelo de tipo orquestador de redes, permitir? armar grupos de participantes a un evento como almuerzo o cena, realizar reservas en restaurantes, realizar el pedido por anticipado, dividir la cuenta para que cada comensal pueda pagar lo consumido desde su propio celular y recibir la confirmaci?n en tiempo real desde el sistema del restaurante. Asimismo el restaurante podr? agilizar el proceso de reserva, pedido y cobro, generando mejoras en sus procesos operativos, como el proceso de preparaci?n de los platos, debido a la anticipaci?n de los pedidos, generando mejoras en los procesos, como mayor rotaci?n de mesas, disminuci?n del proceso de cobro de la cuenta, disminuci?n de los cuellos de botella en la cocina y mejor conocimiento y experiencia del cliente

Analysis of shared heritability in common disorders of the brain

ience, this issue p. eaap8757 Structured Abstract INTRODUCTION Brain disorders may exhibit shared symptoms and substantial epidemiological comorbidity, inciting debate about their etiologic overlap. However, detailed study of phenotypes with different ages of onset, severity, and presentation poses a considerable challenge. Recently developed heritability methods allow us to accurately measure correlation of genome-wide common variant risk between two phenotypes from pools of different individuals and assess how connected they, or at least their genetic risks, are on the genomic level. We used genome-wide association data for 265,218 patients and 784,643 control participants, as well as 17 phenotypes from a total of 1,191,588 individuals, to quantify the degree of overlap for genetic risk factors of 25 common brain disorders. RATIONALE Over the past century, the classification of brain disorders has evolved to reflect the medical and scientific communities' assessments of the presumed root causes of clinical phenomena such as behavioral change, loss of motor function, or alterations of consciousness. Directly observable phenomena (such as the presence of emboli, protein tangles, or unusual electrical activity patterns) generally define and separate neurological disorders from psychiatric disorders. Understanding the genetic underpinnings and categorical distinctions for brain disorders and related phenotypes may inform the search for their biological mechanisms. RESULTS Common variant risk for psychiatric disorders was shown to correlate significantly, especially among attention deficit hyperactivity disorder (ADHD), bipolar disorder, major depressive disorder (MDD), and schizophrenia. By contrast, neurological disorders appear more distinct from one another and from the psychiatric disorders, except for migraine, which was significantly correlated to ADHD, MDD, and Tourette syndrome. We demonstrate that, in the general population, the personality trait neuroticism is significantly correlated with almost every psychiatric disorder and migraine. We also identify significant genetic sharing between disorders and early life cognitive measures (e.g., years of education and college attainment) in the general population, demonstrating positive correlation with several psychiatric disorders (e.g., anorexia nervosa and bipolar disorder) and negative correlation with several neurological phenotypes (e.g., Alzheimer's disease and ischemic stroke), even though the latter are considered to result from specific processes that occur later in life. Extensive simulations were also performed to inform how statistical power, diagnostic misclassification, and phenotypic heterogeneity influence genetic correlations. CONCLUSION The high degree of genetic correlation among many of the psychiatric disorders adds further evidence that their current clinical boundaries do not reflect distinct underlying pathogenic processes, at least on the genetic level. This suggests a deeply interconnected nature for psychiatric disorders, in contrast to neurological disorders, and underscores the need to refine psychiatric diagnostics. Genetically informed analyses may provide important "scaffolding" to support such restructuring of psychiatric nosology, which likely requires incorporating many levels of information. By contrast, we find limited evidence for widespread common genetic risk sharing among neurological disorders or across neurological and psychiatric disorders. We show that both psychiatric and neurological disorders have robust correlations with cognitive and personality measures. Further study is needed to evaluate whether overlapping genetic contributions to psychiatric pathology may influence treatment choices. Ultimately, such developments may pave the way toward reduced heterogeneity and improved diagnosis and treatment of psychiatric disorders