Beyond neural scaling laws: beating power law scaling via data pruning

Widely observed neural scaling laws, in which error falls off as a power of the training set size, model size, or both, have driven substantial performance improvements in deep learning. However, these improvements through scaling alone require considerable costs in compute and energy. Here we focus on the scaling of error with dataset size and show how in theory we can break beyond power law scaling and potentially even reduce it to exponential scaling instead if we have access to a high-quality data pruning metric that ranks the order in which training examples should be discarded to achieve any pruned dataset size. We then test this improved scaling prediction with pruned dataset size empirically, and indeed observe better than power law scaling in practice on ResNets trained on CIFAR-10, SVHN, and ImageNet. Next, given the importance of finding high-quality pruning metrics, we perform the first large-scale benchmarking study of ten different data pruning metrics on ImageNet. We find most existing high performing metrics scale poorly to ImageNet, while the best are computationally intensive and require labels for every image. We therefore developed a new simple, cheap and scalable self-supervised pruning metric that demonstrates comparable performance to the best supervised metrics. Overall, our work suggests that the discovery of good data-pruning metrics may provide a viable path forward to substantially improved neural scaling laws, thereby reducing the resource costs of modern deep learning.Comment: Outstanding Paper Award @ NeurIPS 2022. Added github link to metric score

Characterization of the yeast flora on the surface of grape berries in Israel

Yeast populations were collected from the surface of berries of three grape cultivars during three seasons, from fruit set to maturity. They were studied by RAPD and ap-PCR, each with two primer pairs. In the population, identical isolates were found only rarely on 13 % of the bunches in 1997 and on 58 % of the berries in 1999. From RAPD and ap-PCR, a dendrogram with clusters of similarity was established. Eleven representatives from clusters of the white yeast dendrogram were identified by traditional methods as 10 different yeast species, one of which has not been isolated from grape berry surfaces before. The population size was smaller for Colombard than for Cabernet Sauvignon and Muscat of Alexandria berries.

Introduction to the Armed Forces & Society forum on military reserves in the “New Wars”

This is the final version. Available on open access from SAGE Publications via the DOI in this record. This Armed Forces & Society forum is dedicated to exploring recent trends in the characteristics of military reserves and of the changing character of reserve forces within the armed forces within the military, the civilian sphere, and in between them. To bring new and critical perspectives to the study of reserve forces and civil–military relations, this introduction and the five articles that follow draw on two organizing conceptual models: The first portrays reservists as transmigrants and focuses on the plural membership of reservists in the military and in civilian society and the “travel” between them. The second model focuses on the multiple formal and informal compacts (contracts, agreements, or pacts) between reservists and the military

Galanin Receptor 1 Deletion Exacerbates Hippocampal Neuronal Loss after Systemic Kainate Administration in Mice

Galanin is a neuropeptide with a wide distribution in the central and peripheral nervous systems and whose physiological effects are mediated through three G protein-coupled receptor subtypes, GalR1, GalR2, and GalR3. Several lines of evidence indicate that galanin, as well as activation of the GalR1 receptor, is a potent and effective modulator of neuronal excitability in the hippocampus.In order to test more formally the potential influence of GalR1 on seizure-induced excitotoxic cell death, we conducted functional complementation tests in which transgenic mice that exhibit decreased expression of the GalR1 candidate mRNA underwent kainate-induced status epilepticus to determine if the quantitative trait of susceptibility to seizure-induced cell death is determined by the activity of GalR1. In the present study, we report that reduction of GalR1 mRNA via null mutation or injection of the GalR1 antagonist, galantide, prior to kainate-induced status epilepticus induces hippocampal damage in a mouse strain known to be highly resistant to kainate-induced neuronal injury. Wild-type and GalR1 knockout mice were subjected to systemic kainate administration. Seven days later, Nissl and NeuN immune- staining demonstrated that hippocampal cell death was significantly increased in GalR1 knockout strains and in animals injected with the GalR1 antagonist. Compared to GalR1-expressing mice, GalR1-deficient mice had significantly larger hippocampal lesions after status epilepticus.Our results suggest that a reduction of GalR1 expression in the C57BL/6J mouse strain renders them susceptible to excitotoxic injury following systemic kainate administration. From these results, GalR1 protein emerges as a new molecular target that may have a potential therapeutic value in modulating seizure-induced cell death

Improving BDD Based Symbolic Model Checking with Isomorphism Exploiting Transition Relations

Symbolic model checking by using BDDs has greatly improved the applicability of model checking. Nevertheless, BDD based symbolic model checking can still be very memory and time consuming. One main reason is the complex transition relation of systems. Sometimes, it is even not possible to generate the transition relation, due to its exhaustive memory requirements. To diminish this problem, the use of partitioned transition relations has been proposed. However, there are still systems which can not be verified at all. Furthermore, if the granularity of the partitions is too fine, the time required for verification may increase. In this paper we target the symbolic verification of asynchronous concurrent systems. For such systems we present an approach which uses similarities in the transition relation to get further memory reductions and runtime improvements. By applying our approach, even the verification of systems with an previously intractable transition relation becomes feasible.Comment: In Proceedings GandALF 2011, arXiv:1106.081

Insights into GABA receptor signalling in TM3 Leydig cells

gamma-Aminobutyric acid (GABA) is an emerging signalling molecule in endocrine organs, since it is produced by endocrine cells and acts via GABA(A) receptors in a paracrine/autocrine fashion. Testicular Leydig cells are producers and targets for GABA. These cells express GABA(A) receptor subunits and in the murine Leydig cell line TM3 pharmacological activation leads to increased proliferation. The signalling pathway of GABA in these cells is not known in this study. We therefore attempted to elucidate details of GABA(A) signalling in TM3 and adult mouse Leydig cells using several experimental approaches. TM3 cells not only express GABA(A) receptor subunits, but also bind the GABA agonist {[}H-3] muscimol with a binding affinity in the range reported for other endocrine cells (K-d = 2.740 +/- 0.721 nM). However, they exhibit a low B-max value of 28.08 fmol/mg protein. Typical GABA(A) receptor-associated events, including Cl- currents, changes in resting membrane potential, intracellular Ca2+ or cAMP, were not measurable with the methods employed in TM3 cells, or, as studied in part, in primary mouse Leydig cells. GABA or GABA(A) agonist isoguvacine treatment resulted in increased or decreased levels of several mRNAs, including transcription factors (c-fos, hsf-1, egr-1) and cell cycle-associated genes (Cdk2, cyclin D1). In an attempt to verify the cDNA array results and because egr-1 was recently implied in Leydig cell development, we further studied this factor. RT-PCR and Western blotting confirmed a time-dependent regulation of egr-1 in TM3. In the postnatal testis egr-1 was seen in cytoplasmic and nuclear locations of developing Leydig cells, which bear GABA(A) receptors and correspond well to TM3 cells. Thus, GABA acts via an untypical novel signalling pathway in TM3 cells. Further details of this pathway remain to be elucidated. Copyright (c) 2005 S. Karger AG, Base

Intracellular chloride concentration influences the GABAA receptor subunit composition

GABAA receptors (GABAARs) exist as different subtype variants showing unique functional properties and defined spatio-temporal expression pattern. The molecular mechanisms underlying the developmental expression of different GABAAR are largely unknown. The intracellular concentration of chloride ([Cl−]i), the main ion permeating through GABAARs, also undergoes considerable changes during maturation, being higher at early neuronal stages with respect to adult neurons. Here we investigate the possibility that [Cl−]i could modulate the sequential expression of specific GABAARs subtypes in primary cerebellar neurons. We show that [Cl−]i regulates the expression of α3-1 and δ-containing GABAA receptors, responsible for phasic and tonic inhibition, respectively. Our findings highlight the role of [Cl−]i in tuning the strength of GABAergic responses by acting as an intracellular messenger

The Depolarizing Action of GABA in Cultured Hippocampal Neurons Is Not Due to the Absence of Ketone Bodies

Two recent reports propose that the depolarizing action of GABA in the immature brain is an artifact of in vitro preparations in which glucose is the only energy source. The authors argue that this does not mimic the physiological environment because the suckling rats use ketone bodies and pyruvate as major sources of metabolic energy. Here, we show that availability of physiologically relevant levels of ketone bodies has no impact on the excitatory action of GABA in immature cultured hippocampal neurons. Addition of β-hydroxybutyrate (BHB), the primary ketone body in the neonate rat, affected neither intracellular calcium elevation nor membrane depolarizations induced by the GABA-A receptor agonist muscimol, when assessed with calcium imaging or perforated patch-clamp recording, respectively. These results confirm that the addition of ketone bodies to the extracellular environment to mimic conditions in the neonatal brain does not reverse the chloride gradient and therefore render GABA hyperpolarizing. Our data are consistent with the existence of a genuine “developmental switch” mechanism in which GABA goes from having a predominantly excitatory role in immature cells to a predominantly inhibitory one in adults