Pruning artificial neural networks: a way to find well-generalizing, high-entropy sharp minima

Recently, a race towards the simplification of deep networks has begun, showing that it is effectively possible to reduce the size of these models with minimal or no performance loss. However, there is a general lack in understanding why these pruning strategies are effective. In this work, we are going to compare and analyze pruned solutions with two different pruning approaches, one-shot and gradual, showing the higher effectiveness of the latter. In particular, we find that gradual pruning allows access to narrow, well-generalizing minima, which are typically ignored when using one-shot approaches. In this work we also propose PSP-entropy, a measure to understand how a given neuron correlates to some specific learned classes. Interestingly, we observe that the features extracted by iteratively-pruned models are less correlated to specific classes, potentially making these models a better fit in transfer learning approaches

Learning Sparse Neural Networks via Sensitivity-Driven Regularization

The ever-increasing number of parameters in deep neural networks poses challenges for memory-limited applications. Regularize-and-prune methods aim at meeting these challenges by sparsifying the network weights. In this context we quantify the output sensitivity to the parameters (i.e. their relevance to the network output) and introduce a regularization term that gradually lowers the absolute value of parameters with low sensitivity. Thus, a very large fraction of the parameters approach zero and are eventually set to zero by simple thresholding. Our method surpasses most of the recent techniques both in terms of sparsity and error rates. In some cases, the method reaches twice the sparsity obtained by other techniques at equal error rates

Edge Enhancement Investigations by Means of Experiments and Simulations

Standard neutron imaging procedures are based on the “shadow” of the transmitted radiation, attenuated by the sample material. Under certain conditions significant deviations from pure transmission can be found in the form of enhancement or depression at the edges of the samples. These effects can limit the quantification process in the related regions. Otherwise, an enhancement and improvement of visibility can be achieved e.g. in defect analysis. In systematic studies we investigated the dependency of these effects on the specific material (mainly for common metals), such as the sample-to-detector distance, the beam collimation, the material thickness and the neutron energy. The beam lines ICON and BOA at PSI and ANTARES at TU München were used for these experiments due to their capability for neutron imaging with highest possible spatial resolution (6.5 to 13.5 micro-meter pixel size, respectively) and their cold beam spectrum. Next to the experimental data we used a McStas tool for the description of refraction and reflection features at edges for comparison. Even if minor contributions by coherent in-line propagation phase contrast are underlined, the major effect can be described by refraction of the neutrons at the sample-void interface. Ways to suppress and to magnify the edge effects can be derived from these findings.Fil: Lehmann, E.. Paul Scherrer Institut; SuizaFil: Schulz, M.. Technische Universitat Munchen; AlemaniaFil: Wang, Y.. China Insititute of Atomic Energy; ChinaFil: Tartaglione, Aureliano. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentin

Equilibrium temperature distribution and Hadley circulation in an axisymmetric model

The impact of the equilibrium temperature distribution, θE, on the Hadley circulation simulated by an axisymmetric model is studied. The θE distributions that drive the model are modulated here by two parameters, n and k, the former controlling the horizontal broadness and the latter controlling the vertical stratification of θE. In the present study, variations in the θE distribution mimic changes in the energy input of the atmospheric system, leaving as almost invariant the Equator–poles θE difference. Both equinoctial and time-dependent Hadley circulations are simulated and the results compared. The results give evidence that concentrated θE distributions enhance the meridional circulation and jet wind speed intensities, even with a lower energy input. The meridional circulation and the subtropical jet stream widths are controlled by the broadness of horizontal θE rather than by the vertical stratification, which is important only when θE distribution is concentrated at the Equator. The jet stream position does not show any dependence with n and k, except when the θE distribution is very wide (n = 3) and, in such a case, the jet is located at the mid-latitudes and the model temperature clamps to forcing θE. Using n = 2 and k = 1, we have the formulation of the potential temperature adopted in the classical literature. A comparison with other works is performed, and our results show that the model running in different configurations (equinoctial, solstitial and time dependent) yields results similar to one another

Relationship between forecast precipitation relative errors and skill scores: the case of rare event frequencies

Abstract. This paper addresses the problem of the relationship between skill scores and forecast rainfall relative errors. The problem is approached by using synthetic time series of rainfall data representing the observations. It is assumed that the magnitude of the relative error is known. The forecasts are constructed by adding errors to the observations. We use a threshold to dichotomise forecasts and observations to obtain the skill scores. We perform 1000 simulations for each error magnitude in order to obtain the mean values and uncertainties of the scores. We consider two different precipitation regimes, and we show the influence of these regimes on the precipitation. We find that the relationship between forecast errors and skill scores is strongly influenced by the event frequencies, which in turn depend on the precipitation regime. We find that only when the event frequency of the two regimes is made similar by changing the threshold, the relationship between the scores and relative errors is similar. This suggests that a comparison between two forecast precipitation datasets should account for the difference (if any) in precipitation regimes

Analysis and decomposition of frequency modulated multicomponent signals

Frequency modulated (FM) signals are studied in many research fields, including seismology, astrophysics, biology, acoustics, animal echolocation, radar and sonar. They are referred as multicomponent signals (MCS), as they are generally composed of multiple waveforms, with specific time-dependent frequencies, known as instantaneous frequencies (IFs). Many applications require the extraction of signal characteristics (i.e. amplitudes and IFs). that is why MCS decomposition is an important topic in signal processing. It consists of the recovery of each individual mode and it is often performed by IFs separation. The task becomes very challenging if the signal modes overlap in the TF domain, i.e. they interfere with each other, at the so-called non-separability region. For this reason, a general solution to MCS decomposition is not available yet. As a matter of fact, the existing methods addressing overlapping modes share the same limitations: they are parametric, therefore they adapt only to the assumed signal class, or they rely on signal-dependent and parametric TF representations; otherwise, they are interpolation techniques, i.e. they almost ignore the information corrupted by interference and they recover IF curve by some fitting procedures, resulting in high computational cost and bad performances against noise. This thesis aims at overcoming these drawbacks, providing efficient tools for dealing with MCS with interfering modes. An extended state-of-the-art revision is provided, as well as the mathematical tools and the main definitions needed to introduce the topic. Then, the problem is addressed following two main strategies: the former is an iterative approach that aims at enhancing MCS' resolution in the TF domain; the latter is a transform-based approach, that combines TF analysis and Radon Transform for separating individual modes. As main advantage, the methods derived from both the iterative and the transform-based approaches are non-parametric, as they do not require specific assumptions on the signal class. As confirmed by the experimental results and the comparative studies, the proposed approach contributes to the current state of the-art improvement

What motivates reluctant male readers?

The purpose of this research study was to better understand the role of reading motivation in a group of students at the Thomas E. Bowe School in southern New Jersey. I wanted to what motivates this group of boys to read and how typical they might be to other reluctant male readers. Qualitative inquiry strategies such as surveys, student interviews, interactive graphic organizers and a teacher interview were used. Upon analyzing the data, motivation for reading was poor. However, student book choice and commonalities in genre selection were consistent with current research. Struggling male readers look for teacher input and interaction to provide book recommendations. These reluctant readers will read books that are easily decodable and at their independent reading level. They prefer books that are humorous and entertaining. Student freedom and book choice are critical components of reading for the boys in this study

From Statistical Physics to Algorithms in Deep Neural Systems

L'abstract è presente nell'allegato / the abstract is in the attachmen

Toxin Variability Estimations of 68 Alexandrium ostenfeldii (Dinophyceae) Strains from The Netherlands Reveal a Novel Abundant Gymnodimine

Alexandrium ostenfeldii is a toxic dinoflagellate that has recently bloomed in Ouwerkerkse Kreek, The Netherlands, and which is able to cause a serious threat to shellfish consumers and aquacultures. We used a large set of 68 strains to the aim of fully characterizing the toxin profiles of the Dutch A. ostenfeldii in consideration of recent reports of novel toxins. Alexandrium ostenfeldii is known as a causative species of paralytic shellfish poisoning, and consistently in the Dutch population we determined the presence of several paralytic shellfish toxins (PST) including saxitoxin (STX), GTX2/3 (gonyautoxins), B1 and C1/C2. We also examined the production of spiroimine toxins by the Dutch A. ostenfeldii strains. An extensive liquid chromatography-tandem mass spectrometry (LC-MS/MS) analysis revealed a high intraspecific variability of spirolides (SPX) and gymnodimines (GYM). Spirolides included 13-desMethyl-spirolide C generally as the major compound and several other mostly unknown SPX-like compounds that were detected and characterized. Besides spirolides, the presence of gymnodimine A and 12-Methyl-gymnodimine A was confirmed, together with two new gymnodimines. One of these was tentatively identified as an analogue of gymnodimine D and was the most abundant gymnodimine (calculated cell quota up to 274 pg/ cell, expressed as GYM A equivalents). Our multi-clonal approach adds new analogues to the increasing number of compounds in these toxin classes and revealed a high strain variability in cell quota and in toxin profile of toxic compounds within a single population