A Scale Mixture Perspective of Multiplicative Noise in Neural Networks

Corrupting the input and hidden layers of deep neural networks (DNNs) with multiplicative noise, often drawn from the Bernoulli distribution (or 'dropout'), provides regularization that has significantly contributed to deep learning's success. However, understanding how multiplicative corruptions prevent overfitting has been difficult due to the complexity of a DNN's functional form. In this paper, we show that when a Gaussian prior is placed on a DNN's weights, applying multiplicative noise induces a Gaussian scale mixture, which can be reparameterized to circumvent the problematic likelihood function. Analysis can then proceed by using a type-II maximum likelihood procedure to derive a closed-form expression revealing how regularization evolves as a function of the network's weights. Results show that multiplicative noise forces weights to become either sparse or invariant to rescaling. We find our analysis has implications for model compression as it naturally reveals a weight pruning rule that starkly contrasts with the commonly used signal-to-noise ratio (SNR). While the SNR prunes weights with large variances, seeing them as noisy, our approach recognizes their robustness and retains them. We empirically demonstrate our approach has a strong advantage over the SNR heuristic and is competitive to retraining with soft targets produced from a teacher model

Enhancing workflow-nets with data for trace completion

The growing adoption of IT-systems for modeling and executing (business) processes or services has thrust the scientific investigation towards techniques and tools which support more complex forms of process analysis. Many of them, such as conformance checking, process alignment, mining and enhancement, rely on complete observation of past (tracked and logged) executions. In many real cases, however, the lack of human or IT-support on all the steps of process execution, as well as information hiding and abstraction of model and data, result in incomplete log information of both data and activities. This paper tackles the issue of automatically repairing traces with missing information by notably considering not only activities but also data manipulated by them. Our technique recasts such a problem in a reachability problem and provides an encoding in an action language which allows to virtually use any state-of-the-art planning to return solutions

A General Test of Gaming

An important lesson from the incentive literature is that explicit incentives may elicit dysfunctional and unintended responses, also known as gaming responses. The existence of these responses, however, is difficult to demonstrate in practice because this behavior is typically hidden from the researcher. We present a simple model showing that one can identify gaming by estimating the correlation between a performance measure and the true goal of the organization before and after the measure has been activated. Our hypothesis is that gaming takes place if this correlation decreases with activation. Using data from a public sector organization, we find evidence consistent with our hypothesis. We draw implications for the selection of performance measures.Performance Incentive, Performance Measurement, Gaming, Multitasking, Government Organization.

Comparative biochemical and functional analysis of viral and human secreted tumor necrosis factor (TNF) decoy receptors

© 2015 by The American Society for Biochemistry and Molecular Biology, Inc. The blockade of tumor necrosis factor (TNF) by etanercept, a soluble version of the human TNF receptor 2 (hTNFR2), is a well established strategy to inhibit adverse TNF-mediated inflammatory responses in the clinic. A similar strategy is employed by poxviruses, encoding four viral TNF decoy receptor homologues (vTNFRs) named cytokine response modifier B (CrmB), CrmC, CrmD, and CrmE. These vTNFRs are differentially expressed by poxviral species, suggesting distinct immunomodulatory properties. Whereas the human variola virus and mouse ectromelia virus encode one vTNFR, the broad host range cowpox virus encodes all vTNFRs. We report the first comprehensive study of the functional and binding properties of these four vTNFRs, providing an explanation for their expression profile among different poxviruses. In addition, the vTNFRs activities were compared with the hTNFR2 used in the clinic. Interestingly, CrmB from variola virus, the causative agent of smallpox, is the most potent TNFR of those tested here including hTNFR2. Furthermore, we demonstrate a new immunomodulatory activity of vTNFRs, showing that CrmB and CrmD also inhibit the activity of lymphotoxin β. Similarly, we report for the first time that the hTNFR2 blocks the biological activity of lymphotoxin β. The characterization of vTNFRs optimized during virus-host evolution to modulate the host immune response provides relevant information about their potential role in pathogenesis and may be used to improve anti-inflammatory therapies based on soluble decoy TNFRs.Ministry of Economy and Competitiveness Grants SAF2009-07857 and SAF2012-38957Peer Reviewe

Stochastic stability versus localization in chaotic dynamical systems

We prove stochastic stability of chaotic maps for a general class of Markov random perturbations (including singular ones) satisfying some kind of mixing conditions. One of the consequences of this statement is the proof of Ulam's conjecture about the approximation of the dynamics of a chaotic system by a finite state Markov chain. Conditions under which the localization phenomenon (i.e. stabilization of singular invariant measures) takes place are also considered. Our main tools are the so called bounded variation approach combined with the ergodic theorem of Ionescu-Tulcea and Marinescu, and a random walk argument that we apply to prove the absence of ``traps'' under the action of random perturbations.Comment: 27 pages, LaTe

Rundbalsensilering

Sammandrag av en litteraturstudie av Matts Nysand. Litteraturstudien i sin helhet publiceras på finska som Vakolan tutkimusselostus no 55