Powerpropagation: A sparsity inducing weight reparameterisation

The training of sparse neural networks is becoming an increasingly important tool for reducing the computational footprint of models at training and evaluation, as well enabling the effective scaling up of models. Whereas much work over the years has been dedicated to specialised pruning techniques, little attention has been paid to the inherent effect of gradient based training on model sparsity. In this work, we introduce Powerpropagation, a new weight-parameterisation for neural networks that leads to inherently sparse models. Exploiting the behaviour of gradient descent, our method gives rise to weight updates exhibiting a “rich get richer” dynamic, leaving low-magnitude parameters largely unaffected by learning. Models trained in this manner exhibit similar performance, but have a distribution with markedly higher density at zero, allowing more parameters to be pruned safely. Powerpropagation is general, intuitive, cheap and straight-forward to implement and can readily be combined with various other techniques. To highlight its versatility, we explore it in two very different settings: Firstly, following a recent line of work, we investigate its effect on sparse training for resource-constrained settings. Here, we combine Powerpropagation with a traditional weight-pruning technique as well as recent state-of-the-art sparse-to-sparse algorithms, showing superior performance on the ImageNet benchmark. Secondly, we advocate the use of sparsity in overcoming catastrophic forgetting, where compressed representations allow accommodating a large number of tasks at fixed model capacity. In all cases our reparameterisation considerably increases the efficacy of the off-the-shelf methods

Reinstated episodic context guides sampling-based decisions for reward.

How does experience inform decisions? In episodic sampling, decisions are guided by a few episodic memories of past choices. This process can yield choice patterns similar to model-free reinforcement learning; however, samples can vary from trial to trial, causing decisions to vary. Here we show that context retrieved during episodic sampling can cause choice behavior to deviate sharply from the predictions of reinforcement learning. Specifically, we show that, when a given memory is sampled, choices (in the present) are influenced by the properties of other decisions made in the same context as the sampled event. This effect is mediated by fMRI measures of context retrieval on each trial, suggesting a mechanism whereby cues trigger retrieval of context, which then triggers retrieval of other decisions from that context. This result establishes a new avenue by which experience can guide choice and, as such, has broad implications for the study of decisions

On duality symmetry in perturbative quantum theory

Non-compact symmetries of extended 4d supergravities involve duality rotations of vectors and thus are not manifest off-shell invariances in standard "second-order" formulation. To study how such symmetries are realised in the quantum theory we consider examples in 2 dimensions where vector-vector duality is replaced by scalar-scalar one. Using a "doubled" formulation, where fields and their momenta are treated on an equal footing and the duality becomes a manifest symmetry of the action (at the expense of Lorentz symmetry), we argue that the corresponding on-shell quantum effective action or S-matrix are duality symmetric as well as Lorentz invariant. The simplest case of discrete Z_2 duality corresponds to a symmetry of the S-matrix under flipping the sign of the negative-chirality scalars in 2 dimensions or phase rotations of chiral (definite-helicity) parts of vectors in 4 dimensions. We also briefly discuss some 4d models and comment on implications of our analysis for extended supergravities.Comment: 21 pages, Latex v2: comments and references added v3: references and minor comments adde

Molecular motors robustly drive active gels to a critically connected state

Living systems often exhibit internal driving: active, molecular processes drive nonequilibrium phenomena such as metabolism or migration. Active gels constitute a fascinating class of internally driven matter, where molecular motors exert localized stresses inside polymer networks. There is evidence that network crosslinking is required to allow motors to induce macroscopic contraction. Yet a quantitative understanding of how network connectivity enables contraction is lacking. Here we show experimentally that myosin motors contract crosslinked actin polymer networks to clusters with a scale-free size distribution. This critical behavior occurs over an unexpectedly broad range of crosslink concentrations. To understand this robustness, we develop a quantitative model of contractile networks that takes into account network restructuring: motors reduce connectivity by forcing crosslinks to unbind. Paradoxically, to coordinate global contractions, motor activity should be low. Otherwise, motors drive initially well-connected networks to a critical state where ruptures form across the entire network.Comment: Main text: 21 pages, 5 figures. Supplementary Information: 13 pages, 8 figure

Logarithmic Corrections to Extremal Black Hole Entropy from Quantum Entropy Function

We evaluate the one loop determinant of matter multiplet fields of N=4 supergravity in the near horizon geometry of quarter BPS black holes, and use it to calculate logarithmic corrections to the entropy of these black holes using the quantum entropy function formalism. We show that even though individual fields give non-vanishing logarithmic contribution to the entropy, the net contribution from all the fields in the matter multiplet vanishes. Thus logarithmic corrections to the entropy of quarter BPS black holes, if present, must be independent of the number of matter multiplet fields in the theory. This is consistent with the microscopic results. During our analysis we also determine the complete spectrum of small fluctuations of matter multiplet fields in the near horizon geometry.Comment: LaTeX file, 52 pages; v2: minor corrections, references adde

The Terminal Immunoglobulin-Like Repeats of LigA and LigB of Leptospira Enhance Their Binding to Gelatin Binding Domain of Fibronectin and Host Cells

Leptospira spp. are pathogenic spirochetes that cause the zoonotic disease leptospirosis. Leptospiral immunoglobulin (Ig)-like protein B (LigB) contributes to the binding of Leptospira to extracellular matrix proteins such as fibronectin, fibrinogen, laminin, elastin, tropoelastin and collagen. A high-affinity Fn-binding region of LigB has been localized to LigBCen2, which contains the partial 11th and full 12th Ig-like repeats (LigBCen2R) and 47 amino acids of the non-repeat region (LigBCen2NR) of LigB. In this study, the gelatin binding domain of fibronectin was shown to interact with LigBCen2R (KD = 1.91±0.40 µM). Not only LigBCen2R but also other Ig-like domains of Lig proteins including LigAVar7'-8, LigAVar10, LigAVar11, LigAVar12, LigAVar13, LigBCen7'-8, and LigBCen9 bind to GBD. Interestingly, a large gain in affinity was achieved through an avidity effect, with the terminal domains, 13th (LigA) or 12th (LigB) Ig-like repeat of Lig protein (LigAVar7'-13 and LigBCen7'-12) enhancing binding affinity approximately 51 and 28 fold, respectively, compared to recombinant proteins without this terminal repeat. In addition, the inhibited effect on MDCKs cells can also be promoted by Lig proteins with terminal domains, but these two domains are not required for gelatin binding domain binding and cell adhesion. Interestingly, Lig proteins with the terminal domains could form compact structures with a round shape mediated by multidomain interaction. This is the first report about the interaction of gelatin binding domain of Fn and Lig proteins and provides an example of Lig-gelatin binding domain binding mediating bacterial-host interaction

Dietary soy and meat proteins induce distinct physiological and gene expression changes in rats

This study reports on a comprehensive comparison of the effects of soy and meat proteins given at the recommended level on physiological markers of metabolic syndrome and the hepatic transcriptome. Male rats were fed semi-synthetic diets for 1 wk that differed only regarding protein source, with casein serving as reference. Body weight gain and adipose tissue mass were significantly reduced by soy but not meat proteins. The insulin resistance index was improved by soy, and to a lesser extent by meat proteins. Liver triacylglycerol contents were reduced by both protein sources, which coincided with increased plasma triacylglycerol concentrations. Both soy and meat proteins changed plasma amino acid patterns. The expression of 1571 and 1369 genes were altered by soy and meat proteins respectively. Functional classification revealed that lipid, energy and amino acid metabolic pathways, as well as insulin signaling pathways were regulated differently by soy and meat proteins. Several transcriptional regulators, including NFE2L2, ATF4, Srebf1 and Rictor were identified as potential key upstream regulators. These results suggest that soy and meat proteins induce distinct physiological and gene expression responses in rats and provide novel evidence and suggestions for the health effects of different protein sources in human diets

Activated Platelets in Carotid Artery Thrombosis in Mice Can Be Selectively Targeted with a Radiolabeled Single-Chain Antibody

BACKGROUND: Activated platelets can be found on the surface of inflamed, rupture-prone and ruptured plaques as well as in intravascular thrombosis. They are key players in thrombosis and atherosclerosis. In this study we describe the construction of a radiolabeled single-chain antibody targeting the LIBS-epitope of activated platelets to selectively depict platelet activation and wall-adherent non-occlusive thrombosis in a mouse model with nuclear imaging using in vitro and ex vivo autoradiography as well as small animal SPECT-CT for in vivo analysis. METHODOLOGY/PRINCIPAL FINDINGS: LIBS as well as an unspecific control single-chain antibody were labeled with (111)Indium ((111)In) via bifunctional DTPA ( = (111)In-LIBS/(111)In-control). Autoradiography after incubation with (111)In-LIBS on activated platelets in vitro (mean 3866 ± 28 DLU/mm(2), 4010 ± 630 DLU/mm(2) and 4520 ± 293 DLU/mm(2)) produced a significantly higher ligand uptake compared to (111)In-control (2101 ± 76 DLU/mm(2), 1181 ± 96 DLU/mm(2) and 1866 ± 246 DLU/mm(2)) indicating a specific binding to activated platelets; P<0.05. Applying these findings to an ex vivo mouse model of carotid artery thrombosis revealed a significant increase in ligand uptake after injection of (111)In-LIBS in the presence of small thrombi compared to the non-injured side, as confirmed by histology (49630 ± 10650 DLU/mm(2) vs. 17390 ± 7470 DLU/mm(2); P<0.05). These findings could also be reproduced in vivo. SPECT-CT analysis of the injured carotid artery with (111)In-LIBS resulted in a significant increase of the target-to-background ratio compared to (111)In-control (1.99 ± 0.36 vs. 1.1 ± 0.24; P < 0.01). CONCLUSIONS/SIGNIFICANCE: Nuclear imaging with (111)In-LIBS allows the detection of platelet activation in vitro and ex vivo with high sensitivity. Using SPECT-CT, wall-adherent activated platelets in carotid arteries could be depicted in vivo. These results encourage further studies elucidating the role of activated platelets in plaque pathology and atherosclerosis and might be of interest for further developments towards clinical application

The actin-myosin regulatory MRCK kinases: regulation, biological functions and associations with human cancer

The contractile actin-myosin cytoskeleton provides much of the force required for numerous cellular activities such as motility, adhesion, cytokinesis and changes in morphology. Key elements that respond to various signal pathways are the myosin II regulatory light chains (MLC), which participate in actin-myosin contraction by modulating the ATPase activity and consequent contractile force generation mediated by myosin heavy chain heads. Considerable effort has focussed on the role of MLC kinases, and yet the contributions of the myotonic dystrophy-related Cdc42-binding kinases (MRCK) proteins in MLC phosphorylation and cytoskeleton regulation have not been well characterized. In contrast to the closely related ROCK1 and ROCK2 kinases that are regulated by the RhoA and RhoC GTPases, there is relatively little information about the CDC42-regulated MRCKα, MRCKβ and MRCKγ members of the AGC (PKA, PKG and PKC) kinase family. As well as differences in upstream activation pathways, MRCK and ROCK kinases apparently differ in the way that they spatially regulate MLC phosphorylation, which ultimately affects their influence on the organization and dynamics of the actin-myosin cytoskeleton. In this review, we will summarize the MRCK protein structures, expression patterns, small molecule inhibitors, biological functions and associations with human diseases such as cancer

The porin and the permeating antibiotic: A selective diffusion barrier in gram-negative bacteria

Gram-negative bacteria are responsible for a large proportion of antibiotic resistant bacterial diseases. These bacteria have a complex cell envelope that comprises an outer membrane and an inner membrane that delimit the periplasm. The outer membrane contains various protein channels, called porins, which are involved in the influx of various compounds, including several classes of antibiotics. Bacterial adaptation to reduce influx through porins is an increasing problem worldwide that contributes, together with efflux systems, to the emergence and dissemination of antibiotic resistance. An exciting challenge is to decipher the genetic and molecular basis of membrane impermeability as a bacterial resistance mechanism. This Review outlines the bacterial response towards antibiotic stress on altered membrane permeability and discusses recent advances in molecular approaches that are improving our knowledge of the physico-chemical parameters that govern the translocation of antibiotics through porin channel