Inherent Weight Normalization in Stochastic Neural Networks

Multiplicative stochasticity such as Dropout improves the robustness and generalizability of deep neural networks. Here, we further demonstrate that always-on multiplicative stochasticity combined with simple threshold neurons are sufficient operations for deep neural networks. We call such models Neural Sampling Machines (NSM). We find that the probability of activation of the NSM exhibits a self-normalizing property that mirrors Weight Normalization, a previously studied mechanism that fulfills many of the features of Batch Normalization in an online fashion. The normalization of activities during training speeds up convergence by preventing internal covariate shift caused by changes in the input distribution. The always-on stochasticity of the NSM confers the following advantages: the network is identical in the inference and learning phases, making the NSM suitable for online learning, it can exploit stochasticity inherent to a physical substrate such as analog non-volatile memories for in-memory computing, and it is suitable for Monte Carlo sampling, while requiring almost exclusively addition and comparison operations. We demonstrate NSMs on standard classification benchmarks (MNIST and CIFAR) and event-based classification benchmarks (N-MNIST and DVS Gestures). Our results show that NSMs perform comparably or better than conventional artificial neural networks with the same architecture

Long-term persistence and adherence on urate-lowering treatment can be maintained in primary care-5-year follow-up of a proof-of-concept study

Objectives: To evaluate the persistence and adherence on urate-lowering treatment (ULT) in primary care 5 years after an initial nurse-led treatment of gout. Methods: One hundred gout patients initiated on up-titrated ULT between March and July 2010 were sent a questionnaire that elicited information on current ULT, reasons for discontinuation of ULT if applicable, medication adherence and generic and disease-specific quality-of-life measures in 2015. They were invited for one visit at which height and weight were measured and blood was collected for serum uric acid measurement. Results: Seventy-five patients, mean age 68.13 years ( s . d . 10.07) and disease duration 19.44 years ( s . d . 13), returned completed questionnaires. The 5-year persistence on ULT was 90.7% (95% CI 81.4, 91.6) and 85.3% of responders self-reported taking ULT ⩾6 days/week. Of the 65 patients who attended the study visit, the mean serum uric acid was 292.8 μmol/l ( s . d . 97.2). Conclusion: An initial treatment that includes individualized patient education and involvement in treatment decisions results in excellent adherence and persistence on ULT >4 years after the responsibility of treatment is taken over by the patient's general practitioner, suggesting that this model of gout management should be widely adopted

Neural Sampling Machine with Stochastic Synapse allows Brain-like Learning and Inference

Many real-world mission-critical applications require continual online learning from noisy data and real-time decision making with a defined confidence level. Probabilistic models and stochastic neural networks can explicitly handle uncertainty in data and allow adaptive learning-on-the-fly, but their implementation in a low-power substrate remains a challenge. Here, we introduce a novel hardware fabric that implements a new class of stochastic NN called Neural-Sampling-Machine that exploits stochasticity in synaptic connections for approximate Bayesian inference. Harnessing the inherent non-linearities and stochasticity occurring at the atomic level in emerging materials and devices allows us to capture the synaptic stochasticity occurring at the molecular level in biological synapses. We experimentally demonstrate in-silico hybrid stochastic synapse by pairing a ferroelectric field-effect transistor -based analog weight cell with a two-terminal stochastic selector element. Such a stochastic synapse can be integrated within the well-established crossbar array architecture for compute-in-memory. We experimentally show that the inherent stochastic switching of the selector element between the insulator and metallic state introduces a multiplicative stochastic noise within the synapses of NSM that samples the conductance states of the FeFET, both during learning and inference. We perform network-level simulations to highlight the salient automatic weight normalization feature introduced by the stochastic synapses of the NSM that paves the way for continual online learning without any offline Batch Normalization. We also showcase the Bayesian inferencing capability introduced by the stochastic synapse during inference mode, thus accounting for uncertainty in data. We report 98.25%accuracy on standard image classification task as well as estimation of data uncertainty in rotated samples

Does the initiation of urate lowering treatment during an acute gout attack prolong the current episode and precipitate recurrent attacks: a systematic literature review

Objectives: To systematically review the literature on effect of initiating urate lowering treatment (ULT) during an acute attack of gout on duration of index attack and persistence on ULT. Methods: OVID (MEDLINE), EMBASE and AMED were searched to identify randomized controlled trials (RCTs) of ULT initiation during acute gout attack published in English language. Two reviewers appraised the study quality and extracted data independently. Standardised mean difference (SMD) and relative risk (RR) were used to pool continuous and categorical data. Meta-analysis was carried out using STATA v14. Results: 537 studies were selected. 487 titles and abstracts were reviewed after removing duplicates. Three RCTs were identified. There was evidence from two high quality studies that early initiation of allopurinol did not increase pain severity at days 10 to 15 (SMDpooled (95%CI) 0.18(-0.58, 0.93)). Data from three studies suggested that initiation of ULT during an acute attack of gout did not associate with drop-outs (RRpooled (95%CI) 1.16(0.58, 2.31)). Conclusion: There is moderate-quality evidence that the initiation of ULT during an acute attack of gout does not increase pain severity and risk of ULT discontinuation. Larger studies are required to confirm these findings so that patients with acute gout can be initiated on ULT with confidence