Synaptic Sampling of Neural Networks

Probabilistic artificial neural networks offer intriguing prospects for enabling the uncertainty of artificial intelligence methods to be described explicitly in their function; however, the development of techniques that quantify uncertainty by well-understood methods such as Monte Carlo sampling has been limited by the high costs of stochastic sampling on deterministic computing hardware. Emerging computing systems that are amenable to hardware-level probabilistic computing, such as those that leverage stochastic devices, may make probabilistic neural networks more feasible in the not-too-distant future. This paper describes the scANN technique -- \textit{sampling (by coinflips) artificial neural networks} -- which enables neural networks to be sampled directly by treating the weights as Bernoulli coin flips. This method is natively well suited for probabilistic computing techniques that focus on tunable stochastic devices, nearly matches fully deterministic performance while also describing the uncertainty of correct and incorrect neural network outputs.Comment: 9 pages, accepted to 2023 IEEE International Conference on Rebooting Computin

Dynamic Analysis of Executables to Detect and Characterize Malware

It is needed to ensure the integrity of systems that process sensitive information and control many aspects of everyday life. We examine the use of machine learning algorithms to detect malware using the system calls generated by executables-alleviating attempts at obfuscation as the behavior is monitored rather than the bytes of an executable. We examine several machine learning techniques for detecting malware including random forests, deep learning techniques, and liquid state machines. The experiments examine the effects of concept drift on each algorithm to understand how well the algorithms generalize to novel malware samples by testing them on data that was collected after the training data. The results suggest that each of the examined machine learning algorithms is a viable solution to detect malware-achieving between 90% and 95% class-averaged accuracy (CAA). In real-world scenarios, the performance evaluation on an operational network may not match the performance achieved in training. Namely, the CAA may be about the same, but the values for precision and recall over the malware can change significantly. We structure experiments to highlight these caveats and offer insights into expected performance in operational environments. In addition, we use the induced models to gain a better understanding about what differentiates the malware samples from the goodware, which can further be used as a forensics tool to understand what the malware (or goodware) was doing to provide directions for investigation and remediation.Comment: 9 pages, 6 Tables, 4 Figure

Neurogenesis Deep Learning

Neural machine learning methods, such as deep neural networks (DNN), have achieved remarkable success in a number of complex data processing tasks. These methods have arguably had their strongest impact on tasks such as image and audio processing - data processing domains in which humans have long held clear advantages over conventional algorithms. In contrast to biological neural systems, which are capable of learning continuously, deep artificial networks have a limited ability for incorporating new information in an already trained network. As a result, methods for continuous learning are potentially highly impactful in enabling the application of deep networks to dynamic data sets. Here, inspired by the process of adult neurogenesis in the hippocampus, we explore the potential for adding new neurons to deep layers of artificial neural networks in order to facilitate their acquisition of novel information while preserving previously trained data representations. Our results on the MNIST handwritten digit dataset and the NIST SD 19 dataset, which includes lower and upper case letters and digits, demonstrate that neurogenesis is well suited for addressing the stability-plasticity dilemma that has long challenged adaptive machine learning algorithms.Comment: 8 pages, 8 figures, Accepted to 2017 International Joint Conference on Neural Networks (IJCNN 2017

A Digital Neuromorphic Architecture Efficiently Facilitating Complex Synaptic Response Functions Applied to Liquid State Machines

Information in neural networks is represented as weighted connections, or synapses, between neurons. This poses a problem as the primary computational bottleneck for neural networks is the vector-matrix multiply when inputs are multiplied by the neural network weights. Conventional processing architectures are not well suited for simulating neural networks, often requiring large amounts of energy and time. Additionally, synapses in biological neural networks are not binary connections, but exhibit a nonlinear response function as neurotransmitters are emitted and diffuse between neurons. Inspired by neuroscience principles, we present a digital neuromorphic architecture, the Spiking Temporal Processing Unit (STPU), capable of modeling arbitrary complex synaptic response functions without requiring additional hardware components. We consider the paradigm of spiking neurons with temporally coded information as opposed to non-spiking rate coded neurons used in most neural networks. In this paradigm we examine liquid state machines applied to speech recognition and show how a liquid state machine with temporal dynamics maps onto the STPU-demonstrating the flexibility and efficiency of the STPU for instantiating neural algorithms.Comment: 8 pages, 4 Figures, Preprint of 2017 IJCN

Magnetic Tunnel Junction Random Number Generators Applied to Dynamically Tuned Probability Trees Driven by Spin Orbit Torque

Perpendicular magnetic tunnel junction (pMTJ)-based true-random number generators (RNG) can consume orders of magnitude less energy per bit than CMOS pseudo-RNG. Here, we numerically investigate with a macrospin Landau-Lifshitz-Gilbert equation solver the use of pMTJs driven by spin-orbit torque to directly sample numbers from arbitrary probability distributions with the help of a tunable probability tree. The tree operates by dynamically biasing sequences of pMTJ relaxation events, called 'coinflips', via an additional applied spin-transfer-torque current. Specifically, using a single, ideal pMTJ device we successfully draw integer samples on the interval 0,255 from an exponential distribution based on p-value distribution analysis. In order to investigate device-to-device variations, the thermal stability of the pMTJs are varied based on manufactured device data. It is found that while repeatedly using a varied device inhibits ability to recover the probability distribution, the device variations average out when considering the entire set of devices as a 'bucket' to agnostically draw random numbers from. Further, it is noted that the device variations most significantly impact the highest level of the probability tree, iwth diminishing errors at lower levels. The devices are then used to draw both uniformly and exponentially distributed numbers for the Monte Carlo computation of a problem from particle transport, showing excellent data fit with the analytical solution. Finally, the devices are benchmarked against CMOS and memristor RNG, showing faster bit generation and significantly lower energy use.Comment: 10 pages, 8 figures, 2 table

Beneficial Betrayal Aversion

Many studies demonstrate the social benefits of cooperation. Likewise, recent studies convincingly demonstrate that betrayal aversion hinders trust and discourages cooperation. In this respect, betrayal aversion is unlike socially “beneficial” preferences including altruism, fairness and inequity aversion, all of which encourage cooperation and exchange. To our knowledge, other than the suggestion that it acts as a barrier to rash trust decisions, the benefits of betrayal aversion remain largely unexplored. Here we use laboratory experiments with human participants to show that groups including betrayal-averse agents achieve higher levels of reciprocity and more profitable social exchange than groups lacking betrayal aversion. These results are the first rigorous evidence on the benefits of betrayal aversion, and may help future research investigating cultural differences in betrayal aversion as well as future research on the evolutionary roots of betrayal aversion. Further, our results extend the understanding of how intentions affect social interactions and exchange and provide an effective platform for further research on betrayal aversion and its effects on human behavior

Bone regeneration mediated by a bioactive and biodegradable ECM-like hydrogel based on elastin-like recombinamers

Producción CientíficaThe morbidity of bone fractures and defects is steadily increasing due to changes in the age pyramid. As such, novel biomaterials that are able to promote the healing and regeneration of injured bones are needed in order to overcome the limitations of auto-, allo-, and xenografts, while providing a ready-to-use product that may help to minimize surgical invasiveness and duration. In this regard, recombinant biomaterials, such as elastin-like recombinamers (ELRs), are very promising as their design can be tailored by genetic engineering, thus allowing scalable production and batch-to-batch consistency, amongst others. Furthermore, they can self-assemble into physically cross-linked hydrogels above a certain transition temperature, in this case body temperature, but are injectable below this temperature, thereby markedly reducing surgical invasiveness. Herein we have developed two bioactive hydrogel-forming ELRs, one including the osteogenic and osteoinductive BMP-2 and the other the RGD cell-adhesion motif. The combination of these two novel ELRs results in a BMP-2-loaded extracellular matrix-like hydrogel. Moreover, elastase-sensitive domains were included in both ELR molecules, thereby conferring biodegradation as a result of enzymatic cleavage and avoiding the need for scaffold removal after bone regeneration. Both ELRs and their combination showed excellent cytocompatibility, and the culture of cells on RGD-containing ELRs resulted in optimal cell adhesion. In addition, hydrogels based on a mixture of both ELRs were implanted in a pilot study involving a femoral bone injury model in New Zealand White rabbits, showing complete regeneration in six out of seven cases, with the other showing partial closure of the defect. Moreover, bone neo-formation was confirmed using different techniques, such as radiography, computed tomography and histology. This hydrogel system therefore displays significant potential in the regeneration of bone defects, promoting self-regeneration by the surrounding tissue with no involvement of stem cells or osteogenic factors other than BMP-2, which is released in a controlled manner by elastase-mediated cleavage from the ELR backbone.Ministerio de Economía, Industria y Competitividad (Project (MAT2013-42473-R and MAT2013-41723-R)Junta de Castilla y León (programa de apoyo a proyectos de investigación – Ref. VA244U13 and VA313U14)Centro en Red de Medicina Regenerativa y Terapia Celular de Castilla y LeónComisión Europea (proyectos NMP-2014-646075, HEALTH-F4-2011-278557, PITN-GA-2012-317306 y MSCA-ITN-2014-642687

LIRA: Lifelong Image Restoration from Unknown Blended Distortions

Most existing image restoration networks are designed in a disposable way and catastrophically forget previously learned distortions when trained on a new distortion removal task. To alleviate this problem, we raise the novel lifelong image restoration problem for blended distortions. We first design a base fork-join model in which multiple pre-trained expert models specializing in individual distortion removal task work cooperatively and adaptively to handle blended distortions. When the input is degraded by a new distortion, inspired by adult neurogenesis in human memory system, we develop a neural growing strategy where the previously trained model can incorporate a new expert branch and continually accumulate new knowledge without interfering with learned knowledge. Experimental results show that the proposed approach can not only achieve state-of-the-art performance on blended distortions removal tasks in both PSNR/SSIM metrics, but also maintain old expertise while learning new restoration tasks.Comment: ECCV2020 accepte