Fast and Accurate Sparse Coding of Visual Stimuli with a Simple, Ultra-Low-Energy Spiking Architecture

Memristive crossbars have become a popular means for realizing unsupervised and supervised learning techniques. Often, to preserve mathematical rigor, the crossbar itself is separated from the neuron capacitors. In this work, we sought to simplify the design, removing extraneous components to consume significantly lower power at a minimal cost of accuracy. This work provides derivations for the design of such a network, named the Simple Spiking Locally Competitive Algorithm, or SSLCA, as well as CMOS designs and results on the CIFAR and MNIST datasets. Compared to a non-spiking model which scored 33% on CIFAR-10 with a single-layer classifier, this hardware scored 32% accuracy. When used with a state-of-the-art deep learning classifier, the non-spiking model achieved 82% and our simplified, spiking model achieved 80%, while compressing the input data by 79%. Compared to a previously proposed spiking model, our proposed hardware consumed 99% less energy to do the same work at 21 times the throughput. Accuracy held out with online learning to a write variance of 3% and a read variance of 40%. The proposed architecture\u27s excellent accuracy and significantly lower energy usage demonstrate the utility of our innovations. This work provides a means for extremely low-energy sparse coding in mobile devices, such as cellular phones, or for very sparse coding as is needed by self-driving cars or robotics that must integrate data from multiple, high-resolution sensors

Exploring and Expanding the One-Pixel Attack

In machine learning research, adversarial examples are normal inputs to a classifier that have been specifically perturbed to cause the model to misclassify the input. These perturbations rarely affect the human readability of an input, even though the model’s output is drastically different. Recent work has demonstrated that image-classifying deep neural networks (DNNs) can be reliably fooled with the modification of a single pixel in the input image, without knowledge of a DNN’s internal parameters. This “one-pixel attack” utilizes an iterative evolutionary optimizer known as differential evolution (DE) to find the most effective pixel to perturb, via the evaluation of numerous candidate solutions with a specific fitness function. We first improve upon the original implementation of the attack by designing a fitness function to minimize the magnitude of the perturbation in addition to the network confidence. The original attack achieves a success rate of 37% on our basic model with a mean attack RMSE of 0.02418; the improved attack achieves a success rate of 38% with a mean attack RMSE of 0.01946. We then explore the attack’s efficacy by comparing its performance in neural networks of different depths, and analyze the technique by computing per-pixel heatmaps of vulnerabilities in input images. Our findings highlight the applicability of the technique across networks, while at the same time demonstrating the shortcomings of DE in maximizing the attack potential. Future work could address these shortcomings, as well as extend the one-pixel attack to new domains (e.g. video)

Generating Adversarial Attacks for Sparse Neural Networks

Neural networks provide state-of-the-art accuracy for image classification tasks. However traditional networks are highly susceptible to imperceivable perturbations to their inputs known as adversarial attacks that drastically change the resulting output. The magnitude of these perturbations can be measured as Mean Squared Error (MSE). We use genetic algorithms to produce black-box adversarial attacks and examine MSE on state-of-the-art networks. This method generates an attack that converts 90% confidence on a correct class to 50% confidence of a targeted, incorrect class after 2000 epochs. We will generate and examine attacks and their MSE against several sparse neural networks. We theorize that there exists a sparse architecture used for image classification that reduces input image space and therefore that architecture will cause an increase in the MSE required for a classification change. Our work is relevant for security dependent applications of neural networks, low-power high-performance architectures, and systems architectures

The Design of a Simple, Spiking Sparse Coding Algorithm for Memristive Hardware

Calculating a sparse code for signals with high dimensionality, such as high-resolution images, takes substantial time to compute on a traditional computer architecture. Memristors present the opportunity to combine storage and computing elements into a single, compact device, drastically reducing the area required to perform these calculations. This work focused on the analysis of two existing sparse coding architectures, one of which utilizes memristors, as well as the design of a new, third architecture that employs a memristive crossbar. These architectures implement either a non-spiking or spiking variety of sparse coding based on the Locally Competitive Algorithm (LCA) introduced by Rozell et al. in 2008. Each architecture receives an arbitrary number of input lines and drives an arbitrary number of output lines. Training of the dictionary used for the sparse code was implemented through external control signals that approximate Oja\u27s rule. The resulting designs were capable of representing input in real-time: no resets would be needed between frames of a video, for instance, though some settle time would be needed. The spiking architecture proposed is novel, emphasizing simplicity to achieve lower power than existing designs. The architectures presented were tested for their ability to encode and reconstruct 8 x 8 patches of natural images. The proposed network reconstructed patches with a normalized, root-mean-square error of 0.13, while a more complicated CMOS-only approach yielded 0.095, and a non-spiking approach yielded 0.074. Several outputs competing for representation of the input was shown to improve reconstruction quality and preserve more subtle components in the final encoding; the proposed algorithm lacks this feature. Steps to address this were proposed for future work by scaling input spikes according to the current expected residual, without adding much complexity. The architectures were also tested with the MNIST digit database, passing a sparse code onto a basic classifier. The proposed architecture scored 81% on this test, a CMOS-only spiking variant scored 76%, and the non-spiking algorithm scored 85%. Power calculations were made for each design and compared against other publications. The overall findings showed great promise for spiking memristor-based ASICs, consuming only 28% of the power used by non-spiking architectures and 6.6% as much power as a CMOS-only spiking architecture on this task. The spike-based nature of the novel design was also parameterized into several intuitive parameters that could be adjusted to prefer either performance or power efficiency. The design and analysis of architectures for sparse coding should greatly reduce the amount of future work needed to implement an end-to-end classification pipeline for images or other signal data. When lower power is a primary concern, the proposed architecture should be considered as it surpassed other published algorithms. These pipelines could be used to provide low-power visual assistance, highlighting objects within high-definition video frames in real-time. The technology could also be used to help self-driving cars identify hazards more quickly and efficiently

Spoken Digit Classification by In-Materio Reservoir Computing with Neuromorphic Atomic Switch Networks

Atomic Switch Networks (ASN) comprising silver iodide (AgI) junctions, a material previously unexplored as functional memristive elements within highly-interconnected nanowire networks, were employed as a neuromorphic substrate for physical Reservoir Computing (RC). This new class of ASN-based devices has been physically characterized and utilized to classify spoken digit audio data, demonstrating the utility of substrate-based device architectures where intrinsic material properties can be exploited to perform computation in-materio. This work demonstrates high accuracy in the classification of temporally analyzed Free-Spoken Digit Data (FSDD). These results expand upon the class of viable memristive materials available for the production of functional nanowire networks and bolster the utility of ASN-based devices as unique hardware platforms for neuromorphic computing applications involving memory, adaptation and learning.Comment: 11 pages, 7 figure

Prospectus, August 1988

https://spark.parkland.edu/prospectus_1988/1000/thumbnail.jp

The utilisation of health research in policy-making: Concepts, examples and methods of assessment

The importance of health research utilisation in policy-making, and of understanding the mechanisms involved, is increasingly recognised. Recent reports calling for more resources to improve health in developing countries, and global pressures for accountability, draw greater attention to research-informed policy-making. Key utilisation issues have been described for at least twenty years, but the growing focus on health research systems creates additional dimensions. The utilisation of health research in policy-making should contribute to policies that may eventually lead to desired outcomes, including health gains. In this article, exploration of these issues is combined with a review of various forms of policy-making. When this is linked to analysis of different types of health research, it assists in building a comprehensive account of the diverse meanings of research utilisation. Previous studies report methods and conceptual frameworks that have been applied, if with varying degrees of success, to record utilisation in policy-making. These studies reveal various examples of research impact within a general picture of underutilisation. Factors potentially enhancing utilisation can be identified by exploration of: priority setting; activities of the health research system at the interface between research and policy-making; and the role of the recipients, or 'receptors', of health research. An interfaces and receptors model provides a framework for analysis. Recommendations about possible methods for assessing health research utilisation follow identification of the purposes of such assessments. Our conclusion is that research utilisation can be better understood, and enhanced, by developing assessment methods informed by conceptual analysis and review of previous studies

A systematic review of frameworks for the interrelationships of mental health evidence and policy in low- and middle-income countries

Background: The interrelationships between research evidence and policy-making are complex. Different theoretical frameworks exist to explain general evidence–policy interactions. One largely unexplored element of these interrelationships is how evidence interrelates with, and influences, policy/political agenda-setting. This review aims to identify the elements and processes of theories, frameworks and models on interrelationships of research evidence and health policy-making, with a focus on actionability and agenda-setting in the context of mental health in low- and middle-income countries (LMICs). Methods: A systematic review of theories was conducted based on the BeHeMOTh search method, using a tested and refined search strategy. Nine electronic databases and other relevant sources were searched for peer-reviewed and grey literature. Two reviewers screened the abstracts, reviewed full-text articles, extracted data and performed quality assessments. Analysis was based on a thematic analysis. The included papers had to present an actionable theoretical framework/model on evidence and policy interrelationships, such as knowledge translation or evidence-based policy, specifically target the agenda-setting process, focus on mental health, be from LMICs and published in English. Results: From 236 publications included in the full text analysis, no studies fully complied with our inclusion criteria. Widening the focus by leaving out ‘agenda-setting’, we included ten studies, four of which had unique conceptual frameworks focusing on mental health and LMICs but not agenda-setting. The four analysed frameworks confirmed research gaps from LMICs and mental health, and a lack of focus on agenda-setting. Frameworks and models from other health and policy areas provide interesting conceptual approaches and lessons with regards to agenda-setting. Conclusion: Our systematic review identified frameworks on evidence and policy interrelations that differ in their elements and processes. No framework fulfilled all inclusion criteria. Four actionable frameworks are applicable to mental health and LMICs, but none specifically target agenda-setting. We have identified agenda-setting as a research theory gap in the context of mental health knowledge translation in LMICs. Frameworks from other health/policy areas could offer lessons on agenda-setting and new approaches for creating policy impact for mental health and to tackle the translational gap in LMICs