130 research outputs found
Memristive Reservoirs Learn to Learn
Memristive reservoirs draw inspiration from a novel class of neuromorphic
hardware known as nanowire networks. These systems display emergent brain-like
dynamics, with optimal performance demonstrated at dynamical phase transitions.
In these networks, a limited number of electrodes are available to modulate
system dynamics, in contrast to the global controllability offered by
neuromorphic hardware through random access memories. We demonstrate that the
learn-to-learn framework can effectively address this challenge in the context
of optimization. Using the framework, we successfully identify the optimal
hyperparameters for the reservoir. This finding aligns with previous research,
which suggests that the optimal performance of a memristive reservoir occurs at
the `edge of formation' of a conductive pathway. Furthermore, our results show
that these systems can mimic membrane potential behavior observed in spiking
neurons, and may serve as an interface between spike-based and continuous
processes.Comment: 7 pages, 6 figures, ICONS 2023, accepte
SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks
As the size of large language models continue to scale, so does the
computational resources required to run it. Spiking Neural Networks (SNNs) have
emerged as an energy-efficient approach to deep learning that leverage sparse
and event-driven activations to reduce the computational overhead associated
with model inference. While they have become competitive with non-spiking
models on many computer vision tasks, SNNs have also proven to be more
challenging to train. As a result, their performance lags behind modern deep
learning, and we are yet to see the effectiveness of SNNs in language
generation. In this paper, inspired by the Receptance Weighted Key Value (RWKV)
language model, we successfully implement `SpikeGPT', a generative language
model with binary, event-driven spiking activation units. We train the proposed
model on two model variants: 45M and 216M parameters. To the best of our
knowledge, SpikeGPT is the largest backpropagation-trained SNN model to date,
rendering it suitable for both the generation and comprehension of natural
language. We achieve this by modifying the transformer block to replace
multi-head self attention to reduce quadratic computational complexity O(N^2)
to linear complexity O(N) with increasing sequence length. Input tokens are
instead streamed in sequentially to our attention mechanism (as with typical
SNNs). Our preliminary experiments show that SpikeGPT remains competitive with
non-spiking models on tested benchmarks, while maintaining 20x fewer operations
when processed on neuromorphic hardware that can leverage sparse, event-driven
activations
Memristive Stochastic Computing for Deep Learning Parameter Optimization
Stochastic Computing (SC) is a computing paradigm that allows for the
low-cost and low-power computation of various arithmetic operations using
stochastic bit streams and digital logic. In contrast to conventional
representation schemes used within the binary domain, the sequence of bit
streams in the stochastic domain is inconsequential, and computation is usually
non-deterministic. In this brief, we exploit the stochasticity during switching
of probabilistic Conductive Bridging RAM (CBRAM) devices to efficiently
generate stochastic bit streams in order to perform Deep Learning (DL)
parameter optimization, reducing the size of Multiply and Accumulate (MAC)
units by 5 orders of magnitude. We demonstrate that in using a 40-nm
Complementary Metal Oxide Semiconductor (CMOS) process our scalable
architecture occupies 1.55mm and consumes approximately 167W when
optimizing parameters of a Convolutional Neural Network (CNN) while it is being
trained for a character recognition task, observing no notable reduction in
accuracy post-training.Comment: Accepted by IEEE Transactions on Circuits and Systems Part II:
Express Brief
Patients and Public Involvement in Patient Safety and Treatment Process in Hospitals Affiliated to Kashan University of Medical Sciences, Iran, 2013
Background: In the recent decades, healthcare providers had a perspective of benevolent paternalism. Nowadays, the patients’ role has changed and they have a significant obligation to participate in their caring decisions.
Objectives: The current study aimed to investigate the involvement of patients and public in the patient safety and treatment process in hospitals affiliated to Kashan University of Medical Sciences, Kashan, Iran, 2013.
Patients and Methods: A cross-sectional study was conducted in the hospitals affiliated to Kashan University of Medical Sciences in 2013. Subjects included all of the 18 chief managers, 10% of nurses in each hospital, and 375 patients. Data collection instruments included a questionnaire and a checklist designed according to the research objectives. Data analysis was performed using the SPSS ver. 13. Descriptive statistics, percentage and frequencies, were calculated for all variables and analyzed by Chi-square test.
Results: In the treatment process, 81 patients (21.61%), 50 nurses (80.6%) and 15 chief managers (83.3%) had awareness about Patient Bill of Rights. In patient Safety, 19.73% of the patients stated that hospitals received their feedbacks. Management activities were weak in evaluation. All of the six hospitals (100%) had a defined process to perform satisfaction surveys’ quality improvement and patient authentication policy.
Conclusions: Patient and public participation in Kashan hospitals are not adhered well. As the patient has an important role in improving the quality of services, more use of mass media especially local newspapers, hospital websites, and training programs are suggested to inform both the patients and public on their rights and roles in improving the healthcare services
Side-channel attack analysis on in-memory computing architectures
In-memory computing (IMC) systems have great potential for accelerating
data-intensive tasks such as deep neural networks (DNNs). As DNN models are
generally highly proprietary, the neural network architectures become valuable
targets for attacks. In IMC systems, since the whole model is mapped on chip
and weight memory read can be restricted, the system acts as a "black box" for
customers. However, the localized and stationary weight and data patterns may
subject IMC systems to other attacks. In this paper, we propose a side-channel
attack methodology on IMC architectures. We show that it is possible to extract
model architectural information from power trace measurements without any prior
knowledge of the neural network. We first developed a simulation framework that
can emulate the dynamic power traces of the IMC macros. We then performed
side-channel attacks to extract information such as the stored layer type,
layer sequence, output channel/feature size and convolution kernel size from
power traces of the IMC macros. Based on the extracted information, full
networks can potentially be reconstructed without any knowledge of the neural
network. Finally, we discuss potential countermeasures for building IMC systems
that offer resistance to these model extraction attack
PowerGAN: A Machine Learning Approach for Power Side-Channel Attack on Compute-in-Memory Accelerators
Analog compute-in-memory (CIM) accelerators are becoming increasingly popular
for deep neural network (DNN) inference due to their energy efficiency and
in-situ vector-matrix multiplication (VMM) capabilities. However, as the use of
DNNs expands, protecting user input privacy has become increasingly important.
In this paper, we identify a security vulnerability wherein an adversary can
reconstruct the user's private input data from a power side-channel attack,
under proper data acquisition and pre-processing, even without knowledge of the
DNN model. We further demonstrate a machine learning-based attack approach
using a generative adversarial network (GAN) to enhance the reconstruction. Our
results show that the attack methodology is effective in reconstructing user
inputs from analog CIM accelerator power leakage, even when at large noise
levels and countermeasures are applied. Specifically, we demonstrate the
efficacy of our approach on the U-Net for brain tumor detection in magnetic
resonance imaging (MRI) medical images, with a noise-level of 20% standard
deviation of the maximum power signal value. Our study highlights a significant
security vulnerability in analog CIM accelerators and proposes an effective
attack methodology using a GAN to breach user privacy
Analog Weights in ReRAM DNN Accelerators
Artificial neural networks have become ubiquitous in modern life, which has
triggered the emergence of a new class of application specific integrated
circuits for their acceleration. ReRAM-based accelerators have gained
significant traction due to their ability to leverage in-memory computations.
In a crossbar structure, they can perform multiply-and-accumulate operations
more efficiently than standard CMOS logic. By virtue of being resistive
switches, ReRAM switches can only reliably store one of two states. This is a
severe limitation on the range of values in a computational kernel. This paper
presents a novel scheme in alleviating the single-bit-per-device restriction by
exploiting frequency dependence of v-i plane hysteresis, and assigning kernel
information not only to the device conductance but also partially distributing
it to the frequency of a time-varying input. We show this approach reduces
average power consumption for a single crossbar convolution by up to a factor
of x16 for an unsigned 8-bit input image, where each convolutional process
consumes a worst-case of 1.1mW, and reduces area by a factor of x8, without
reducing accuracy to the level of binarized neural networks. This presents a
massive saving in computing cost when there are many simultaneous in-situ
multiply-and-accumulate processes occurring across different crossbars.Comment: 2019 IEEE International Conference on Artificial Intelligence
Circuits and Systems, 5 pages, 4 figure
Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications
With the advent of dedicated Deep Learning (DL) accelerators and neuromorphic
processors, new opportunities are emerging for applying deep and Spiking Neural
Network (SNN) algorithms to healthcare and biomedical applications at the edge.
This can facilitate the advancement of the medical Internet of Things (IoT)
systems and Point of Care (PoC) devices. In this paper, we provide a tutorial
describing how various technologies ranging from emerging memristive devices,
to established Field Programmable Gate Arrays (FPGAs), and mature Complementary
Metal Oxide Semiconductor (CMOS) technology can be used to develop efficient DL
accelerators to solve a wide variety of diagnostic, pattern recognition, and
signal processing problems in healthcare. Furthermore, we explore how spiking
neuromorphic processors can complement their DL counterparts for processing
biomedical signals. After providing the required background, we unify the
sparsely distributed research on neural network and neuromorphic hardware
implementations as applied to the healthcare domain. In addition, we benchmark
various hardware platforms by performing a biomedical electromyography (EMG)
signal processing task and drawing comparisons among them in terms of inference
delay and energy. Finally, we provide our analysis of the field and share a
perspective on the advantages, disadvantages, challenges, and opportunities
that different accelerators and neuromorphic processors introduce to healthcare
and biomedical domains. This paper can serve a large audience, ranging from
nanoelectronics researchers, to biomedical and healthcare practitioners in
grasping the fundamental interplay between hardware, algorithms, and clinical
adoption of these tools, as we shed light on the future of deep networks and
spiking neuromorphic processing systems as proponents for driving biomedical
circuits and systems forward.Comment: Submitted to IEEE Transactions on Biomedical Circuits and Systems (21
pages, 10 figures, 5 tables
- …