1,039 research outputs found
Self-Binarizing Networks
We present a method to train self-binarizing neural networks, that is,
networks that evolve their weights and activations during training to become
binary. To obtain similar binary networks, existing methods rely on the sign
activation function. This function, however, has no gradients for non-zero
values, which makes standard backpropagation impossible. To circumvent the
difficulty of training a network relying on the sign activation function, these
methods alternate between floating-point and binary representations of the
network during training, which is sub-optimal and inefficient. We approach the
binarization task by training on a unique representation involving a smooth
activation function, which is iteratively sharpened during training until it
becomes a binary representation equivalent to the sign activation function.
Additionally, we introduce a new technique to perform binary batch
normalization that simplifies the conventional batch normalization by
transforming it into a simple comparison operation. This is unlike existing
methods, which are forced to the retain the conventional floating-point-based
batch normalization. Our binary networks, apart from displaying advantages of
lower memory and computation as compared to conventional floating-point and
binary networks, also show higher classification accuracy than existing
state-of-the-art methods on multiple benchmark datasets.Comment: 9 pages, 5 figure
Self-Binarizing Networks
We present a method to train self-binarizing neural networks, that is, networks that evolve their weights and activations during training to become binary. To obtain similar binary networks, existing methods rely on the sign activation function. This function, however, has no gradients for non-zero values, which makes standard backpropagation impossible. To circumvent the difficulty of training a network relying on the sign activation function, these methods alternate between floating-point and binary representations of the network during training, which is sub-optimal and inefficient. We approach the binarization task by training on a unique representation involving a smooth activation function, which is iteratively sharpened during training until it becomes a binary representation equivalent to the sign activation function. Additionally, we introduce a new technique to perform binary batch normalization that simplifies the conventional batch normalization by transforming it into a simple comparison operation. This is unlike existing methods, which are forced to the retain the conventional floating-point-based batch normalization. Our binary networks, apart from displaying advantages of lower memory and computation as compared to conventional floating-point and binary networks, also show higher classification accuracy than existing state-of-the-art methods on multiple benchmark datasets
Binarized Spectral Compressive Imaging
Existing deep learning models for hyperspectral image (HSI) reconstruction
achieve good performance but require powerful hardwares with enormous memory
and computational resources. Consequently, these methods can hardly be deployed
on resource-limited mobile devices. In this paper, we propose a novel method,
Binarized Spectral-Redistribution Network (BiSRNet), for efficient and
practical HSI restoration from compressed measurement in snapshot compressive
imaging (SCI) systems. Firstly, we redesign a compact and easy-to-deploy base
model to be binarized. Then we present the basic unit, Binarized
Spectral-Redistribution Convolution (BiSR-Conv). BiSR-Conv can adaptively
redistribute the HSI representations before binarizing activation and uses a
scalable hyperbolic tangent function to closer approximate the Sign function in
backpropagation. Based on our BiSR-Conv, we customize four binarized
convolutional modules to address the dimension mismatch and propagate
full-precision information throughout the whole network. Finally, our BiSRNet
is derived by using the proposed techniques to binarize the base model.
Comprehensive quantitative and qualitative experiments manifest that our
proposed BiSRNet outperforms state-of-the-art binarization methods and achieves
comparable performance with full-precision algorithms. Code and models are
publicly available at https://github.com/caiyuanhao1998/BiSCI and
https://github.com/caiyuanhao1998/MSTComment: NeurIPS 2023; The first work to study binarized spectral compressive
imaging reconstruction proble
Training Progressively Binarizing Deep Networks Using FPGAs
While hardware implementations of inference routines for Binarized Neural
Networks (BNNs) are plentiful, current realizations of efficient BNN hardware
training accelerators, suitable for Internet of Things (IoT) edge devices,
leave much to be desired. Conventional BNN hardware training accelerators
perform forward and backward propagations with parameters adopting binary
representations, and optimization using parameters adopting floating or
fixed-point real-valued representations--requiring two distinct sets of network
parameters. In this paper, we propose a hardware-friendly training method that,
contrary to conventional methods, progressively binarizes a singular set of
fixed-point network parameters, yielding notable reductions in power and
resource utilizations. We use the Intel FPGA SDK for OpenCL development
environment to train our progressively binarizing DNNs on an OpenVINO FPGA. We
benchmark our training approach on both GPUs and FPGAs using CIFAR-10 and
compare it to conventional BNNs.Comment: Accepted at 2020 IEEE International Symposium on Circuits and Systems
(ISCAS
Connectivity-based parcellation of the thalamus explains specific cognitive and behavioural symptoms in patients with bilateral thalamic infarct
A novel approach based on diffusion tractography was used here to characterise the cortico-thalamic connectivity in two patients, both presenting with an isolated bilateral infarct in the thalamus, but exhibiting partially different cognitive and behavioural profiles. Both patients (G.P. and R.F.) had a pervasive deficit in episodic memory, but only one of them (R.F.) suffered also from a dysexecutive syndrome. Both patients had an MRI scan at 3T, including a T1-weighted volume. Their lesions were manually segmented. T1-volumes were normalised to standard space, and the same transformations were applied to the lesion masks. Nineteen healthy controls underwent a diffusion-tensor imaging (DTI) scan. Their DTI data were normalised to standard space and averaged. An atlas of Brodmann areas was used to parcellate the prefrontal cortex. Probabilistic tractography was used to assess the probability of connection between each voxel of the thalamus and a set of prefrontal areas. The resulting map of corticothalamic connections was superimposed onto the patients' lesion masks, to assess whether the location of the thalamic lesions in R.F. (but not in G. P.) implied connections with prefrontal areas involved in dysexecutive syndromes. In G.P., the lesion fell within areas of the thalamus poorly connected with prefrontal areas, showing only a modest probability of connection with the anterior cingulate cortex (ACC). Conversely, R.F.'s lesion fell within thalamic areas extensively connected with the ACC bilaterally, with the right dorsolateral prefrontal cortex, and with the left supplementary motor area. Despite a similar, bilateral involvement of the thalamus, the use of connectivity-based segmentation clarified that R.F.'s lesions only were located within nuclei highly connected with the prefrontal cortical areas, thus explaining the patient's frontal syndrome. This study confirms that DTI tractography is a useful tool to examine in vivo the effect of focal lesions on interconnectivity brain patterns
- …