134,928 research outputs found
Recent Advances in Efficient Computation of Deep Convolutional Neural Networks
Deep neural networks have evolved remarkably over the past few years and they
are currently the fundamental tools of many intelligent systems. At the same
time, the computational complexity and resource consumption of these networks
also continue to increase. This will pose a significant challenge to the
deployment of such networks, especially in real-time applications or on
resource-limited devices. Thus, network acceleration has become a hot topic
within the deep learning community. As for hardware implementation of deep
neural networks, a batch of accelerators based on FPGA/ASIC have been proposed
in recent years. In this paper, we provide a comprehensive survey of recent
advances in network acceleration, compression and accelerator design from both
algorithm and hardware points of view. Specifically, we provide a thorough
analysis of each of the following topics: network pruning, low-rank
approximation, network quantization, teacher-student networks, compact network
design and hardware accelerators. Finally, we will introduce and discuss a few
possible future directions.Comment: 14 pages, 3 figure
Machine Learning for Vehicular Networks
The emerging vehicular networks are expected to make everyday vehicular
operation safer, greener, and more efficient, and pave the path to autonomous
driving in the advent of the fifth generation (5G) cellular system. Machine
learning, as a major branch of artificial intelligence, has been recently
applied to wireless networks to provide a data-driven approach to solve
traditionally challenging problems. In this article, we review recent advances
in applying machine learning in vehicular networks and attempt to bring more
attention to this emerging area. After a brief overview of the major concept of
machine learning, we present some application examples of machine learning in
solving problems arising in vehicular networks. We finally discuss and
highlight several open issues that warrant further research.Comment: Accepted by IEEE Vehicular Technology Magazin
Dynamic Channel Pruning: Feature Boosting and Suppression
Making deep convolutional neural networks more accurate typically comes at
the cost of increased computational and memory resources. In this paper, we
reduce this cost by exploiting the fact that the importance of features
computed by convolutional layers is highly input-dependent, and propose feature
boosting and suppression (FBS), a new method to predictively amplify salient
convolutional channels and skip unimportant ones at run-time. FBS introduces
small auxiliary connections to existing convolutional layers. In contrast to
channel pruning methods which permanently remove channels, it preserves the
full network structures and accelerates convolution by dynamically skipping
unimportant input and output channels. FBS-augmented networks are trained with
conventional stochastic gradient descent, making it readily available for many
state-of-the-art CNNs. We compare FBS to a range of existing channel pruning
and dynamic execution schemes and demonstrate large improvements on ImageNet
classification. Experiments show that FBS can respectively provide
and savings in compute on VGG-16 and ResNet-18, both with less than
top-5 accuracy loss.Comment: 14 pages, 5 figures, 4 tables, published as a conference paper at
ICLR 201
Select, Attend, and Transfer: Light, Learnable Skip Connections
Skip connections in deep networks have improved both segmentation and
classification performance by facilitating the training of deeper network
architectures, and reducing the risks for vanishing gradients. They equip
encoder-decoder-like networks with richer feature representations, but at the
cost of higher memory usage, computation, and possibly resulting in
transferring non-discriminative feature maps. In this paper, we focus on
improving skip connections used in segmentation networks (e.g., U-Net, V-Net,
and The One Hundred Layers Tiramisu (DensNet) architectures). We propose light,
learnable skip connections which learn to first select the most discriminative
channels and then attend to the most discriminative regions of the selected
feature maps. The output of the proposed skip connections is a unique feature
map which not only reduces the memory usage and network parameters to a high
extent, but also improves segmentation accuracy. We evaluate the proposed
method on three different 2D and volumetric datasets and demonstrate that the
proposed light, learnable skip connections can outperform the traditional heavy
skip connections in terms of segmentation accuracy, memory usage, and number of
network parameters
A Unified End-to-End Framework for Efficient Deep Image Compression
Image compression is a widely used technique to reduce the spatial redundancy
in images. Recently, learning based image compression has achieved significant
progress by using the powerful representation ability from neural networks.
However, the current state-of-the-art learning based image compression methods
suffer from the huge computational cost, which limits their capacity for
practical applications. In this paper, we propose a unified framework called
Efficient Deep Image Compression (EDIC) based on three new technologies,
including a channel attention module, a Gaussian mixture model and a
decoder-side enhancement module. Specifically, we design an auto-encoder style
network for learning based image compression. To improve the coding efficiency,
we exploit the channel relationship between latent representations by using the
channel attention module. Besides, the Gaussian mixture model is introduced for
the entropy model and improves the accuracy for bitrate estimation.
Furthermore, we introduce the decoder-side enhancement module to further
improve image compression performance. Our EDIC method can also be readily
incorporated with the Deep Video Compression (DVC) framework to further improve
the video compression performance. Simultaneously, our EDIC method boosts the
coding performance significantly while bringing slightly increased
computational cost. More importantly, experimental results demonstrate that the
proposed approach outperforms the current state-of-the-art image compression
methods and is up to more than 150 times faster in terms of decoding speed when
compared with Minnen's method. The proposed framework also successfully
improves the performance of the recent deep video compression system DVC. Our
code will be released at https://github.com/liujiaheng/compression.Comment: We will released our code and training dat
Accurate Scene Text Detection through Border Semantics Awareness and Bootstrapping
This paper presents a scene text detection technique that exploits
bootstrapping and text border semantics for accurate localization of texts in
scenes. A novel bootstrapping technique is designed which samples multiple
'subsections' of a word or text line and accordingly relieves the constraint of
limited training data effectively. At the same time, the repeated sampling of
text 'subsections' improves the consistency of the predicted text feature maps
which is critical in predicting a single complete instead of multiple broken
boxes for long words or text lines. In addition, a semantics-aware text border
detection technique is designed which produces four types of text border
segments for each scene text. With semantics-aware text borders, scene texts
can be localized more accurately by regressing text pixels around the ends of
words or text lines instead of all text pixels which often leads to inaccurate
localization while dealing with long words or text lines. Extensive experiments
demonstrate the effectiveness of the proposed techniques, and superior
performance is obtained over several public datasets, e. g. 80.1 f-score for
the MSRA-TD500, 67.1 f-score for the ICDAR2017-RCTW, etc.Comment: 14 pages, 8 figures, accepted by ECCV 201
Deep neural networks on graph signals for brain imaging analysis
Brain imaging data such as EEG or MEG are high-dimensional spatiotemporal
data often degraded by complex, non-Gaussian noise. For reliable analysis of
brain imaging data, it is important to extract discriminative, low-dimensional
intrinsic representation of the recorded data. This work proposes a new method
to learn the low-dimensional representations from the noise-degraded
measurements. In particular, our work proposes a new deep neural network design
that integrates graph information such as brain connectivity with
fully-connected layers. Our work leverages efficient graph filter design using
Chebyshev polynomial and recent work on convolutional nets on graph-structured
data. Our approach exploits graph structure as the prior side information,
localized graph filter for feature extraction and neural networks for high
capacity learning. Experiments on real MEG datasets show that our approach can
extract more discriminative representations, leading to improved accuracy in a
supervised classification task.Comment: Accepted by ICIP 201
Model-Driven Deep Learning for Physical Layer Communications
Intelligent communication is gradually considered as the mainstream direction
in future wireless communications. As a major branch of machine learning, deep
learning (DL) has been applied in physical layer communications and has
demonstrated an impressive performance improvement in recent years. However,
most of the existing works related to DL focus on data-driven approaches, which
consider the communication system as a black box and train it by using a huge
volume of data. Training a network requires sufficient computing resources and
extensive time, both of which are rarely found in communication devices. By
contrast, model-driven DL approaches combine communication domain knowledge
with DL to reduce the demand for computing resources and training time. This
article reviews the recent advancements in the application of model-driven DL
approaches in physical layer communications, including transmission scheme,
receiver design, and channel information recovery. Several open issues for
further research are also highlighted after presenting the comprehensive
survey.Comment: 20 pages,6 figure
A machine learning approach to drug repositioning based on drug expression profiles: Applications to schizophrenia and depression/anxiety disorders
Development of new medications is a very lengthy and costly process. Finding
novel indications for existing drugs, or drug repositioning, can serve as a
useful strategy to shorten the development cycle. In this study, we present an
approach to drug discovery or repositioning by predicting indication for a
particular disease based on expression profiles of drugs, with a focus on
applications in psychiatry. Drugs that are not originally indicated for the
disease but with high predicted probabilities serve as good candidates for
repurposing. This framework is widely applicable to any chemicals or drugs with
expression profiles measured, even if the drug targets are unknown. It is also
highly flexible as virtually any supervised learning algorithms can be used. We
applied this approach to identify repositioning opportunities for schizophrenia
as well as depression and anxiety disorders. We applied various
state-of-the-art machine learning (ML) approaches for prediction, including
deep neural networks, support vector machines (SVM), elastic net, random forest
and gradient boosted machines. The performance of the five approaches did not
differ substantially, with SVM slightly outperformed the others. However,
methods with lower predictive accuracy can still reveal literature-supported
candidates that are of different mechanisms of actions. As a further
validation, we showed that the repositioning hits are enriched for psychiatric
medications considered in clinical trials. Notably, many top repositioning hits
are supported by previous preclinical or clinical studies. Finally, we propose
that ML approaches may provide a new avenue to explore drug mechanisms via
examining the variable importance of gene features
Topology optimization of 2D structures with nonlinearities using deep learning
The field of optimal design of linear elastic structures has seen many
exciting successes that resulted in new architected materials and structural
designs. With the availability of cloud computing, including high-performance
computing, machine learning, and simulation, searching for optimal nonlinear
structures is now within reach. In this study, we develop convolutional neural
network models to predict optimized designs for a given set of boundary
conditions, loads, and optimization constraints. We have considered the case of
materials with a linear elastic response with and without stress constraint.
Also, we have considered the case of materials with a hyperelastic response,
where material and geometric nonlinearities are involved. For the nonlinear
elastic case, the neo-Hookean model is utilized. For this purpose, we generate
datasets composed of the optimized designs paired with the corresponding
boundary conditions, loads, and constraints, using a topology optimization
framework to train and validate the neural network models. The developed models
are capable of accurately predicting the optimized designs without requiring an
iterative scheme and with negligible inference computational time. The
suggested pipeline can be generalized to other nonlinear mechanics scenarios
and design domains
- …