2 research outputs found
SegBlocks: Block-Based Dynamic Resolution Networks for Real-Time Segmentation
SegBlocks reduces the computational cost of existing neural networks, by
dynamically adjusting the processing resolution of image regions based on their
complexity. Our method splits an image into blocks and downsamples blocks of
low complexity, reducing the number of operations and memory consumption. A
lightweight policy network, selecting the complex regions, is trained using
reinforcement learning. In addition, we introduce several modules implemented
in CUDA to process images in blocks. Most important, our novel BlockPad module
prevents the feature discontinuities at block borders of which existing methods
suffer, while keeping memory consumption under control. Our experiments on
Cityscapes and Mapillary Vistas semantic segmentation show that dynamically
processing images offers a better accuracy versus complexity trade-off compared
to static baselines of similar complexity. For instance, our method reduces the
number of floating-point operations of SwiftNet-RN18 by 60% and increases the
inference speed by 50%, with only 0.3% decrease in mIoU accuracy on Cityscapes.Comment: long version, 11 page
Dynamic Neural Networks: A Survey
Dynamic neural network is an emerging research topic in deep learning.
Compared to static models which have fixed computational graphs and parameters
at the inference stage, dynamic networks can adapt their structures or
parameters to different inputs, leading to notable advantages in terms of
accuracy, computational efficiency, adaptiveness, etc. In this survey, we
comprehensively review this rapidly developing area by dividing dynamic
networks into three main categories: 1) instance-wise dynamic models that
process each instance with data-dependent architectures or parameters; 2)
spatial-wise dynamic networks that conduct adaptive computation with respect to
different spatial locations of image data and 3) temporal-wise dynamic models
that perform adaptive inference along the temporal dimension for sequential
data such as videos and texts. The important research problems of dynamic
networks, e.g., architecture design, decision making scheme, optimization
technique and applications, are reviewed systematically. Finally, we discuss
the open problems in this field together with interesting future research
directions