418 research outputs found
Balanced Group Convolution: An Improved Group Convolution Based on Approximability Estimates
The performance of neural networks has been significantly improved by
increasing the number of channels in convolutional layers. However, this
increase in performance comes with a higher computational cost, resulting in
numerous studies focused on reducing it. One promising approach to address this
issue is group convolution, which effectively reduces the computational cost by
grouping channels. However, to the best of our knowledge, there has been no
theoretical analysis on how well the group convolution approximates the
standard convolution. In this paper, we mathematically analyze the
approximation of the group convolution to the standard convolution with respect
to the number of groups. Furthermore, we propose a novel variant of the group
convolution called balanced group convolution, which shows a higher
approximation with a small additional computational cost. We provide
experimental results that validate our theoretical findings and demonstrate the
superior performance of the balanced group convolution over other variants of
group convolution.Comment: 26pages, 2 figure
Parareal Neural Networks Emulating a Parallel-in-time Algorithm
As deep neural networks (DNNs) become deeper, the training time increases. In
this perspective, multi-GPU parallel computing has become a key tool in
accelerating the training of DNNs. In this paper, we introduce a novel
methodology to construct a parallel neural network that can utilize multiple
GPUs simultaneously from a given DNN. We observe that layers of DNN can be
interpreted as the time step of a time-dependent problem and can be
parallelized by emulating a parallel-in-time algorithm called parareal. The
parareal algorithm consists of fine structures which can be implemented in
parallel and a coarse structure which gives suitable approximations to the fine
structures. By emulating it, the layers of DNN are torn to form a parallel
structure which is connected using a suitable coarse network. We report
accelerated and accuracy-preserved results of the proposed methodology applied
to VGG-16 and ResNet-1001 on several datasets
Which Tasks Will Technology Take? A New Systematic Methodology to Measure Task Automation
With the rapid advance of digital technologies, task automation has recently come to the forefront of the debate on skill-biased technological change. Building on a network theory, this study develops a new systematic methodology to identify comprehensive task types in the overall economy, and to quantitatively measure the degree of automation for each task type. Using comprehensive dataset on occupational skill requirements in 2015, we construct a two-mode network, and identify 13 task types using a non-parametric clustering algorithm. Our findings suggest that routine cognitive task and information processing are most automated tasks, and that flexible thinking and dynamic physical task are least susceptible to automation in 2015. The major contribution of our approach lies in the estimation of degree of automation for different task types. The methodology presents a promising avenue for evaluating the impact of automation on labor market outcomes, such as wage inequality and job polarization
- …