A Unified Approximation Framework for Compressing and Accelerating Deep Neural Networks

Deep neural networks (DNNs) have achieved significant success in a variety of real world applications, i.e., image classification. However, tons of parameters in the networks restrict the efficiency of neural networks due to the large model size and the intensive computation. To address this issue, various approximation techniques have been investigated, which seek for a light weighted network with little performance degradation in exchange of smaller model size or faster inference. Both low-rankness and sparsity are appealing properties for the network approximation. In this paper we propose a unified framework to compress the convolutional neural networks (CNNs) by combining these two properties, while taking the nonlinear activation into consideration. Each layer in the network is approximated by the sum of a structured sparse component and a low-rank component, which is formulated as an optimization problem. Then, an extended version of alternating direction method of multipliers (ADMM) with guaranteed convergence is presented to solve the relaxed optimization problem. Experiments are carried out on VGG-16, AlexNet and GoogLeNet with large image classification datasets. The results outperform previous work in terms of accuracy degradation, compression rate and speedup ratio. The proposed method is able to remarkably compress the model (with up to 4.9x reduction of parameters) at a cost of little loss or without loss on accuracy.Comment: 8 pages, 5 figures, 6 table

Joint Video Multi-Frame Interpolation and Deblurring under Unknown Exposure Time

Natural videos captured by consumer cameras often suffer from low framerate and motion blur due to the combination of dynamic scene complexity, lens and sensor imperfection, and less than ideal exposure setting. As a result, computational methods that jointly perform video frame interpolation and deblurring begin to emerge with the unrealistic assumption that the exposure time is known and fixed. In this work, we aim ambitiously for a more realistic and challenging task - joint video multi-frame interpolation and deblurring under unknown exposure time. Toward this goal, we first adopt a variant of supervised contrastive learning to construct an exposure-aware representation from input blurred frames. We then train two U-Nets for intra-motion and inter-motion analysis, respectively, adapting to the learned exposure representation via gain tuning. We finally build our video reconstruction network upon the exposure and motion representation by progressive exposure-adaptive convolution and motion refinement. Extensive experiments on both simulated and real-world datasets show that our optimized method achieves notable performance gains over the state-of-the-art on the joint video x8 interpolation and deblurring task. Moreover, on the seemingly implausible x16 interpolation task, our method outperforms existing methods by more than 1.5 dB in terms of PSNR.Comment: Accepted by CVPR 2023, available at https://github.com/shangwei5/VIDU

Convection-permitting fully coupled WRF-Hydro ensemble simulations in high mountain environment: impact of boundary layer- and lateral flow parameterizations on land–atmosphere interactions

Numerical climate models have been upgraded by the improved description of terrestrial hydrological processes across different scales. The goal of this study is to explore the role of terrestrial hydrological processes on land–atmosphere interactions within the context of modeling uncertainties related to model physics parameterization. The models applied are the Weather Research and Forecasting (WRF) model and its coupled hydrological modeling system WRF-Hydro, which depicts the lateral terrestrial hydrological processes and further allows their feedback to the atmosphere. We conducted convection-permitting simulations (3 km) over the Heihe River Basin in Northwest China for the period 2008–2010, and particularly focused on its upper reach area of complex high mountains. In order to account for the modeling uncertainties associated with model physics parameterization, an ensemble of simulations is generated by varying the planetary boundary layer (PBL) schemes. We embedded the fully three-dimensional atmospheric water tagging method in both WRF and WRF-Hydro for quantifying the strength of land–atmosphere interactions. The impact of PBL parameterization on land–atmosphere interactions is evaluated through its direct effect on vertical mixing. Results suggest that enabled lateral terrestrial flow in WRF-Hydro distinctly increases soil moisture and evapotranspiration near the surface in the high mountains, thereby modifies the atmospheric condition regardless of the applied PBL scheme. The local precipitation recycling ratio in the study area increases from 1.52 to 1.9% due to the description of lateral terrestrial flow, and such positive feedback processes are irrespective of the modeling variability caused by PBL parameterizations. This study highlights the non-negligible contribution of lateral terrestrial flow to local precipitation recycling, indicating the potential of the fully coupled modeling in land–atmosphere interactions research