1,412 research outputs found
SWAP: Sparse Entropic Wasserstein Regression for Robust Network Pruning
This study addresses the challenge of inaccurate gradients in computing the
empirical Fisher Information Matrix during neural network pruning. We introduce
SWAP, a formulation of Entropic Wasserstein regression (EWR) for pruning,
capitalizing on the geometric properties of the optimal transport problem. The
``swap'' of the commonly used linear regression with the EWR in optimization is
analytically demonstrated to offer noise mitigation effects by incorporating
neighborhood interpolation across data points with only marginal additional
computational cost. The unique strength of SWAP is its intrinsic ability to
balance noise reduction and covariance information preservation effectively.
Extensive experiments performed on various networks and datasets show
comparable performance of SWAP with state-of-the-art (SoTA) network pruning
algorithms. Our proposed method outperforms the SoTA when the network size or
the target sparsity is large, the gain is even larger with the existence of
noisy gradients, possibly from noisy data, analog memory, or adversarial
attacks. Notably, our proposed method achieves a gain of 6% improvement in
accuracy and 8% improvement in testing loss for MobileNetV1 with less than
one-fourth of the network parameters remaining.Comment: Published as a conference paper at ICLR 202
Transmitting Data Through Reconfigurable Intelligent Surface: A Spatial Sigma-Delta Modulation Approach
Transmitting data using the phases on reconfigurable intelligent surfaces
(RIS) is a promising solution for future energy-efficient communication
systems. Recent work showed that a virtual phased massive multiuser
multiple-input-multiple-out (MIMO) transmitter can be formed using only one
active antenna and a large passive RIS. In this paper, we are interested in
using such a system to perform MIMO downlink precoding. In this context, we may
not be able to apply conventional MIMO precoding schemes, such as the simple
zero-forcing (ZF) scheme, and we typically need to design the phase signals by
solving optimization problems with constant modulus constraints or with
discrete phase constraints, which pose challenges with high computational
complexities. In this work, we propose an alternative approach based on
Sigma-Delta () modulation, which is classically famous for its
noise-shaping ability. Specifically, first-order modulation is
applied in the spatial domain to handle phase quantization in generating
constant envelope signals. Under some mild assumptions, the proposed phased
modulator allows us to use the ZF scheme to synthesize the RIS
reflection phases with negligible complexity. The proposed approach is
empirically shown to achieve comparable bit error rate performance to the
unquantized ZF scheme
- …