Building recognition on subregion’s multi-scale gist feature extraction and corresponding columns information based dimensionality reduction

Peer reviewedPublisher PD

Understanding the Role of Optimization in Double Descent

The phenomenon of model-wise double descent, where the test error peaks and then reduces as the model size increases, is an interesting topic that has attracted the attention of researchers due to the striking observed gap between theory and practice \citep{Belkin2018ReconcilingMM}. Additionally, while double descent has been observed in various tasks and architectures, the peak of double descent can sometimes be noticeably absent or diminished, even without explicit regularization, such as weight decay and early stopping. In this paper, we investigate this intriguing phenomenon from the optimization perspective and propose a simple optimization-based explanation for why double descent sometimes occurs weakly or not at all. To the best of our knowledge, we are the first to demonstrate that many disparate factors contributing to model-wise double descent (initialization, normalization, batch size, learning rate, optimization algorithm) are unified from the viewpoint of optimization: model-wise double descent is observed if and only if the optimizer can find a sufficiently low-loss minimum. These factors directly affect the condition number of the optimization problem or the optimizer and thus affect the final minimum found by the optimizer, reducing or increasing the height of the double descent peak. We conduct a series of controlled experiments on random feature models and two-layer neural networks under various optimization settings, demonstrating this optimization-based unified view. Our results suggest the following implication: Double descent is unlikely to be a problem for real-world machine learning setups. Additionally, our results help explain the gap between weak double descent peaks in practice and strong peaks observable in carefully designed setups.Comment: NeurIPS Workshop 2023 Optimization for Machine Learnin

Message Passing in C-RAN: Joint User Activity and Signal Detection

In cloud radio access network (C-RAN), remote radio heads (RRHs) and users are uniformly distributed in a large area such that the channel matrix can be considered as sparse. Based on this phenomenon, RRHs only need to detect the relatively strong signals from nearby users and ignore the weak signals from far users, which is helpful to develop low-complexity detection algorithms without causing much performance loss. However, before detection, RRHs require to obtain the realtime user activity information by the dynamic grant procedure, which causes the enormous latency. To address this issue, in this paper, we consider a grant-free C-RAN system and propose a low-complexity Bernoulli-Gaussian message passing (BGMP) algorithm based on the sparsified channel, which jointly detects the user activity and signal. Since active users are assumed to transmit Gaussian signals at any time, the user activity can be regarded as a Bernoulli variable and the signals from all users obey a Bernoulli-Gaussian distribution. In the BGMP, the detection functions for signals are designed with respect to the Bernoulli-Gaussian variable. Numerical results demonstrate the robustness and effectivity of the BGMP. That is, for different sparsified channels, the BGMP can approach the mean-square error (MSE) of the genie-aided sparse minimum mean-square error (GA-SMMSE) which exactly knows the user activity information. Meanwhile, the fast convergence and strong recovery capability for user activity of the BGMP are also verified.Comment: Conference, 6 pages, 7 figures, accepted by IEEE Globecom 201

Rigidity of 3D spherical caps via μ\mu-bubbles

By using Gromov's $\mu$-bubble technique, we show that the $3$-dimensional spherical caps are rigid under perturbations that do not reduce the metric, the scalar curvature, and the mean curvature along its boundary. Several generalizations of this result will be discussed.Comment: 20 pages, 1 figure, All comments are welcom