Efficient-Adam: Communication-Efficient Distributed Adam

Distributed adaptive stochastic gradient methods have been widely used for large-scale nonconvex optimization, such as training deep learning models. However, their communication complexity on finding $\varepsilon$-stationary points has rarely been analyzed in the nonconvex setting. In this work, we present a novel communication-efficient distributed Adam in the parameter-server model for stochastic nonconvex optimization, dubbed {\em Efficient-Adam}. Specifically, we incorporate a two-way quantization scheme into Efficient-Adam to reduce the communication cost between the workers and server. Simultaneously, we adopt a two-way error feedback strategy to reduce the biases caused by the two-way quantization on both the server and workers, respectively. In addition, we establish the iteration complexity for the proposed Efficient-Adam with a class of quantization operators, and further characterize its communication complexity between the server and workers when an $\varepsilon$-stationary point is achieved. Finally, we apply Efficient-Adam to solve a toy stochastic convex optimization problem and train deep learning models on real-world vision and language tasks. Extensive experiments together with a theoretical guarantee justify the merits of Efficient Adam.Comment: IEEE Transactions on Signal Processin

The swimming behavior of the aquatic larva of Neoneuromus ignobilis (Megaloptera: Corydalidae: Corydalinae).

In order to explore the pattern and significance of swimming, through photos and videos we observed and recorded the swimming behavior of the aquatic larvae of Megaloptera in detail for the first time using the endemic Chinese species Neoneuromus ignobilis Navas, 1932 as the test insect, which were collected from the Dadu River and reared in nature-simulated environments. Four swimming postures are recognized and described herein in detail, i. e., vertical, parallel, back and side swimming, and these postures were used by the larvae disproportionately, with a frequency of 89.08%, 5. 49%, 4. 40% and 0. 61% , respectively. The swimming larvae tend to pose their body into an S-shape, with various degree of sinuation. By changing the directions of the head and tail, they can easily rise up or sink and change swimming postures. The propulsion was generated by the wriggling of the body while the legs were mostly held close to the body. Larvae of different instars varied greatly in swimming ability, the 6th ins tar larvae being the best and most active swimmer compared to the 2nd and final instars. The larvae may also employ complex defense behaviors not often known from relatively ancient insect groups, like chemical defense as secretion from the end of abdomen

Suppression of microglial Ccl2 reduces neuropathic pain associated with chronic spinal compression

IntroductionChronic spinal compression is a common complication of spinal cord injury (SCI), which can lead to spinal stenosis or herniated discs. The ensuing neuropathic pain is often associated with the activation of microglia. In this investigation, our objective was to explore whether modifying the levels of chemokine (C-C motif) ligand 2 (Ccl2) in microglia could alleviate neuropathic pain resulting from chronic spinal compression.MethodsWe used a public database to look for major altered gene associated in a SCI model established in rats. We then employed adeno-associated virus (AAV) vectors, expressing siRNA for the identified significantly altered gene under a microglia-specific TMEM119 promoter. We also tested the impact of this treatment in microglia in vivo on the severity of chronic spinal compression and associated pain using a ttw mouse model for progressive spinal compression.ResultsWe identified chemokine (C-C motif) ligand 2 (Ccl2) as the primary gene altered in microglia within a rat SCI model, utilizing a public database. Microglial Ccl2 levels were then found to be significantly elevated in disc specimens from SCI patients diagnosed with chronic spinal compression and strongly correlated with the Thompson classification of the degeneration level and pain score. Depletion of Ccl2 in microglia-specific TMEM119 promoter were developed to transfect mouse microglia in vitro, resulting in a proinflammatory to anti-inflammatory phenotypic adaption. In vivo depletion of Ccl2 in microglia mitigated the severity of chronic spinal compression and related pain in ttw mice, likely due to significant changes in pain-associated cytokines and factors.ConclusionDisc microglia expressing high levels of Ccl2 may contribute to chronic spinal compression and SCI-associated pain. Therapeutically targeting Ccl2 in microglia could offer a potential avenue for treating chronic spinal compression and SCI-associated pain

Configuring Intelligent Reflecting Surface with Performance Guarantees: Blind Beamforming

This work gives a blind beamforming strategy for intelligent reflecting surface (IRS), aiming to boost the received signal-to-noise ratio (SNR) by coordinating phase shifts across reflective elements in the absence of channel information. While the existing methods of IRS beamforming typically first estimate channels and then optimize phase shifts, we propose a conditional sample mean based statistical approach that explores the wireless environment via random sampling without performing any channel estimation. Remarkably, the new method just requires a polynomial number of random samples to yield an SNR boost that is quadratic in the number of reflective elements, whereas the standard random-max sampling algorithm can only achieve a linear boost under the same condition. Moreover, we gain additional insight into blind beamforming by interpreting it as a least squares problem. Field tests demonstrate the significant advantages of the proposed blind beamforming algorithm over the benchmark algorithms in enhancing wireless transmission.Comment: 16 pages, 15 figure

A Linear Time Algorithm for the Optimal Discrete IRS Beamforming

It remains an open problem to find the optimal configuration of phase shifts under the discrete constraint for intelligent reflecting surface (IRS) in polynomial time. The above problem is widely believed to be difficult because it is not linked to any known combinatorial problems that can be solved efficiently. The branch-and-bound algorithms and the approximation algorithms constitute the best results in this area. Nevertheless, this work shows that the global optimum can actually be reached in linear time in terms of the number of reflective elements (REs) of IRS. The main idea is to geometrically interpret the discrete beamforming problem as choosing the optimal point on the unit circle. Although the number of possible combinations of phase shifts grows exponentially with the number of REs, it turns out that there are merely a linear number of points on the unit circle to consider. Furthermore, the proposed algorithm can be viewed as a novel approach to a special case of the discrete quadratic program (QP).Comment: 5 page

SegViT: Semantic Segmentation with Plain Vision Transformers

We explore the capability of plain Vision Transformers (ViTs) for semantic segmentation and propose the SegVit. Previous ViT-based segmentation networks usually learn a pixel-level representation from the output of the ViT. Differently, we make use of the fundamental component -- attention mechanism, to generate masks for semantic segmentation. Specifically, we propose the Attention-to-Mask (ATM) module, in which the similarity maps between a set of learnable class tokens and the spatial feature maps are transferred to the segmentation masks. Experiments show that our proposed SegVit using the ATM module outperforms its counterparts using the plain ViT backbone on the ADE20K dataset and achieves new state-of-the-art performance on COCO-Stuff-10K and PASCAL-Context datasets. Furthermore, to reduce the computational cost of the ViT backbone, we propose query-based down-sampling (QD) and query-based up-sampling (QU) to build a Shrunk structure. With the proposed Shrunk structure, the model can save up to $40\%$ computations while maintaining competitive performance.Comment: 9 Pages, NeurIPS 202

Sympathetic feedback cooling in the optomechanical system consisting of two coupled cantilevers

We present sympathetic cooling in an optomechanical system consisting of two coupled cantilevers. The hybridization of the cantilevers creates a symmetric mode, which is feedback cooled, and an anti-symmetric mode not directly controllable by the feedback. The scheme of sympathetic cooling is adopted to cool the anti-symmetric mode indirectly by parametrically coupling to the feedback-cooled symmetric mode, from which the cooling power can be transferred. Experiment shows that the realization of coherent dynamics plays an essential role in sympathetic cooling, in which optimal cooling is achieved when the mechanical dissipation rate and the strength of coupling become comparable. The sympathetic cooling is improved by increasing the strength of mode coupling to enhance the transfer of cooling power. Also, the limit of sympathetic cooling imposed by the capacity of feedback cooling is reached as the effective temperatures of the two modes approach the strong coherent coupling condition. Our research provides the prospect of extending the cooling techniques to coupled mechanical resonators for a broad application in sensing and information processing