Adaptive Top-K in SGD for Communication-Efficient Distributed Learning

Distributed stochastic gradient descent (SGD) with gradient compression has become a popular communication-efficient solution for accelerating distributed learning. One commonly used method for gradient compression is Top-K sparsification, which sparsifies the gradients by a fixed degree during model training. However, there has been a lack of an adaptive approach to adjust the sparsification degree to maximize the potential of the model's performance or training speed. This paper proposes a novel adaptive Top-K in SGD framework that enables an adaptive degree of sparsification for each gradient descent step to optimize the convergence performance by balancing the trade-off between communication cost and convergence error. Firstly, an upper bound of convergence error is derived for the adaptive sparsification scheme and the loss function. Secondly, an algorithm is designed to minimize the convergence error under the communication cost constraints. Finally, numerical results on the MNIST and CIFAR-10 datasets demonstrate that the proposed adaptive Top-K algorithm in SGD achieves a significantly better convergence rate compared to state-of-the-art methods, even after considering error compensation.Comment: 6 pages, 10 figures, has been accepted by GlobeCom 202

Emma: An accurate, efficient, and multi-modality strategy for autonomous vehicle angle prediction

Autonomous driving and self-driving vehicles have become the most popular selection for customers for their convenience. Vehicle angle prediction is one of the most prevalent topics in the autonomous driving industry, that is, realizing real-time vehicle angle prediction. However, existing methods of vehicle angle prediction utilize only single-modal data to achieve model prediction, such as images captured by the camera, which limits the performance and efficiency of the prediction system. In this paper, we present Emma, a novel vehicle angle prediction strategy that achieves multi-modal prediction and is more efficient. Specifically, Emma exploits both images and inertial measurement unit (IMU) signals with a fusion network for multi-modal data fusion and vehicle angle prediction. Moreover, we design and implement a few-shot learning module in Emma for fast domain adaptation to varied scenarios (e.g., different vehicle models). Evaluation results demonstrate that Emma achieves overall 97.5% accuracy in predicting three vehicle angle parameters (yaw, pitch, and roll), which outperforms traditional single-modalities by approximately 16.7%–36.8%. Additionally, the few-shot learning module presents promising adaptive ability and shows overall 79.8% and 88.3% accuracy in 5-shot and 10-shot settings, respectively. Finally, empirical results show that Emma reduces energy consumption by 39.7% when running on the Arduino UNO board

Hα\alpha chromospheric activity of F-, G-, and K-type stars observed by the LAMOST Medium-Resolution Spectroscopic Survey

Distribution of stellar $\mathrm{H}\alpha$ chromospheric activity with respect to stellar atmospheric parameters (effective temperature $T_\mathrm{eff}$, surface gravity $\log\,g$, and metallicity $\mathrm{[Fe/H]}$) and main-sequence/giant categories is investigated for the F-, G-, and K-type stars observed by the LAMOST Medium-Resolution Spectroscopic Survey (MRS). A total of 329,294 MRS spectra from LAMOST DR8 are utilized in the analysis. The $\mathrm{H}\alpha$ activity index ($I_{\mathrm{H}{\alpha}}$) and the $\mathrm{H}\alpha$ $R$-index ($R_{\mathrm{H}{\alpha}}$) are evaluated for the MRS spectra. The $\mathrm{H}\alpha$ chromospheric activity distributions with individual stellar parameters as well as in the $T_\mathrm{eff}$ -- $\log\,g$ and $T_\mathrm{eff}$ -- $\mathrm{[Fe/H]}$ parameter spaces are analyzed based on the $R_{\mathrm{H}{\alpha}}$ index data. It is found that: (1) for the main-sequence sample, the $R_{\mathrm{H}{\alpha}}$ distribution with $T_\mathrm{eff}$ has a bowl-shaped lower envelope with a minimum at about 6200 K, a hill-shaped middle envelope with a maximum at about 5600 K, and an upper envelope continuing to increase from hotter to cooler stars; (2) for the giant sample, the middle and upper envelopes of the $R_{\mathrm{H}{\alpha}}$ distribution first increase with decrease of $T_\mathrm{eff}$ and then drop to a lower activity level at about 4300 K, revealing the different activity characteristics at different stages of stellar evolution; (3) for both the main-sequence and giant samples, the upper envelope of the $R_{\mathrm{H}{\alpha}}$ distribution with metallicity is higher for stars with $\mathrm{[Fe/H]}$ greater than about $-1.0$, and the lowest-metallicity stars hardly exhibit high $\mathrm{H}\alpha$ indices. A dataset of $\mathrm{H}\alpha$ activity indices for the LAMOST MRS spectra analyzed is provided with this paper.Comment: 32 pages, 12 figures, 1 table, accepted for publication in Astrophysics and Space Scienc

Large-scale Huygens metasurfaces for holographic 3D near-eye displays

Novel display technologies aim at providing the users with increasingly immersive experiences. In this regard, it is a long-sought dream to generate three-dimensional (3D) scenes with high resolution and continuous depth, which can be overlaid with the real world. Current attempts to do so, however, fail in providing either truly 3D information, or a large viewing area and angle, strongly limiting the user immersion. Here, we report a proof-of-concept solution for this problem, and realize a compact holographic 3D near-eye display with a large exit pupil of 10mm x 8.66mm. The 3D image is generated from a highly transparent Huygens metasurface hologram with large (>10^8) pixel count and subwavelength pixels, fabricated via deep-ultraviolet immersion photolithography on 300 mm glass wafers. We experimentally demonstrate high quality virtual 3D scenes with ~50k active data points and continuous depth ranging from 0.5m to 2m, overlaid with the real world and easily viewed by naked eye. To do so, we introduce a new design method for holographic near-eye displays that, inherently, is able to provide both parallax and accommodation cues, fundamentally solving the vergence-accommodation conflict that exists in current commercial 3D displays.Comment: 21 pages, 9 figure

A new chromosome-scale duck genome shows a major histocompatibility complex with several expanded multigene families

BACKGROUND: The duck (Anas platyrhynchos) is one of the principal natural hosts of influenza A virus (IAV), harbors almost all subtypes of IAVs and resists to many IAVs which cause extreme virulence in chicken and human. However, the response of duck's adaptive immune system to IAV infection is poorly characterized due to lack of a detailed gene map of the major histocompatibility complex (MHC).RESULTS: We herein reported a chromosome-scale Beijing duck assembly by integrating Nanopore, Bionano, and Hi-C data. This new reference genome SKLA1.0 covers 40 chromosomes, improves the contig N50 of the previous duck assembly with highest contiguity (ZJU1.0) of more than a 5.79-fold, surpasses the chicken and zebra finch references in sequence contiguity and contains a complete genomic map of the MHC. Our 3D MHC genomic map demonstrated that gene family arrangement in this region was primordial; however, families such as AnplMHCI, AnplMHCIIβ, AnplDMB, NKRL (NK cell receptor-like genes) and BTN underwent gene expansion events making this area complex. These gene families are distributed in two TADs and genes sharing the same TAD may work in a co-regulated model.CONCLUSIONS: These observations supported the hypothesis that duck's adaptive immunity had been optimized with expanded and diversified key immune genes which might help duck to combat influenza virus. This work provided a high-quality Beijing duck genome for biological research and shed light on new strategies for AIV control.</p

Two-Step Induction of Trabecular Meshwork Cells from Induced Pluripotent Stem Cells for Glaucoma

Glaucoma is a leading cause of irreversible blindness worldwide. Reducing intraocular pressure is currently the only effective treatment. Elevated intraocular pressure is associated with increased resistance of the outflow pathway, mainly the trabecular meshwork (TM). Despite great progress in the field, the development of novel and effective treatment for glaucoma is still challenging. In this study, we reported that human induced pluripotent stem cells (iPSCs) can be cultured as colonies and monolayer cells expressing OCT4, alkaline phosphatase, SSEA4 and SSEA1. After induction to neural crest cells (NCCs) positive to NGFR and HNK1, the iPSCs can differentiate into TM cells. The induced iPSC-TM cells expressed TM cell marker CHI3L1, were responsive to dexamethasone treatment with increased expression of myocilin, ANGPTL7, and formed CLANs, comparable to primary TM cells. To the best of our knowledge, this is the first study that induces iPSCs to TM cells through a middle neural crest stage, which ensures a stable NCC pool and ensures the high output of the same TM cells. This system can be used to develop personalized treatments using patient-derived iPSCs, explore high throughput screening of new drugs focusing on TM response for controlling intraocular pressure, and investigate stem cell-based therapy for TM regeneration

Vibrational spectroscopy and microwave dielectric properties of AY2Si3O10 (A=Sr, Ba) ceramics for 5G applications

AY2Si3O10 (A = Sr, Ba) trisilicate ceramics were synthesized by traditional high temperature solid state reaction method. X-ray diffraction patterns and Rietveld refinement revealed that AY2Si3O10 (A = Sr, Ba) ceramics belonged to triclinic and monoclinic crystal systems with Pī and P21/m space groups, respectively. The vibrational modes of [SiO4] tetrahedra, [YO6] octahedra and [(Sr/Ba)O8] polyhedra were analyzed by Raman spectroscopy. The infrared spectroscopy fitting analysis was used to determine intrinsic dielectric properties. Excellent microwave dielectric properties were measured for SrY2Si3O10 and BaY2Si3O10 with ɛr = 9.3, Qf = 64100 GHz, τf = −31 ppm/°C and ɛr = 9.5, Qf = 65600 GHz, τf = −28 ppm/°C, respectively. Both trisilicate ceramics are considered potential candidates for 5G and mm wave technology, provided τf can be further tuned

More is less: Effect of ICF-based early progressive mobilization on severe aneurysmal subarachnoid hemorrhage in the NICU

IntroductionAneurysmal subarachnoid hemorrhage (aSAH) is a type of stroke that occurs due to a ruptured intracranial aneurysm. Although advanced therapies have been applied to treat aSAH, patients still suffer from functional impairment leading to prolonged stays in the NICU. The effect of early progressive mobilization as an intervention implemented in the ICU setting for critically ill patients remains unclear.MethodsThis retrospective study evaluated ICF-based early progressive mobilization's validity, safety, and feasibility in severe aSAH patients. Sixty-eight patients with aSAH with Hunt-Hess grades III-IV were included. They were divided into two groups—progressive mobilization and passive movement. Patients in the progressive mobilization group received progressive ICF-based mobilization intervention, and those in the passive movement group received passive joint movement training. The incidence of pneumonia, duration of mechanical ventilation, length of NICU stay, and incidence of deep vein thrombosis were evaluated for validity. In contrast, the incidence of cerebral vasospasm, abnormally high ICP, and other safety events were assessed for safety. We also described the feasibility of the early mobilization initiation time and the rate of participation at each level for patients in the progressive mobilization group.ResultsThe results showed that the incidence of pneumonia, duration of mechanical ventilation, and length of NICU stay were significantly lower among patients in the progressive mobilization group than in the passive movement group (P = 0.031, P = 0.004, P = 0.012), but the incidence of deep vein thrombosis did not significantly differ between the two groups. Regarding safety, patients in the progressive mobilization group had a lower incidence of cerebral vasospasm than those in the passive movement group. Considering the effect of an external ventricular drain on cerebral vasospasm (P = 0.015), we further analyzed those patients in the progressive mobilization group who had a lower incidence of cerebral vasospasm in patients who did not have an external ventricular drain (P = 0.011). Although we found 2 events of abnormally increased intracranial pressure in the progressive mobilization group, there was no abnormal decrease in cerebral perfusion pressure in the 2 events. In addition, among other safety events, there was no difference in the occurrence of adverse events between the two groups (P = 0.073), but the number of potential adverse events was higher in the progressive mobilization group (P = 0.001). Regarding feasibility, patients in the progressive mobilization group were commonly initiated 72 h after admission to the NICU, and 47.06% were in the third level of the mobilization protocol.DiscussionWe conclude that the ICF-based early progressive mobilization protocol is an effective and feasible intervention tool. For validity, more mobilization interventions might lead to less pneumonia, duration of mechanical ventilation and length of stay for patients with severe aSAH in the NICU, Moreover, it is necessary to pay attention over potential adverse events (especially line problems), although we did not find serious safety events

MetaAdvDet: Towards Robust Detection of Evolving Adversarial Attacks

Deep neural networks (DNNs) are vulnerable to adversarial attack which is maliciously implemented by adding human-imperceptible perturbation to images and thus leads to incorrect prediction. Existing studies have proposed various methods to detect the new adversarial attacks. However, new attack methods keep evolving constantly and yield new adversarial examples to bypass the existing detectors. It needs to collect tens of thousands samples to train detectors, while the new attacks evolve much more frequently than the high-cost data collection. Thus, this situation leads the newly evolved attack samples to remain in small scales. To solve such few-shot problem with the evolving attack, we propose a meta-learning based robust detection method to detect new adversarial attacks with limited examples. Specifically, the learning consists of a double-network framework: a task-dedicated network and a master network which alternatively learn the detection capability for either seen attack or a new attack. To validate the effectiveness of our approach, we construct the benchmarks with few-shot-fashion protocols based on three conventional datasets, i.e. CIFAR-10, MNIST and Fashion-MNIST. Comprehensive experiments are conducted on them to verify the superiority of our approach with respect to the traditional adversarial attack detection methods.Comment: 10 pages, 2 figures, accepted as the conference paper of Proceedings of the 27th ACM International Conference on Multimedia (MM'19