Sparse-SignSGD with Majority Vote for Communication-Efficient Distributed Learning

The training efficiency of complex deep learning models can be significantly improved through the use of distributed optimization. However, this process is often hindered by a large amount of communication cost between workers and a parameter server during iterations. To address this bottleneck, in this paper, we present a new communication-efficient algorithm that offers the synergistic benefits of both sparsification and sign quantization, called ${\sf S}^3$GD-MV. The workers in ${\sf S}^3$GD-MV select the top-$K$ magnitude components of their local gradient vector and only send the signs of these components to the server. The server then aggregates the signs and returns the results via a majority vote rule. Our analysis shows that, under certain mild conditions, ${\sf S}^3$GD-MV can converge at the same rate as signSGD while significantly reducing communication costs, if the sparsification parameter $K$ is properly chosen based on the number of workers and the size of the deep learning model. Experimental results using both independent and identically distributed (IID) and non-IID datasets demonstrate that the ${\sf S}^3$GD-MV attains higher accuracy than signSGD, significantly reducing communication costs. These findings highlight the potential of ${\sf S}^3$GD-MV as a promising solution for communication-efficient distributed optimization in deep learning.Comment: 13 pages, 7 figure

Bi-directional Contrastive Learning for Domain Adaptive Semantic Segmentation

We present a novel unsupervised domain adaptation method for semantic segmentation that generalizes a model trained with source images and corresponding ground-truth labels to a target domain. A key to domain adaptive semantic segmentation is to learn domain-invariant and discriminative features without target ground-truth labels. To this end, we propose a bi-directional pixel-prototype contrastive learning framework that minimizes intra-class variations of features for the same object class, while maximizing inter-class variations for different ones, regardless of domains. Specifically, our framework aligns pixel-level features and a prototype of the same object class in target and source images (i.e., positive pairs), respectively, sets them apart for different classes (i.e., negative pairs), and performs the alignment and separation processes toward the other direction with pixel-level features in the source image and a prototype in the target image. The cross-domain matching encourages domain-invariant feature representations, while the bidirectional pixel-prototype correspondences aggregate features for the same object class, providing discriminative features. To establish training pairs for contrastive learning, we propose to generate dynamic pseudo labels of target images using a non-parametric label transfer, that is, pixel-prototype correspondences across different domains. We also present a calibration method compensating class-wise domain biases of prototypes gradually during training.Comment: Accepted to ECCV 202

Strength can be controlled by edge dislocations in refractory high-entropy alloys

Energy efficiency is motivating the search for new high-temperature (high-T) metals. Some new body-centered-cubic (BCC) random multicomponent “high-entropy alloys (HEAs)” based on refractory elements (Cr-Mo-Nb-Ta-V-W-Hf-Ti-Zr) possess exceptional strengths at high temperatures but the physical origins of this outstanding behavior are not known. Here we show, using integrated in-situ neutron-diffraction (ND), high-resolution transmission electron microscopy (HRTEM), and recent theory, that the high strength and strength retention of a NbTaTiV alloy and a high-strength/low-density CrMoNbV alloy are attributable to edge dislocations. This finding is surprising because plastic flows in BCC elemental metals and dilute alloys are generally controlled by screw dislocations. We use the insight and theory to perform a computationally-guided search over 10(7) BCC HEAs and identify over 10(6) possible ultra-strong high-T alloy compositions for future exploration

Beyond 5G URLLC Evolution: New Service Modes and Practical Considerations

Ultra-reliable low latency communications (URLLC) arose to serve industrial IoT (IIoT) use cases within the 5G. Currently, it has inherent limitations to support future services. Based on state-of-the-art research and practical deployment experience, in this article, we introduce and advocate for three variants: broadband, scalable and extreme URLLC. We discuss use cases and key performance indicators and identify technology enablers for the new service modes. We bring practical considerations from the IIoT testbed and provide an outlook toward some new research directions.Comment: Submitted to IEEE Wireless Commun. Ma

Layered composite membranes based on porous PVDF coated with a thin, dense PBI layer for vanadium redox flow batteries

A commercial porous polyvinylidene fluoride membrane (pore size 0.65 μm, nominally 125 μm thick) is spray coated with 1.2–4 μm thick layers of polybenzimidazole. The area resistance of the porous support is 36.4 mΩ cm2 in 2 M sulfuric acid, in comparison to 540 mΩ cm2 for a 27 μm thick acid doped polybenzimidazole membrane, and 124 mΩ cm2 for PVDF-P20 (4 μm thick blocking layer). Addition of vanadium ions to the supporting electrolyte increases the resistance, but less than for Nafion. The expected reason is a change in the osmotic pressure when the ionic strength of the electrolyte is increased, reducing the water contents in the membrane. The orientation of the composite membranes has a strong impact. Lower permeability values are found when the blocking layer is oriented towards the vanadium-lean side in ex-situ measurements. Cells with the blocking layer on the positive side have significantly lower capacity fade, also much lower than cells using Nafion 212. The coulombic efficiency of cells with PVDF-PBI membranes (98.4%) is higher than that of cells using Nafion 212 (93.6%), whereas the voltage efficiency is just slightly lower, resulting in energy efficiencies of 85.1 and 83.3%, respectively, at 80 mA/cm2

Transcriptional Regulation of The Porcine GnRH Receptor Gene by Glucocorticoids

Binding of GnRH to its receptor (GnRHR) stimulates the synthesis and secretion of the gonadotropins, as well as up-regulation of GnRHR. Thus, the interaction between GnRH and GnRHR represents a central point for regulation of reproduction. Glucocorticoids alter reproduction by reducing GnRH responsiveness of gonadotropes within the anterior pituitary gland, potentially via transcriptional regulation of the GnRHR gene. Investigation into this mechanism, however, revealed that the murine GnRHR gene was stimulated by glucocorticoids. To determine the effect of glucocorticoids on porcine GnRHR gene expression, gonadotrope-derived αT3-1 cells were transiently transfected with a vector containing 5118 bp of 5’ flanking sequence for the porcine GnRHR gene fused to luciferase for 12 h and treated with increasing concentrations of the glucocorticoid agonist, dexamethasone (0, 1, 10, 100 and 1,000 nM) for an additional 12 h prior to harvest. Maximal induction of luciferase activity was detected at 100 nM of dexamethasone (2-fold over vehicle; P \u3c 0.05). Deletion from 274 to 323 bp of proximal promoter eliminated glucocorticoid responsiveness, suggestive of a glucocorticoid response element (GRE). Electrophoretic mobility shift assays (EMSAs) using a radiolabeled oligonucleotide spanning -290/-270 bp of proximal promoter revealed increased binding of nuclear extracts from αT3-1 cells treated with 100 nM dexamethasone compared to vehicle. Mass spectrometry analysis of isolated proteins from a pull-down using a biotinylated oligonucleotide (-290/-270 bp) identified PARP-1 as the binding component. EMSAs with either GR or PARP-1 antibodies resulted in a supershift of the specific binding complex, whereas addition of both antibodies abolished the supershift. Inhibition of p38 and ERK1/2 mitogen-activated protein kinase (MAPK) pathways decreased dexamethasone-induced promoter activity (P \u3c 0.05), indicating their involvement in glucocorticoid stimulation of the promoter. Thus, our working model for glucocorticoid responsiveness of the porcine GnRHR gene suggests that binding of glucocorticoid to its receptor (GR), triggers GR phosophorylation by p38 and ERK1/2 MAPK pathways, resulting in the recruitment of PARP-1 by phosphorylated, ligand-bound GR to a GRE located within -290/-270 bp of the porcine GnRHR promoter. Adviser: Brett R. Whit