Unbiased Compression Saves Communication in Distributed Optimization: When and How Much?

Communication compression is a common technique in distributed optimization that can alleviate communication overhead by transmitting compressed gradients and model parameters. However, compression can introduce information distortion, which slows down convergence and incurs more communication rounds to achieve desired solutions. Given the trade-off between lower per-round communication costs and additional rounds of communication, it is unclear whether communication compression reduces the total communication cost. This paper explores the conditions under which unbiased compression, a widely used form of compression, can reduce the total communication cost, as well as the extent to which it can do so. To this end, we present the first theoretical formulation for characterizing the total communication cost in distributed optimization with communication compression. We demonstrate that unbiased compression alone does not necessarily save the total communication cost, but this outcome can be achieved if the compressors used by all workers are further assumed independent. We establish lower bounds on the communication rounds required by algorithms using independent unbiased compressors to minimize smooth convex functions and show that these lower bounds are tight by refining the analysis for ADIANA. Our results reveal that using independent unbiased compression can reduce the total communication cost by a factor of up to $\Theta(\sqrt{\min\{n, \kappa\}})$ when all local smoothness constants are constrained by a common upper bound, where $n$ is the number of workers and $\kappa$ is the condition number of the functions being minimized. These theoretical findings are supported by experimental results.Comment: Accepted by NeurIPS 202

Modified propagation of gravitational waves from the early radiation era

We study the propagation of cosmological gravitational wave (GW) backgrounds from the early radiation era until the present day in modified theories of gravity. Comparing to general relativity (GR), we study the effects that Horndeski parameters, such as the run rate of the effective Planck mass $\alpha_{\rm M}$ and the tensor speed excess $\alpha_{\rm T}$, have on the present-day GW spectrum. We use both the WKB estimate, which provides an analytical description but fails at superhorizon scales, and numerical simulations that allow us to go beyond the WKB approximation. We show that $\alpha_{\rm T}$ makes relatively insignificant changes to the GR solution, especially taking into account the constraints on its value from GW observations by the LIGO-Virgo collaboration, while $\alpha_{\rm M}$ can introduce modifications to the spectral slopes of the GW energy spectrum in the low-frequency regime depending on the considered time evolution of $\alpha_{\rm M}$. The latter effect is additional to the damping or growth occurring equally at all scales that can be predicted by the WKB approximation. In light of the recent observations by pulsar timing array collaborations and future detectors such as SKA, LISA, DECIGO, BBO, or ET, we show that, in most of the cases, constraints can not be placed on the effects of $\alpha_{\rm M}$ and the initial GW energy density $\mathcal{E}_{\rm GW}^*$ separately, but only on the combined effects of the two.Comment: 31 pages, 11 figures, 2 table

Determinants of Executive Compensation in China: The Role and Effect of Corporate Governance and Firm Performance ——Empirical Evidence from Listed Manufacturing Firms

In response to public outrage over the executive pay scandals, this paper examines the role and effect of firm performance and corporate governance (CG) on executive compensation in China based on the optimal contract theory and the managerial power theory, contributing to the empirical research on compensation in developing countries. The sample in this research includes 1078 listed manufacturing firms whose executive compensation, performance and corporate governance data from 2015 to 2018 are collected through the CSMAR and CCER databases. On top of that, quantitative analysis is employed to analyze this panel data. By conducting two-way fixed-effect regression models, this research finds the positive connections between firm performance and executive cash compensation, among which the pay-for-market-performance sensitivity is limited and generally lower than the pay-for-operation-performance sensitivity. However, it is also found that CEOs of non-state-owned enterprises (non-SOEs) might increase their pay and decouple the positive connection between executive pay and operation performance when they concurrently serve as the chairman of the board of directors. Moreover, little evidence is found to support the direct and moderating effect of the board independence, presence of compensation committee and the ownership concentration. It indicates that the quality and monitoring role of large shareholders and the board might be overestimated. Therefore, in the future policymaking process, it is necessary to find a way to truly enhance the monitoring role and to increase the bargaining power of large shareholders and the board regarding the design and implementation of optimal executive compensation contracts

Enhanced cancer therapy with cold-controlled drug release and photothermal warming enabled by one nanoplatform

Stimuli-responsive nanoparticles hold great promise for drug delivery to improve the safety and efficacy of cancer therapy. One of the most investigated stimuli-responsive strategies is to induce drug release by heating with laser, ultrasound, or electromagnetic field. More recently, cryosurgery (also called cryotherapy and cryoablation), destruction of diseased tissues by first cooling/freezing and then warming back, has been used to treat various diseases including cancer in the clinic. Here we developed a cold-responsive nanoparticle for controlled drug release as a result of the irreversible disassembly of the nanoparticle when cooled to below ∼10 °C. Furthermore, this nanoparticle can be used to generate localized heating under near infrared (NIR) laser irradiation, which can facilitate the warming process after cooling/freezing during cryosurgery. Indeed, the combination of this cold-responsive nanoparticle with ice cooling and NIR laser irradiation can greatly augment cancer destruction both in vitro and in vivo with no evident systemic toxicity

CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representations

Geo-tagged images are publicly available in large quantities, whereas labels such as object classes are rather scarce and expensive to collect. Meanwhile, contrastive learning has achieved tremendous success in various natural image and language tasks with limited labeled data. However, existing methods fail to fully leverage geospatial information, which can be paramount to distinguishing objects that are visually similar. To directly leverage the abundant geospatial information associated with images in pre-training, fine-tuning, and inference stages, we present Contrastive Spatial Pre-Training (CSP), a self-supervised learning framework for geo-tagged images. We use a dual-encoder to separately encode the images and their corresponding geo-locations, and use contrastive objectives to learn effective location representations from images, which can be transferred to downstream supervised tasks such as image classification. Experiments show that CSP can improve model performance on both iNat2018 and fMoW datasets. Especially, on iNat2018, CSP significantly boosts the model performance with 10-34% relative improvement with various labeled training data sampling ratios.Comment: In: ICML 2023, Jul 23 - 29, 2023, Honolulu, Hawaii, US

OEBench: Investigating Open Environment Challenges in Real-World Relational Data Streams

How to get insights from relational data streams in a timely manner is a hot research topic. This type of data stream can present unique challenges, such as distribution drifts, outliers, emerging classes, and changing features, which have recently been described as open environment challenges for machine learning. While existing studies have been done on incremental learning for data streams, their evaluations are mostly conducted with manually partitioned datasets. Thus, a natural question is how those open environment challenges look like in real-world relational data streams and how existing incremental learning algorithms perform on real datasets. To fill this gap, we develop an Open Environment Benchmark named OEBench to evaluate open environment challenges in relational data streams. Specifically, we investigate 55 real-world relational data streams and establish that open environment scenarios are indeed widespread in real-world datasets, which presents significant challenges for stream learning algorithms. Through benchmarks with existing incremental learning algorithms, we find that increased data quantity may not consistently enhance the model accuracy when applied in open environment scenarios, where machine learning models can be significantly compromised by missing values, distribution shifts, or anomalies in real-world data streams. The current techniques are insufficient in effectively mitigating these challenges posed by open environments. More researches are needed to address real-world open environment challenges. All datasets and code are open-sourced in https://github.com/sjtudyq/OEBench

RA-ICM: A Novel Independent Cascade Model Incorporating User Relationships and Attitudes

The rapid development of social networks has a wide range of social effects, which facilitates the study of social issues. Accurately forecasting the information propagation process within social networks is crucial for promptly understanding the event direction and effectively addressing social problems in a scientific manner. The relationships between non-adjacent users and the attitudes of users significantly influence the information propagation process within social networks. However, existing research has ignored these two elements, which poses challenges for accurately predicting the information propagation process. This limitation significantly hinders the study of emotional contagion and influence maximization in social networks. To address these issues, by considering the relationships between non-adjacent users and the influence of user attitudes, we propose a new information propagation model based on the independent cascade model. Experimental results obtained from six real Weibo datasets validate the effectiveness of the proposed model, which is reflected in increased prediction accuracy and reduced time complexity. Furthermore, the information dissemination trend in social networks predicted by the proposed model closely resembles the actual information propagation process, which demonstrates the superiority of the proposed model