Theoretical Analysis of Primal-Dual Algorithm for Non-Convex Stochastic Decentralized Optimization

In recent years, decentralized learning has emerged as a powerful tool not only for large-scale machine learning, but also for preserving privacy. One of the key challenges in decentralized learning is that the data distribution held by each node is statistically heterogeneous. To address this challenge, the primal-dual algorithm called the Edge-Consensus Learning (ECL) was proposed and was experimentally shown to be robust to the heterogeneity of data distributions. However, the convergence rate of the ECL is provided only when the objective function is convex, and has not been shown in a standard machine learning setting where the objective function is non-convex. Furthermore, the intuitive reason why the ECL is robust to the heterogeneity of data distributions has not been investigated. In this work, we first investigate the relationship between the ECL and Gossip algorithm and show that the update formulas of the ECL can be regarded as correcting the local stochastic gradient in the Gossip algorithm. Then, we propose the Generalized ECL (G-ECL), which contains the ECL as a special case, and provide the convergence rates of the G-ECL in both (strongly) convex and non-convex settings, which do not depend on the heterogeneity of data distributions. Through synthetic experiments, we demonstrate that the numerical results of both the G-ECL and ECL coincide with the convergence rate of the G-ECL

Embarrassingly Simple Text Watermarks

We propose Easymark, a family of embarrassingly simple yet effective watermarks. Text watermarking is becoming increasingly important with the advent of Large Language Models (LLM). LLMs can generate texts that cannot be distinguished from human-written texts. This is a serious problem for the credibility of the text. Easymark is a simple yet effective solution to this problem. Easymark can inject a watermark without changing the meaning of the text at all while a validator can detect if a text was generated from a system that adopted Easymark or not with high credibility. Easymark is extremely easy to implement so that it only requires a few lines of code. Easymark does not require access to LLMs, so it can be implemented on the user-side when the LLM providers do not offer watermarked LLMs. In spite of its simplicity, it achieves higher detection accuracy and BLEU scores than the state-of-the-art text watermarking methods. We also prove the impossibility theorem of perfect watermarking, which is valuable in its own right. This theorem shows that no matter how sophisticated a watermark is, a malicious user could remove it from the text, which motivate us to use a simple watermark such as Easymark. We carry out experiments with LLM-generated texts and confirm that Easymark can be detected reliably without any degradation of BLEU and perplexity, and outperform state-of-the-art watermarks in terms of both quality and reliability

Reconstruction of ancestral L-amino acid oxidases to broaden substrate selectivity

Characteristic functions of enzymes, such as high thermal stability and substrate specificity, are attained during the evolutionary process. Ancestral sequence reconstruction (ASR) is applied to infer the process by designing artificial enzymes which are located on ancestral node of phylogenetic tree; here, the inferred enzymes called ancestral enzymes. Ancestral enzymes often exhibit substrate promiscuity and high thermal stability of which functions are suitable to perform enzyme engineering. In addition, applicability of the ASR is high because the method requires only sequence data to design ancestral enzymes. Thus, we believe that artificial enzymes contributing to progress in enzyme engineering can be designed by ASR. Please click Additional Files below to see the full abstract

Momentum Tracking: Momentum Acceleration for Decentralized Deep Learning on Heterogeneous Data

SGD with momentum acceleration is one of the key components for improving the performance of neural networks. For decentralized learning, a straightforward approach using momentum acceleration is Distributed SGD (DSGD) with momentum acceleration (DSGDm). However, DSGDm performs worse than DSGD when the data distributions are statistically heterogeneous. Recently, several studies have addressed this issue and proposed methods with momentum acceleration that are more robust to data heterogeneity than DSGDm, although their convergence rates remain dependent on data heterogeneity and decrease when the data distributions are heterogeneous. In this study, we propose Momentum Tracking, which is a method with momentum acceleration whose convergence rate is proven to be independent of data heterogeneity. More specifically, we analyze the convergence rate of Momentum Tracking in the standard deep learning setting, where the objective function is non-convex and the stochastic gradient is used. Then, we identify that it is independent of data heterogeneity for any momentum coefficient $\beta\in [0, 1)$. Through image classification tasks, we demonstrate that Momentum Tracking is more robust to data heterogeneity than the existing decentralized learning methods with momentum acceleration and can consistently outperform these existing methods when the data distributions are heterogeneous

How are fine sediments described in sediment sheet flow?

Stony debris flow transits to sediment sheet flow when the river bed gradient becomes gentle. The sediment sheet flow consists of a water flow layer and a sediment moving layer. Fine sediments are expected to behave as a part of the fluid rather than a solid phase in the sediment moving layer. Further, it can be thought that a part of fine sediment can be suspended in the water flow layer. However, it was not possible to physically express whether the fine sediment behaves as a solid phase or a fluid phase in the numerical simulation model. Here we physically modeled fine sediment behavior in sediment sheet flow. We confirmed the applicability of the new model to describe the longitudinal deposited sediment gradient in flume experiments

A rapid and enhanced DNA detection method for crop cultivar discrimination

In many crops species, the development of a rapid and precise cultivar discrimination system has been required for plant breeding and patent protection of plant cultivars and agricultural products. Here, we successfully evaluated strawberry cultivars via a novel method, namely, the single tag hybridization (STH) chromatographic printed array strip (PAS) using the PCR products of eight genomic regions. In a previous study, we showed that genotyping of eight genomic regions derived from FaRE1 retrotransposon insertion site enabled to discriminate 32 strawberry cultivars precisely, however, this method required agarose/acrylamide gel electrophoresis, thus has the difficulty for practical application. In contrast, novel DNA detection method in this study has some great advantages over standard DNA detection methods, including agarose/acrylamide gel electrophoresis, because it produces signals for DNA detection with dramatically higher sensitivity in a shorter time without any preparation or staining of a gel. Moreover, this method enables the visualization of multiplex signals simultaneously in a single reaction using several independent amplification products. We expect that this novel method will become a rapid and convenient cultivar screening assay for practical purposes, and will be widely applied to various situations, including laboratory research, and on-site inspection of plant cultivars and agricultural products

Transcutaneous Electrical Nerve Stimulation on the PC-5 and PC-6 Points Alleviated Hypotension after Epidural Anaesthesia, Depending on the Stimulus Frequency

Neuraxial blockade causes arterial hypotension. Transcutaneous electrical nerve stimulation (TENS) at the Neiguan (PC-6) and Jianshi (PC-5) reduces the severity of hypotension after spinal anaesthesia, but did not clarify the optimal stimulus frequency. We hypothesized that the stimulus frequency of TENS at the PC-6 and PC-5 points would influence the severity of hypotension after epidural anaesthesia. 65 ASA I or II male patients presenting for inguinal hernia repair were randomized to five groups: the control group received no treatment; the 2 Hz, 10 Hz, 20 Hz, and 40 Hz groups received TENS at a frequency of 2 Hz, 10 Hz, 20 Hz, and 40 Hz, respectively. The lowest SBP was significantly higher in the 40 Hz group [the control, 84 (74–110) mmHg; the 2 Hz, 96 (62–116) mmHg; the 10 Hz, 100 (68–110) mmHg; the 20 Hz, 96 (64–115) mmHg; the 40 Hz, 104 (75–140) mmHg: P = 0.004]. Significantly less patients experienced hypotension in the 40 Hz group [the control, 78%; the 2 Hz, 43%; the 10 Hz, 38%; the 20 Hz, 38%; the 40 Hz, 8%: P = 0.008]. TENS on the PC-6 and PC-5 points reduced the severity and incidence of hypotension after epidural anaesthesia, depending on the stimulus frequency

MAP Kinase Pathways in Brain Endothelial Cells and Crosstalk with Pericytes and Astrocytes Mediate Contrast-Induced Blood–Brain Barrier Disruption

Neurointervention with contrast media (CM) has rapidly increased, but the impact of CM extravasation and the related side effects remain controversial. This study investigated the effect of CM on blood–brain barrier (BBB) integrity. We established in vitro BBB models using primary cultures of rat BBB-related cells. To assess the effects of CM on BBB functions, we evaluated transendothelial electrical resistance, permeability, and tight junction (TJ) protein expression using immunohistochemistry (IHC) and Western blotting. To investigate the mechanism of iopamidol-induced barrier dysfunction, the role of mitogen-activated protein (MAP) kinases in brain endothelial cells was examined. We assessed the effect of conditioned medium derived from astrocytes and pericytes under iopamidol treatment. Short-term iopamidol exposure on the luminal side induced transient, while on the abluminal side caused persistent BBB dysfunction. IHC and immunoblotting revealed CM decreased the expression of TJ proteins. Iopamidol-induced barrier dysfunction was improved via the regulation of MAP kinase pathways. Conditioned medium from CM-exposed pericytes or astrocytes lacks the ability to enhance barrier function. CM may cause BBB dysfunction. MAP kinase pathways in brain endothelial cells and the interactions of astrocytes and pericytes mediate iopamidol-induced barrier dysfunction. CM extravasation may have negative effects on clinical outcomes in patients

ACE2 knockout hinders SARS-CoV-2 propagation in iPS cell-derived airway and alveolar epithelial cells

Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), the causative agent of COVID-19, continues to spread around the world with serious cases and deaths. It has also been suggested that different genetic variants in the human genome affect both the susceptibility to infection and severity of disease in COVID-19 patients. Angiotensin-converting enzyme 2 (ACE2) has been identified as a cell surface receptor for SARS-CoV and SARS-CoV-2 entry into cells. The construction of an experimental model system using human iPS cells would enable further studies of the association between viral characteristics and genetic variants. Airway and alveolar epithelial cells are cell types of the lung that express high levels of ACE2 and are suitable for in vitro infection experiments. Here, we show that human iPS cell-derived airway and alveolar epithelial cells are highly susceptible to viral infection of SARS-CoV-2. Using gene knockout with CRISPR-Cas9 in human iPS cells we demonstrate that ACE2 plays an essential role in the airway and alveolar epithelial cell entry of SARS-CoV-2 in vitro. Replication of SARS-CoV-2 was strongly suppressed in ACE2 knockout (KO) lung cells. Our model system based on human iPS cell-derived lung cells may be applied to understand the molecular biology regulating viral respiratory infection leading to potential therapeutic developments for COVID-19 and the prevention of future pandemics