Decentralized Stochastic Bilevel Optimization with Improved per-Iteration Complexity

Bilevel optimization recently has received tremendous attention due to its great success in solving important machine learning problems like meta learning, reinforcement learning, and hyperparameter optimization. Extending single-agent training on bilevel problems to the decentralized setting is a natural generalization, and there has been a flurry of work studying decentralized bilevel optimization algorithms. However, it remains unknown how to design the distributed algorithm with sample complexity and convergence rate comparable to SGD for stochastic optimization, and at the same time without directly computing the exact Hessian or Jacobian matrices. In this paper we propose such an algorithm. More specifically, we propose a novel decentralized stochastic bilevel optimization (DSBO) algorithm that only requires first order stochastic oracle, Hessian-vector product and Jacobian-vector product oracle. The sample complexity of our algorithm matches the currently best known results for DSBO, and the advantage of our algorithm is that it does not require estimating the full Hessian and Jacobian matrices, thereby having improved per-iteration complexity.Comment: ICML 202

A cost-effective colourimetric assay for quantifying hydrogen peroxide in honey

Honey is a natural product with many beneficial properties including antimicrobial action. Production of hydrogen peroxide (H2O2) in diluted honey is central to this action. Here, we describe an optimized method for measuring levels of H2O2 in honey. This method is based on established methods, with the level of dilution, the time between dilution and reading the assay, and aeration of the samples during the assay identified as critical points for ensuring reliability and reproducibility. The method is cost-effective and easy to perform using common laboratory equipment. Using this method, we quantified the hydrogen peroxide content of five different, unprocessed polyfloral honeys collected in NC, USA. Our results show that H2O2 production by these honeys varies greatly, with some samples producing negligible levels of H2O2. We assessed the effect of colour on the assay by measuring the recovery of spiked H2O2 from light and dark honey and from serially diluted dark corn syrup, and found the amount of H2O2 that could be detected was lower in dark corn syrup and darker honey samples

Two Disease-Causing SNAP-25B Mutations Selectively Impair SNARE C-terminal Assembly

Synaptic exocytosis relies on assembly of three soluble N-ethylmaleimide-sensitive factor attachment receptor (SNARE) proteins into a parallel four-helix bundle to drive membrane fusion. SNARE assembly occurs by stepwise zippering of the vesicle-associated SNARE (v-SNARE) onto a binary SNARE complex on the target plasma membrane (t-SNARE). Zippering begins with slow N-terminal association followed by rapid C-terminal zippering, which serves as a power stroke to drive membrane fusion. SNARE mutations have been associated with numerous diseases, especially neurological disorders. It remains unclear how these mutations affect SNARE zippering, partly due to difficulties to quantify the energetics and kinetics of SNARE assembly. Here, we used single-molecule optical tweezers to measure the assembly energy and kinetics of SNARE complexes containing single mutations I67T/N in neuronal SNARE synaptosomal-associated protein of 25 kDa (SNAP-25B), which disrupt neurotransmitter release and have been implicated in neurological disorders. We found that both mutations significantly reduced the energy of C-terminal zippering by ~ 10 kBT, but did not affect N-terminal assembly. In addition, we observed that both mutations lead to unfolding of the C-terminal region in the t-SNARE complex. Our findings suggest that both SNAP-25B mutations impair synaptic exocytosis by destabilizing SNARE assembly, rather than stabilizing SNARE assembly as previously proposed. Therefore, our measurements provide insights into the molecular mechanism of the disease caused by SNARE mutations

CBX4-mediated SUMO modification regulates BMI1 recruitment at sites of DNA damage

Polycomb group (PcG) proteins are involved in epigenetic silencing where they function as major determinants of cell identity, stem cell pluripotency and the epigenetic gene silencing involved in cancer development. Recently numerous PcG proteins, including CBX4, have been shown to accumulate at sites of DNA damage. However, it remains unclear whether or not CBX4 or its E3 sumo ligase activity is directly involved in the DNA damage response (DDR). Here we define a novel role for CBX4 as an early DDR protein that mediates SUMO conjugation at sites of DNA lesions. DNA damage stimulates sumoylation of BMI1 by CBX4 at lysine 88, which is required for the accumulation of BMI1 at DNA damage sites. Moreover, we establish that CBX4 recruitment to the sites of laser micro-irradiation-induced DNA damage requires PARP activity but does not require H2AX, RNF8, BMI1 nor PI-3-related kinases. The importance of CBX4 in the DDR was confirmed by the depletion of CBX4, which resulted in decreased cellular resistance to ionizing radiation. Our results reveal a direct role for CBX4 in the DDR pathway

A Flexible Approach for Highly Multiplexed Candidate Gene Targeted Resequencing

We have developed an integrated strategy for targeted resequencing and analysis of gene subsets from the human exome for variants. Our capture technology is geared towards resequencing gene subsets substantially larger than can be done efficiently with simplex or multiplex PCR but smaller in scale than exome sequencing. We describe all the steps from the initial capture assay to single nucleotide variant (SNV) discovery. The capture methodology uses in-solution 80-mer oligonucleotides. To provide optimal flexibility in choosing human gene targets, we designed an in silico set of oligonucleotides, the Human OligoExome, that covers the gene exons annotated by the Consensus Coding Sequencing Project (CCDS). This resource is openly available as an Internet accessible database where one can download capture oligonucleotides sequences for any CCDS gene and design custom capture assays. Using this resource, we demonstrated the flexibility of this assay by custom designing capture assays ranging from 10 to over 100 gene targets with total capture sizes from over 100 Kilobases to nearly one Megabase. We established a method to reduce capture variability and incorporated indexing schemes to increase sample throughput. Our approach has multiple applications that include but are not limited to population targeted resequencing studies of specific gene subsets, validation of variants discovered in whole genome sequencing surveys and possible diagnostic analysis of disease gene subsets. We also present a cost analysis demonstrating its cost-effectiveness for large population studies

Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models

Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers. In this paper we discuss the SW/HW co-designed solution for high-performance distributed training of large-scale DLRMs. We introduce a high-performance scalable software stack based on PyTorch and pair it with the new evolution of Zion platform, namely ZionEX. We demonstrate the capability to train very large DLRMs with up to 12 Trillion parameters and show that we can attain 40X speedup in terms of time to solution over previous systems. We achieve this by (i) designing the ZionEX platform with dedicated scale-out network, provisioned with high bandwidth, optimal topology and efficient transport (ii) implementing an optimized PyTorch-based training stack supporting both model and data parallelism (iii) developing sharding algorithms capable of hierarchical partitioning of the embedding tables along row, column dimensions and load balancing them across multiple workers; (iv) adding high-performance core operators while retaining flexibility to support optimizers with fully deterministic updates (v) leveraging reduced precision communications, multi-level memory hierarchy (HBM+DDR+SSD) and pipelining. Furthermore, we develop and briefly comment on distributed data ingestion and other supporting services that are required for the robust and efficient end-to-end training in production environments

Investigations into a putative role for the novel BRASSIKIN pseudokinases in compatible pollen-stigma interactions in Arabidopsis thaliana.

BACKGROUND: In the Brassicaceae, the early stages of compatible pollen-stigma interactions are tightly controlled with early checkpoints regulating pollen adhesion, hydration and germination, and pollen tube entry into the stigmatic surface. However, the early signalling events in the stigma which trigger these compatible interactions remain unknown. RESULTS: A set of stigma-expressed pseudokinase genes, termed BRASSIKINs (BKNs), were identified and found to be present in only core Brassicaceae genomes. In Arabidopsis thaliana Col-0, BKN1 displayed stigma-specific expression while the BKN2 gene was expressed in other tissues as well. CRISPR deletion mutations were generated for the two tandemly linked BKNs, and very mild hydration defects were observed for wild-type Col-0 pollen when placed on the bkn1/2 mutant stigmas. In further analyses, the predominant transcript for the stigma-specific BKN1 was found to have a premature stop codon in the Col-0 ecotype, but a survey of the 1001 Arabidopsis genomes uncovered three ecotypes that encoded a full-length BKN1 protein. Furthermore, phylogenetic analyses identified intact BKN1 orthologues in the closely related outcrossing Arabidopsis species, A. lyrata and A. halleri. Finally, the BKN pseudokinases were found to be plasma-membrane localized through the dual lipid modification of myristoylation and palmitoylation, and this localization would be consistent with a role in signaling complexes. CONCLUSION: In this study, we have characterized the novel Brassicaceae-specific family of BKN pseudokinase genes, and examined the function of BKN1 and BKN2 in the context of pollen-stigma interactions in A. thaliana Col-0. Additionally, premature stop codons were identified in the predicted stigma specific BKN1 gene in a number of the 1001 A. thaliana ecotype genomes, and this was in contrast to the out-crossing Arabidopsis species which carried intact copies of BKN1. Thus, understanding the function of BKN1 in other Brassicaceae species will be a key direction for future studies

A census of baryons in the Universe from localized fast radio bursts

More than three quarters of the baryonic content of the Universe resides in a highly diffuse state that is difficult to observe, with only a small fraction directly observed in galaxies and galaxy clusters. Censuses of the nearby Universe have used absorption line spectroscopy to observe these invisible baryons, but these measurements rely on large and uncertain corrections and are insensitive to the majority of the volume, and likely mass. Specifically, quasar spectroscopy is sensitive either to only the very trace amounts of Hydrogen that exists in the atomic state, or highly ionized and enriched gas in denser regions near galaxies. Sunyaev-Zel'dovich analyses provide evidence of some of the gas in filamentary structures and studies of X-ray emission are most sensitive to gas near galaxy clusters. Here we report the direct measurement of the baryon content of the Universe using the dispersion of a sample of localized fast radio bursts (FRBs), thus utilizing an effect that measures the electron column density along each sight line and accounts for every ionised baryon. We augment the sample of published arcsecond-localized FRBs with a further four new localizations to host galaxies which have measured redshifts of 0.291, 0.118, 0.378 and 0.522, completing a sample sufficiently large to account for dispersion variations along the line of sight and in the host galaxy environment to derive a cosmic baryon density of $\Omega_{b} = 0.051_{-0.025}^{+0.021} \, h_{70}^{-1}$ (95% confidence). This independent measurement is consistent with Cosmic Microwave Background and Big Bang Nucleosynthesis values.Comment: Published online in Nature 27 May, 202