Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models

Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers. In this paper we discuss the SW/HW co-designed solution for high-performance distributed training of large-scale DLRMs. We introduce a high-performance scalable software stack based on PyTorch and pair it with the new evolution of Zion platform, namely ZionEX. We demonstrate the capability to train very large DLRMs with up to 12 Trillion parameters and show that we can attain 40X speedup in terms of time to solution over previous systems. We achieve this by (i) designing the ZionEX platform with dedicated scale-out network, provisioned with high bandwidth, optimal topology and efficient transport (ii) implementing an optimized PyTorch-based training stack supporting both model and data parallelism (iii) developing sharding algorithms capable of hierarchical partitioning of the embedding tables along row, column dimensions and load balancing them across multiple workers; (iv) adding high-performance core operators while retaining flexibility to support optimizers with fully deterministic updates (v) leveraging reduced precision communications, multi-level memory hierarchy (HBM+DDR+SSD) and pipelining. Furthermore, we develop and briefly comment on distributed data ingestion and other supporting services that are required for the robust and efficient end-to-end training in production environments

Gas Sensors Based on Semiconducting Metal Oxide One-Dimensional Nanostructures

This article provides a comprehensive review of recent (2008 and 2009) progress in gas sensors based on semiconducting metal oxide one-dimensional (1D) nanostructures. During last few years, gas sensors based on semiconducting oxide 1D nanostructures have been widely investigated. Additionally, modified or doped oxide nanowires/nanobelts have also been synthesized and used for gas sensor applications. Moreover, novel device structures such as electronic noses and low power consumption self-heated gas sensors have been invented and their gas sensing performance has also been evaluated. Finally, we also point out some challenges for future investigation and practical application

Inflammation gene variants and susceptibility to albuminuria in the U.S. population: analysis in the Third National Health and Nutrition Examination Survey (NHANES III), 1991-1994

<p>Abstract</p> <p>Background</p> <p>Albuminuria, a common marker of kidney damage, serves as an important predictive factor for the progression of kidney disease and for the development of cardiovascular disease. While the underlying etiology is unclear, chronic, low-grade inflammation is a suspected key factor. Genetic variants within genes involved in inflammatory processes may, therefore, contribute to the development of albuminuria.</p> <p>Methods</p> <p>We evaluated 60 polymorphisms within 27 inflammatory response genes in participants from the second phase (1991-1994) of the Third National Health and Nutrition Examination Survey (NHANES III), a population-based and nationally representative survey of the United States. Albuminuria was evaluated as logarithm-transformed albumin-to-creatinine ratio (ACR), as ACR ≥ 30 mg/g, and as ACR above sex-specific thresholds. Multivariable linear regression and haplotype trend analyses were conducted to test for genetic associations in 5321 participants aged 20 years or older. Differences in allele and genotype distributions among non-Hispanic whites, non-Hispanic blacks, and Mexican Americans were tested in additive and codominant genetic models.</p> <p>Results</p> <p>Variants in several genes were found to be marginally associated (uncorrected P value < 0.05) with log(ACR) in at least one race/ethnic group, but none remained significant in crude or fully-adjusted models when correcting for the false-discovery rate (FDR). In analyses of sex-specific albuminuria, <it>IL1B </it>(rs1143623) among Mexican Americans remained significantly associated with increased odds, while <it>IL1B </it>(rs1143623), <it>CRP </it>(rs1800947) and <it>NOS3 </it>(rs2070744) were significantly associated with ACR ≥ 30 mg/g in this population (additive models, FDR-P < 0.05). In contrast, no variants were found to be associated with albuminuria among non-Hispanic blacks after adjustment for multiple testing. The only variant among non-Hispanic whites significantly associated with any outcome was <it>TNF </it>rs1800750, which failed the test for Hardy-Weinberg proportions in this population. Haplotypes within <it>MBL2</it>, <it>CRP</it>, <it>ADRB2, IL4R</it>, <it>NOS3</it>, and <it>VDR </it>were significantly associated (FDR-P < 0.05) with log(ACR) or albuminuria in at least one race/ethnic group.</p> <p>Conclusions</p> <p>Our findings suggest a small role for genetic variation within inflammation-related genes to the susceptibility to albuminuria. Additional studies are needed to further assess whether genetic variation in these, and untested, inflammation genes alter the susceptibility to kidney damage.</p

Myc and cell cycle control

Soon after the discovery of the Myc gene (c-Myc), it became clear thatMyc expression levels tightly correlate to cell proliferation. The entry in cell cycle of quiescent cells upon Myc enforced expression has been described in manymodels. Also, the downregulation or inactivation ofMyc results in the impairment of cell cycle progression. Given the frequent deregulation of Myc oncogene in human cancer it is important to dissect out the mechanisms underlying the role ofMyc on cell cycle control. Several parallel mechanisms account forMyc-mediated stimulation of the cell cycle. First,most of the critical positive cell cycle regulators are encoded by genes induced byMyc. These Myc target genes include Cdks, cyclins and E2F transcription factors. Apart from its direct effects on the transcription, Myc is able to hyperactivate cyclin/Cdk complexes through the induction of Cdk activating kinase (CAK) and Cdc25 phosphatases. Moreover, Myc antagonizes the activity of cell cycle inhibitors as p21 and p27 through different mechanisms. Thus, Myc is able to block p21 transcription or to induce Skp2, a protein involved in p27 degradation. Finally, Myc induces DNA replication by binding to replication origins and by upregulating genes encoding proteins required for replication initiation. Myc also regulates genes involved in the mitotic control. A promising approach to treat tumors with deregulated Myc is the synthetic lethality based on the inhibition of Cdks. Thus, the knowledge of the Myc-dependent cell cycle regulatory mechanisms will help to discover new therapeutic approaches directed against malignancies with deregulated Myc. This article is part of a Special Issue entitled: Myc proteins in cell biology and pathology.The work in the laboratory of the authors is funded by grants SAF11-23796 from Spanish Ministry of Industry and Innovation, and ISCIII-RETIC RD12/0036/0033 from Spanish Ministry of Health to JL, and FIS 11/00397 to MDD. GB is recipient of a fellowship form the FPI Program. We apologize to colleagues whose work has not been cited in the form of their original papers but in reviews and whose work has not been discussed due to space limitations or unintentional omission

Robust estimation of bacterial cell count from optical density

Optical density (OD) is widely used to estimate the density of cells in liquid culture, but cannot be compared between instruments without a standardized calibration protocol and is challenging to relate to actual cell count. We address this with an interlaboratory study comparing three simple, low-cost, and highly accessible OD calibration protocols across 244 laboratories, applied to eight strains of constitutive GFP-expressing E. coli. Based on our results, we recommend calibrating OD to estimated cell count using serial dilution of silica microspheres, which produces highly precise calibration (95.5% of residuals &lt;1.2-fold), is easily assessed for quality control, also assesses instrument effective linear range, and can be combined with fluorescence calibration to obtain units of Molecules of Equivalent Fluorescein (MEFL) per cell, allowing direct comparison and data fusion with flow cytometry measurements: in our study, fluorescence per cell measurements showed only a 1.07-fold mean difference between plate reader and flow cytometry data

Global estimates on the number of people blind or visually impaired by cataract: a meta-analysis from 2000 to 2020

Background: To estimate global and regional trends from 2000 to 2020 of the number of persons visually impaired by cataract and their proportion of the total number of vision-impaired individuals. Methods: A systematic review and meta-analysis of published population studies and gray literature from 2000 to 2020 was carried out to estimate global and regional trends. We developed prevalence estimates based on modeled distance visual impairment and blindness due to cataract, producing location-, year-, age-, and sex-specific estimates of moderate to severe vision impairment (MSVI presenting visual acuity &lt;6/18, ≥3/60) and blindness (presenting visual acuity &lt;3/60). Estimates are age-standardized using the GBD standard population. Results: In 2020, among overall (all ages) 43.3 million blind and 295 million with MSVI, 17.0 million (39.6%) people were blind and 83.5 million (28.3%) had MSVI due to cataract blind 60% female, MSVI 59% female. From 1990 to 2020, the count of persons blind (MSVI) due to cataract increased by 29.7%(93.1%) whereas the age-standardized global prevalence of cataract-related blindness improved by −27.5% and MSVI increased by 7.2%. The contribution of cataract to the age-standardized prevalence of blindness exceeded the global figure only in South Asia (62.9%) and Southeast Asia and Oceania (47.9%). Conclusions: The number of people blind and with MSVI due to cataract has risen over the past 30 years, despite a decrease in the age-standardized prevalence of cataract. This indicates that cataract treatment programs have been beneficial, but population growth and aging have outpaced their impact. Growing numbers of cataract blind indicate that more, better-directed, resources are needed to increase global capacity for cataract surgery.</p

Atrasentan and renal events in patients with type 2 diabetes and chronic kidney disease (SONAR): a double-blind, randomised, placebo-controlled trial

Background: Short-term treatment for people with type 2 diabetes using a low dose of the selective endothelin A receptor antagonist atrasentan reduces albuminuria without causing significant sodium retention. We report the long-term effects of treatment with atrasentan on major renal outcomes. Methods: We did this double-blind, randomised, placebo-controlled trial at 689 sites in 41 countries. We enrolled adults aged 18–85 years with type 2 diabetes, estimated glomerular filtration rate (eGFR)25–75 mL/min per 1·73 m 2 of body surface area, and a urine albumin-to-creatinine ratio (UACR)of 300–5000 mg/g who had received maximum labelled or tolerated renin–angiotensin system inhibition for at least 4 weeks. Participants were given atrasentan 0·75 mg orally daily during an enrichment period before random group assignment. Those with a UACR decrease of at least 30% with no substantial fluid retention during the enrichment period (responders)were included in the double-blind treatment period. Responders were randomly assigned to receive either atrasentan 0·75 mg orally daily or placebo. All patients and investigators were masked to treatment assignment. The primary endpoint was a composite of doubling of serum creatinine (sustained for ≥30 days)or end-stage kidney disease (eGFR <15 mL/min per 1·73 m 2 sustained for ≥90 days, chronic dialysis for ≥90 days, kidney transplantation, or death from kidney failure)in the intention-to-treat population of all responders. Safety was assessed in all patients who received at least one dose of their assigned study treatment. The study is registered with ClinicalTrials.gov, number NCT01858532. Findings: Between May 17, 2013, and July 13, 2017, 11 087 patients were screened; 5117 entered the enrichment period, and 4711 completed the enrichment period. Of these, 2648 patients were responders and were randomly assigned to the atrasentan group (n=1325)or placebo group (n=1323). Median follow-up was 2·2 years (IQR 1·4–2·9). 79 (6·0%)of 1325 patients in the atrasentan group and 105 (7·9%)of 1323 in the placebo group had a primary composite renal endpoint event (hazard ratio [HR]0·65 [95% CI 0·49–0·88]; p=0·0047). Fluid retention and anaemia adverse events, which have been previously attributed to endothelin receptor antagonists, were more frequent in the atrasentan group than in the placebo group. Hospital admission for heart failure occurred in 47 (3·5%)of 1325 patients in the atrasentan group and 34 (2·6%)of 1323 patients in the placebo group (HR 1·33 [95% CI 0·85–2·07]; p=0·208). 58 (4·4%)patients in the atrasentan group and 52 (3·9%)in the placebo group died (HR 1·09 [95% CI 0·75–1·59]; p=0·65). Interpretation: Atrasentan reduced the risk of renal events in patients with diabetes and chronic kidney disease who were selected to optimise efficacy and safety. These data support a potential role for selective endothelin receptor antagonists in protecting renal function in patients with type 2 diabetes at high risk of developing end-stage kidney disease. Funding: AbbVie

All-sky search for continuous gravitational waves from isolated neutron stars in the early O3 LIGO data

We report on an all-sky search for continuous gravitational waves in the frequency band 20-2000 Hz and with a frequency time derivative in the range of [-1.0,+0.1]×10-8 Hz/s. Such a signal could be produced by a nearby, spinning and slightly nonaxisymmetric isolated neutron star in our Galaxy. This search uses the LIGO data from the first six months of Advanced LIGO's and Advanced Virgo's third observational run, O3. No periodic gravitational wave signals are observed, and 95% confidence-level (C.L.) frequentist upper limits are placed on their strengths. The lowest upper limits on worst-case (linearly polarized) strain amplitude h0 are ∼1.7×10-25 near 200 Hz. For a circularly polarized source (most favorable orientation), the lowest upper limits are ∼6.3×10-26. These strict frequentist upper limits refer to all sky locations and the entire range of frequency derivative values. For a population-averaged ensemble of sky locations and stellar orientations, the lowest 95% C.L. upper limits on the strain amplitude are ∼1.4×10-25. These upper limits improve upon our previously published all-sky results, with the greatest improvement (factor of ∼2) seen at higher frequencies, in part because quantum squeezing has dramatically improved the detector noise level relative to the second observational run, O2. These limits are the most constraining to date over most of the parameter space searched