Elixir: Train a Large Language Model on a Small GPU Cluster

In recent years, the number of parameters of one deep learning (DL) model has been growing much faster than the growth of GPU memory space. People who are inaccessible to a large number of GPUs resort to heterogeneous training systems for storing model parameters in CPU memory. Existing heterogeneous systems are based on parallelization plans in the scope of the whole model. They apply a consistent parallel training method for all the operators in the computation. Therefore, engineers need to pay a huge effort to incorporate a new type of model parallelism and patch its compatibility with other parallelisms. For example, Mixture-of-Experts (MoE) is still incompatible with ZeRO-3 in Deepspeed. Also, current systems face efficiency problems on small scale, since they are designed and tuned for large-scale training. In this paper, we propose Elixir, a new parallel heterogeneous training system, which is designed for efficiency and flexibility. Elixir utilizes memory resources and computing resources of both GPU and CPU. For flexibility, Elixir generates parallelization plans in the granularity of operators. Any new type of model parallelism can be incorporated by assigning a parallel pattern to the operator. For efficiency, Elixir implements a hierarchical distributed memory management scheme to accelerate inter-GPU communications and CPU-GPU data transmissions. As a result, Elixir can train a 30B OPT model on an A100 with 40GB CUDA memory, meanwhile reaching 84% efficiency of Pytorch GPU training. With its super-linear scalability, the training efficiency becomes the same as Pytorch GPU training on multiple GPUs. Also, large MoE models can be trained 5.3x faster than dense models of the same size. Now Elixir is integrated into ColossalAI and is available on its main branch

Temperature-independent thermal radiation

Thermal emission is the process by which all objects at non-zero temperatures emit light, and is well-described by the classic Planck, Kirchhoff, and Stefan-Boltzmann laws. For most solids, the thermally emitted power increases monotonically with temperature in a one-to-one relationship that enables applications such as infrared imaging and non-contact thermometry. Here, we demonstrate ultrathin thermal emitters that violate this one-to-one relationship via the use of samarium nickel oxide (SmNiO3), a strongly correlated quantum material that undergoes a fully reversible, temperature-driven solid-state phase transition. The smooth and hysteresis-free nature of this unique insulator-to-metal (IMT) phase transition allows us to engineer the temperature dependence of emissivity to precisely cancel out the intrinsic blackbody profile described by the Stefan-Boltzmann law, for both heating and cooling. Our design results in temperature-independent thermally emitted power within the long-wave atmospheric transparency window (wavelengths of 8 - 14 um), across a broad temperature range of ~30 {\deg}C, centered around ~120 {\deg}C. The ability to decouple temperature and thermal emission opens a new gateway for controlling the visibility of objects to infrared cameras and, more broadly, new opportunities for quantum materials in controlling heat transfer.Comment: Main text and supplementar

Sodium Alginate-Based Green Packaging Films Functionalized by Guava Leaf Extracts and Their Bioactivities

The aim of this work was to develop green and bioactive films with sodium alginate incorporating guava leaf extracts. Seven formulations were performed with a different sodium alginate: Guava leaf water extract (WE)/ethanolic extract (EE) proportions (100:0, 90:10, 85:15, 80:20), and glycerol were used as a plasticizer. The HPLC-PDA analysis showed the main phenolic compounds in WE were gallic acid, ellagic acid, quercetin-3-O-β-D-xylopyranoside, avicularin and quercetin. The main polyphenols in EE were rutin, isoquercitrin, quercetin-3-O-β-D-xylopyranoside, avicularin, quercitrin, quercetin and kaempferol. Guava leaf extracts could greatly enhance the antioxidant activity, antibacterial activity, tensile strength and water solubility of the sodium alginate film as well as the water barrier property, while inducing a decrease in the moisture content and elongation at the break. The FTIR and SEM analyses indicated that intermolecular hydrogen bonding between the guava leaf extract and sodium alginate resulted in a more compact structure in the composite films. These results indicated that sodium alginate-guava leaf extract films might be developed into antiradical and antimicrobial food packaging materials

Raw biomass electroreforming coupled to green hydrogen generation

AbstractDespite the tremendous progress of coupling organic electrooxidation with hydrogen generation in a hybrid electrolysis, electroreforming of raw biomass coupled to green hydrogen generation has not been reported yet due to the rigid polymeric structures of raw biomass. Herein, we electrooxidize the most abundant natural amino biopolymer chitin to acetate with over 90% yield in hybrid electrolysis. The overall energy consumption of electrolysis can be reduced by 15% due to the thermodynamically and kinetically more favorable chitin oxidation over water oxidation. In obvious contrast to small organics as the anodic reactant, the abundance of chitin endows the new oxidation reaction excellent scalability. A solar-driven electroreforming of chitin and chitin-containing shrimp shell waste is coupled to safe green hydrogen production thanks to the liquid anodic product and suppression of oxygen evolution. Our work thus demonstrates a scalable and safe process for resource upcycling and green hydrogen production for a sustainable energy future.</jats:p

Comparative Analysis of Quinolone Resistance in Clinical Isolates of Klebsiella pneumoniae and Escherichia coli from Chinese Children and Adults

The objective of this study was to compare quinolone resistance and gyrA mutations in clinical isolates of Klebsiella pneumoniae and Escherichia coli from Chinese adults who used quinolone in the preceding month and children without any known history of quinolone administration. The antimicrobial susceptibilities of 61 isolates from children and 79 isolates from adults were determined. The mutations in the quinolone resistance-determining regions in gyrA gene were detected by PCR and DNA sequencing. Fluoroquinolone resistance and types of gyrA mutations in isolates from children and adults were compared and statistically analyzed. No significant differences were detected in the resistance rates of ciprofloxacin and levofloxacin between children and adults among isolates of the two species (all P>0.05). The double mutation Ser83→Leu + Asp87→Asn in the ciprofloxacin-resistant isolates occurred in 73.7% isolates from the children and 67.9% from the adults, respectively (P=0.5444). Children with no known history of quinolone administration were found to carry fluoroquinolone-resistant Enterobacteriaceae isolates. The occurrence of ciprofloxacin resistance and the major types of gyrA mutations in the isolates from the children were similar to those from adults. The results indicate that precautions should be taken on environmental issues resulting from widespread transmission of quinolone resistance

Identification of key pharmacological components and targets for Aidi injection in the treatment of pancreatic cancer by UPLC-MS, network pharmacology, and in vivo experiments

Abstract Background Pancreatic cancer is one of the most lethal cancers worldwide. Aidi injection (ADI) is a representative antitumor medication based on Chinese herbal injection, but its antitumor mechanisms are still poorly understood. Materials and methods In this work, the subcutaneous xenograft model of human pancreatic cancer cell line Panc-1 was established in nude mice to investigate the anticancer effect of ADI in vivo. We then determined the components of ADI using ultra-performance liquid chromatography-tandem mass spectrometry (UPLC-MS) and explored the possible molecular mechanisms against pancreatic cancer using network pharmacology. Results In vivo experiments, the volume, weight, and degree of histological abnormalities of implanted tumors were significantly lower in the medium and high concentration ADI injection groups than in the control group. Network pharmacology analysis identified four active components of ADI and seven key targets, TNF, VEGFA, HSP90AA1, MAPK14, CASP3, P53 and JUN. Molecular docking also revealed high affinity between the active components and the target proteins, including Astragaloside IV to P53 and VEGFA, Ginsenoside Rb1 to CASP3 and Formononetin to JUN. Conclusion ADI could reduce the growth rate of tumor tissue and alleviate the structural abnormalities in tumor tissue. ADI is predicted to act on VEGFA, P53, CASP3, and JUN in ADI-mediated treatment of pancreatic cancer

Mettl3-m6A-Creb1 forms an intrinsic regulatory axis in maintaining iNKT cell pool and functional differentiation

Summary: N6-methyladenosine (m6A) methyltransferase Mettl3 is involved in conventional T cell immunity; however, its role in innate immune cells remains largely unknown. Here, we show that Mettl3 intrinsically regulates invariant natural killer T (iNKT) cell development and function in an m6A-dependent manner. Conditional ablation of Mettl3 in CD4+CD8+ double-positive (DP) thymocytes impairs iNKT cell proliferation, differentiation, and cytokine secretion, which synergistically causes defects in B16F10 melanoma resistance. Transcriptomic and epi-transcriptomic analyses reveal that Mettl3 deficiency disturbs the expression of iNKT cell-related genes with altered m6A modification. Strikingly, Mettl3 modulates the stability of the Creb1 transcript, which in turn controls the protein and phosphorylation levels of Creb1. Furthermore, conditional targeting of Creb1 in DP thymocytes results in similar phenotypes of iNKT cells lacking Mettl3. Importantly, ectopic expression of Creb1 largely rectifies such developmental defects in Mettl3-deficient iNKT cells. These findings reveal that the Mettl3-m6A-Creb1 axis plays critical roles in regulating iNKT cells at the post-transcriptional layer