ATP: Adaptive Tensor Parallelism for Foundation Models

Foundation models have impressive performance and generalization capabilities across a wide range of applications. The increasing size of the models introduces great challenges for the training. Tensor parallelism is a critical technique that is currently used in almost all foundation model training and has a significant impact on overall training performance. However, current tensor parallelism in machine learning frameworks misses optimization opportunities in fitting various interconnection topologies. In this work, we present ATP, an adaptive tensor parallelism framework for foundation models, which can automatically select the optimal parallel strategy on different interconnections. We propose column- and row-first tensor parallelism based on 2D device meshes and construct a search space. Combined with the hierarchical communication matrix, ATP can identify the optimal strategy in the search space. We also propose chunk-based overlapping to reduce communication overhead. Our evaluations show ATP consistently outperforms the state-of-the-art approaches for various model sizes and interconnects, achieving end-to-end training performance improvements of up to 37-64% on specific interconnects. Based on our theoretical model, the communication overhead of ATP decreases with scaling, indicating a qualitative leap forward

Optimizing medium for producing ethanol from industrial crop Jerusalem artichoke by one- step fermentation and recombinant Saccharomyces cerevisiae

In order to obtain a high ethanol yield from the Jerusalem artichoke raw extract and reduce the fermentation cost, we have engineered a new recombinant Saccharomyces cerevisiae strain that could produce ex-inulinase. The response surface methodology based on Plackett-Burman and Box-Behnken design was used to optimize the medium for the ethanol production from the Jerusalem artichoke raw extracts by the recombinant strain. In the first optimization step, Plackett-Burman design was employed to select significant factors, including concentrations of yeast extract, inoculum, and MgSO(4)7H(2)O. In the second step, the steepest ascent experiment was carried out to determine the center point with the three significant factors; the selected combinations were further optimized using the Box-Behnken design. The maximum ethanol production rate was predicted at 91.1g/l, which was based on a medium consisting of yeast extract 9.24g/l, inoculum 39.8ml/l, and MgSO(4)7H(2)O 0.45g/l. In the validating experiment, the ethanol fermentation rate reached 102.1g/l, closely matching the predicted rate.In order to obtain a high ethanol yield from the Jerusalem artichoke raw extract and reduce the fermentation cost, we have engineered a new recombinant Saccharomyces cerevisiae strain that could produce ex-inulinase. The response surface methodology based on Plackett-Burman and Box-Behnken design was used to optimize the medium for the ethanol production from the Jerusalem artichoke raw extracts by the recombinant strain. In the first optimization step, Plackett-Burman design was employed to select significant factors, including concentrations of yeast extract, inoculum, and MgSO(4)7H(2)O. In the second step, the steepest ascent experiment was carried out to determine the center point with the three significant factors; the selected combinations were further optimized using the Box-Behnken design. The maximum ethanol production rate was predicted at 91.1g/l, which was based on a medium consisting of yeast extract 9.24g/l, inoculum 39.8ml/l, and MgSO(4)7H(2)O 0.45g/l. In the validating experiment, the ethanol fermentation rate reached 102.1g/l, closely matching the predicted rate

Microarray analysis of differential gene expression in sensitive and resistant pig to Escherichia coli F18. Anim Genet 43: 525–534

Summary In this study, Agilent two-colour microarray-based gene expression profiling was used to detect differential gene expression in duodenal tissues collected from eight full-sib pairs of Sutai pigs differing in adhesion phenotype (sensitivity and resistance to Escherichia coli F18). Using a two-fold change minimum threshold, we found 18 genes that were differentially expressed (10 up-regulated and eight down-regulated) between the sensitive and resistant animal groups. Our gene ontology analysis revealed that these differentially expressed genes are involved in a variety of biological processes, including immune responses, extracellular modification (e.g. glycosylation), cell adhesion and signal transduction, all of which are related to the anabolic metabolism of glycolipids, as well as to inflammation-and immunerelated pathways. Based on the genes identified in the screen and the pathway analysis results, real-time PCR was used to test the involvement of ST3GAL1 and A genes (of glycolipid-related pathways), SLA-1 and SLA-3 genes (of inflammation-and immune-related pathways), as well as the differential genes FUT1, TAP1 and SLA-DQA. Subsequently, realtime PCR was performed to validate seven differentially expressed genes screened out by the microarray approach, and sufficient consistency was observed between the two methods. The results support the conclusion that these genes are related to the E. coli F18 receptor and susceptibility to E. coli F18

Direct production of bioethanol from Jerusalem artichoke inulin by gene-engineering Saccharomyces cerevisiae 6525 with exoinulinase gene

Jerusalem artichoke (Helianthus tuberosus L.), an important crop, containing over 50% inulin in its tubers on a dry weight basis is an agricultural and industrial crop with a great potential for production of ethanol and industrial products. Inulin is a good substrate for bioethanol production. Saccharomyces cerevisiae 6525 can produce high concentrations of ethanol, but it cannot synthesize inulinase. In this study, a new integration vector carrying inuA1 gene encoding exoinulinase was constructed and transformed into 18SrDNA site of industrial strain S. cerevisiae 6525. The obtained transformant, BR8, produced 1.1UmL(-1) inulinase activity within 72h and the dry cell weight reached 12.3gL(-1) within 48h. In a small-scale fermentation, BR8 produced 9.5% (v/v) ethanol, with a productivity rate of 0.385g ethanol per gram inulin, while wild-type S. cerevisiae 6525 produced only 3.3% (v/v) ethanol in the same conditions. In a 5-L fermentation, BR8 produced 14.0% (v/v) ethanol in fermentation medium containing inulin and 1% (w/v) (NH4)(2)SO4. The engineered S. cerevisiae 6525 carrying inuA1 converted pure nonhydrolyzed inulin directly into high concentrations of ethanol.Jerusalem artichoke (Helianthus tuberosus L.), an important crop, containing over 50% inulin in its tubers on a dry weight basis is an agricultural and industrial crop with a great potential for production of ethanol and industrial products. Inulin is a good substrate for bioethanol production. Saccharomyces cerevisiae 6525 can produce high concentrations of ethanol, but it cannot synthesize inulinase. In this study, a new integration vector carrying inuA1 gene encoding exoinulinase was constructed and transformed into 18SrDNA site of industrial strain S. cerevisiae 6525. The obtained transformant, BR8, produced 1.1UmL(-1) inulinase activity within 72h and the dry cell weight reached 12.3gL(-1) within 48h. In a small-scale fermentation, BR8 produced 9.5% (v/v) ethanol, with a productivity rate of 0.385g ethanol per gram inulin, while wild-type S. cerevisiae 6525 produced only 3.3% (v/v) ethanol in the same conditions. In a 5-L fermentation, BR8 produced 14.0% (v/v) ethanol in fermentation medium containing inulin and 1% (w/v) (NH4)(2)SO4. The engineered S. cerevisiae 6525 carrying inuA1 converted pure nonhydrolyzed inulin directly into high concentrations of ethanol