Empowering Large Language Models with Efficient and Automated Systems

Large Language Models (LLMs) have shown remarkable capabilities in a variety of tasks, including chatting, programming, and searching. However, the high costs of LLMs are preventing these models from being deployed for the vast majority of applications. In this dissertation, we focus on building efficient and automated systems to reduce costs and democratize access to large language models. We first introduce systems to optimize computational efficiency and reduce the engineering overhead for distributed LLM training. We develop TeraPipe, which proposes a new dimension to perform pipeline parallel training for LLMs, and also Alpa, the world’s first compiler capable of automatically distributing arbitrary neural networks with all existing parallelization methods. While training is typically a one-time cost, deploying and serving an LLM requires running LLM inference continuously, which is the top blocker for the real-world deployment of LLMs. We improve the serving scalability with AlpaServe through model parallelism, and increase the memory utilization and the LLM inference throughput with a new attention algorithm, PagedAttention, and an end-to-end serving system, vLLM.Overall, these systems provide comprehensive solutions that significantly improve both training and inference efficiency for large language models. Together, these systems lower the high costs associated with large language models, democratizing their deployment across various real-world applications

Unsupervised machine learning for identifying phase transition using two-times clustering

In recent years, developing unsupervised machine learning for identifying phase transition is a research direction. In this paper, we introduce a two-times clustering method that can help select perfect configurations from a set of degenerate samples and assign the configuration with labels in a manner of unsupervised machine learning. These perfect configurations can then be used to train a neural network to classify phases. The derivatives of the predicted classification in the phase diagram, show peaks at the phase transition points. The effectiveness of our method is tested for the Ising, Potts, and Blume-Capel models. By using the ordered configuration from two-times clustering, our method can provide a useful way to obtain phase diagrams.Comment: 8 pages, 7 figure

Reducing of industrial atmospheric emissions using electrocyclone

The article is focused on capturing process-related dust at industrial enterprises (in chemical, metallurgical and energy industries). An electrocyclone can be recommended for the purification of gases emitted into the atmosphere from particulates, such as sodium percarbonate (efficiency 97.5%–99.9%), iron-vanadium concentrate (98.0% - 99.9%), fly ash (99.0%–99.9%). However, the fumes from copper-smelting furnaces cannot be purified with high efficiency (less than 50–60%) because of their properties. Using electrocyclone will reduce the amount of aerosol emissions, and in some cases, let the emission reach the values set by standards