Dissecting the Runtime Performance of the Training, Fine-tuning, and Inference of Large Language Models

Large Language Models (LLMs) have seen great advance in both academia and industry, and their popularity results in numerous open-source frameworks and techniques in accelerating LLM pre-training, fine-tuning, and inference. Training and deploying LLMs are expensive as it requires considerable computing resources and memory, hence many efficient approaches have been developed for improving system pipelines as well as operators. However, the runtime performance can vary significantly across hardware and software stacks, which makes it difficult to choose the best configuration. In this work, we aim to benchmark the performance from both macro and micro perspectives. First, we benchmark the end-to-end performance of pre-training, fine-tuning, and serving LLMs in different sizes , i.e., 7, 13, and 70 billion parameters (7B, 13B, and 70B) on three 8-GPU platforms with and without individual optimization techniques, including ZeRO, quantization, recomputation, FlashAttention. Then, we dive deeper to provide a detailed runtime analysis of the sub-modules, including computing and communication operators in LLMs. For end users, our benchmark and findings help better understand different optimization techniques, training and inference frameworks, together with hardware platforms in choosing configurations for deploying LLMs. For researchers, our in-depth module-wise analyses discover potential opportunities for future work to further optimize the runtime performance of LLMs

CodeApex: A Bilingual Programming Evaluation Benchmark for Large Language Models

With the emergence of Large Language Models (LLMs), there has been a significant improvement in the programming capabilities of models, attracting growing attention from researchers. We propose CodeApex, a bilingual benchmark dataset focusing on the programming comprehension and code generation abilities of LLMs. CodeApex comprises three types of multiple-choice questions: conceptual understanding, commonsense reasoning, and multi-hop reasoning, designed to evaluate LLMs on programming comprehension tasks. Additionally, CodeApex utilizes algorithmic questions and corresponding test cases to assess the code quality generated by LLMs. We evaluate 14 state-of-the-art LLMs, including both general-purpose and specialized models. GPT exhibits the best programming capabilities, achieving approximate accuracies of 50% and 56% on the two tasks, respectively. There is still significant room for improvement in programming tasks. We hope that CodeApex can serve as a reference for evaluating the coding capabilities of LLMs, further promoting their development and growth. Datasets are released at https://github.com/APEXLAB/CodeApex.git. CodeApex submission website is https://apex.sjtu.edu.cn/codeapex/.Comment: 21 page

Contribution of Hepatitis B Virus Infection to the Aggressiveness of Primary Liver Cancer: A Clinical Epidemiological Study in Eastern China

Background and aims: The contribution of hepatitis B virus (HBV) infection to the aggressiveness of primary liver cancer (PLC) remains controversial. We aimed to characterize this in eastern China.Methods: We enrolled 8,515 PLC patients whose specimens were reserved at the BioBank of the hepatobiliary hospital (Shanghai, China) during 2007–2016. Of those, 3,124 who received primary radical resection were involved in survival analysis. A nomogram was constructed to predict the survivals using preoperative parameters.Results: Hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma (ICC), and combined hepatocellular cholangiocarcinoma (CHC) accounted for 94.6, 3.7, and 1.7%, respectively. The rates of HBV infection were 87.5, 49.2, and 80.6%, respectively. HBV infection was significantly associated with 10-year earlier onset, more cirrhosis, higher α-fetoprotein, higher carbohydrate antigen 19-9 (CA19-9), more microvascular invasion (MVI), lower neutrophil-to-lymphocyte ratio (NLR), and lower platelet-to-lymphocyte ratio (PLR) in HCC. HBV infection was also associated with 7-year earlier onset, more cirrhosis, higher α-fetoprotein, more MVI, and lower PLR in ICC. In the multivariate Cox analysis, high circulating HBV DNA, α-fetoprotein, CA19-9, NLR, tumor size, number, encapsulation, Barcelona Clinic Liver Cancer (BCLC) stage, and MVI predicted an unfavorable prognosis in HCC; only CA19-9 and BCLC stage, rather than HBV-related parameters, had prognostic values in ICC. A nomogram constructed with preoperative HBV-related parameters including HBV load, ultrasonic cirrhosis, and α-fetoprotein perform better than the current staging systems in predicting postoperative survival in HCC.Conclusion: HBV promotes the aggressiveness of HCC in Chinese population. The contributions of HBV to ICC and other etiological factors to HCC might be indirect via arousing non-resolving inflammation