ASPEN: High-Throughput LoRA Fine-Tuning of Large Language Models with a Single GPU

Transformer-based large language models (LLMs) have demonstrated outstanding performance across diverse domains, particularly when fine-turned for specific domains. Recent studies suggest that the resources required for fine-tuning LLMs can be economized through parameter-efficient methods such as Low-Rank Adaptation (LoRA). While LoRA effectively reduces computational burdens and resource demands, it currently supports only a single-job fine-tuning setup. In this paper, we present ASPEN, a high-throughput framework for fine-tuning LLMs. ASPEN efficiently trains multiple jobs on a single GPU using the LoRA method, leveraging shared pre-trained model and adaptive scheduling. ASPEN is compatible with transformer-based language models like LLaMA and ChatGLM, etc. Experiments show that ASPEN saves 53% of GPU memory when training multiple LLaMA-7B models on NVIDIA A100 80GB GPU and boosts training throughput by about 17% compared to existing methods when training with various pre-trained models on different GPUs. The adaptive scheduling algorithm reduces turnaround time by 24%, end-to-end training latency by 12%, prioritizing jobs and preventing out-of-memory issues.Comment: 14 pages, 14 figure

BRAF Exon 15 T1799A Mutation Is Common in Melanocytic Nevi, but Less Prevalent in Cutaneous Malignant Melanoma, in Chinese Han

Frequent somatic mutations of BRAF (v-raf murine sarcoma viral oncogene homolog B) exon T1799A, which are implicated in the initial events of promutagenic cellular proliferation, are detected in both malignant melanomas (MM) and melanocytic nevi (MN). Most of the data regarding BRAF exon T1799A mutation have been from Caucasian cohorts, and a comprehensive screening of a homogeneous population is lacking. A total of 379 cases of MN and 195 cases of MM were collected from Chinese Han living in three geographical regions in China, i.e., northeast, southwest, and northwest China. BRAF exon T1799A mutation was detected by PCR and sequencing from microdissected tumors. In all, 59.8% cases of MN harbored BRAF exon T1799A mutation. Samples from regions with high UV exposure had higher detection rates than regions with lower UV exposure (73.5, 67.0, and 38.9%, respectively; χ2=31.674, P=1.59E–7). There were no differences in mutation rates between congenital and acquired MN; however, acquired MN with advanced age of onset had a higher mutation rate than those with younger age of onset (χ2=13.23, P=0.02). In all, 15.0% cases of MM harbored the BRAF mutation. The mutation rate in MM was not affected by region, histological type, gender, pattern of UV exposure, and age. The study suggests that the mutation is not necessarily associated with malignant transformation