1 research outputs found
LMTuner: An user-friendly and highly-integrable Training Framework for fine-tuning Large Language Models
With the burgeoning development in the realm of large language models (LLMs),
the demand for efficient incremental training tailored to specific industries
and domains continues to increase. Currently, the predominantly employed
frameworks lack modular design, it often takes a lot of coding work to
kickstart the training of LLM. To address this, we present "LMTuner", a highly
usable, integrable, and scalable system for training LLMs expeditiously and
with minimal user-input. LMTuner comprises three main modules - the
Interaction, Training, and Inference Modules. We advocate that LMTuner's
usability and integrality alleviate the complexities in training large language
models. Remarkably, even a novice user could commence training large language
models within five minutes. Furthermore, it integrates DeepSpeed frameworks and
supports Efficient Fine-Tuning methodologies like Low Rank Adaptation (LoRA),
Quantized LoRA (QLoRA), etc., enabling the training of language models scaling
from 300M to a whopping 130B parameters using a single server. The LMTuner's
homepage (https://wengsyx.github.io/LMTuner/)and screencast video
(https://youtu.be/nsXmWOmN3rE) are now publicly available