2 research outputs found
LM-Cocktail: Resilient Tuning of Language Models via Model Merging
The pre-trained language models are continually fine-tuned to better support
downstream applications. However, this operation may result in significant
performance degeneration on general tasks beyond the targeted domain. To
overcome this problem, we propose LM-Cocktail which enables the fine-tuned
model to stay resilient in general perspectives. Our method is conducted in the
form of model merging, where the fine-tuned language model is merged with the
pre-trained base model or the peer models from other domains through weighted
average. Despite simplicity, LM-Cocktail is surprisingly effective: the
resulted model is able to achieve a strong empirical performance in the whole
scope of general tasks while preserving a superior capacity in its targeted
domain. We conduct comprehensive experiments with LLama and BGE model on
popular benchmarks, including FLAN, MMLU, MTEB, whose results validate the
efficacy of our proposed method. The code and checkpoints are available at
https://github.com/FlagOpen/FlagEmbedding/tree/master/LM_Cocktail.Comment: Work is in progres
Quantifying and Attributing the Hallucination of Large Language Models via Association Analysis
Although demonstrating superb performance on various NLP tasks, large
language models (LLMs) still suffer from the hallucination problem, which
threatens the reliability of LLMs. To measure the level of hallucination of
LLMs, previous works first categorize the hallucination according to the
phenomenon similarity, then quantify the proportion that model outputs contain
hallucinatory contents. However, such hallucination rates could easily be
distorted by confounders. Moreover, such hallucination rates could not reflect
the reasons for the hallucination, as similar hallucinatory phenomena may
originate from different sources. To address these issues, we propose to
combine the hallucination level quantification and hallucination reason
investigation through an association analysis, which builds the relationship
between the hallucination rate of LLMs with a set of risk factors. In this way,
we are able to observe the hallucination level under each value of each risk
factor, examining the contribution and statistical significance of each risk
factor, meanwhile excluding the confounding effect of other factors.
Additionally, by recognizing the risk factors according to a taxonomy of model
capability, we reveal a set of potential deficiencies in commonsense
memorization, relational reasoning, and instruction following, which may
further provide guidance for the pretraining and supervised fine-tuning process
of LLMs to mitigate the hallucination