Evaluation and Analysis of Hallucination in Large Vision-Language Models

Large Vision-Language Models (LVLMs) have recently achieved remarkable success. However, LVLMs are still plagued by the hallucination problem, which limits the practicality in many scenarios. Hallucination refers to the information of LVLMs' responses that does not exist in the visual input, which poses potential risks of substantial consequences. There has been limited work studying hallucination evaluation in LVLMs. In this paper, we propose Hallucination Evaluation based on Large Language Models (HaELM), an LLM-based hallucination evaluation framework. HaELM achieves an approximate 95% performance comparable to ChatGPT and has additional advantages including low cost, reproducibility, privacy preservation and local deployment. Leveraging the HaELM, we evaluate the hallucination in current LVLMs. Furthermore, we analyze the factors contributing to hallucination in LVLMs and offer helpful suggestions to mitigate the hallucination problem. Our training data and human annotation hallucination data will be made public soon.Comment: 11 pages, 5 figure

mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration

Multi-modal Large Language Models (MLLMs) have demonstrated impressive instruction abilities across various open-ended tasks. However, previous methods primarily focus on enhancing multi-modal capabilities. In this work, we introduce a versatile multi-modal large language model, mPLUG-Owl2, which effectively leverages modality collaboration to improve performance in both text and multi-modal tasks. mPLUG-Owl2 utilizes a modularized network design, with the language decoder acting as a universal interface for managing different modalities. Specifically, mPLUG-Owl2 incorporates shared functional modules to facilitate modality collaboration and introduces a modality-adaptive module that preserves modality-specific features. Extensive experiments reveal that mPLUG-Owl2 is capable of generalizing both text tasks and multi-modal tasks and achieving state-of-the-art performances with a single generic model. Notably, mPLUG-Owl2 is the first MLLM model that demonstrates the modality collaboration phenomenon in both pure-text and multi-modal scenarios, setting a pioneering path in the development of future multi-modal foundation models

Cloning and characterization of Lxr and Srebp1, and their potential roles in regulation of LC-PUFA biosynthesis in rabbitfish Siganus canaliculatus

Rabbitfish Siganus canaliculatus was the first marine teleost demonstrated to have the ability to biosynthesize C20-22 long-chain polyunsaturated fatty acids (LC-PUFA) from C18 PUFA precursors, which is generally absent or low in marine teleosts. Thus, understanding the molecular basis of LC-PUFA biosynthesis in rabbitfish will contribute to efforts aimed at optimizing LC-PUFA biosynthesis in teleosts, especially marine species. In the present study, the importance of the transcription factors liver X receptor (Lxr) and sterol regulatory element-binding protein 1 (Srebp1) in regulation of LC-PUFA biosynthesis in rabbitfish was investigated. First, full-length cDNAs of Lxr and Srebp1 were cloned and characterized. The Lxr mRNA displayed a ubiquitous tissue expression pattern while Srebp1 was highly expressed in eyes, brain and intestine. In rabbitfish primary hepatocytes treated with Lxr agonist T0901317, the expression of Lxr and Srebp1 was activated, accompanied by elevated mRNA levels of Δ4 and Δ6/Δ5 fatty acyl desaturases (Fad), key enzymes of LC-PUFA biosynthesis, as well as peroxisome proliferator-activated receptor γ (Pparγ). In addition, Srebp1 displayed higher expression levels in liver of rabbitfish fed a vegetable oil diet or reared at 10 ppt salinity, which were conditions reported to increase the liver expression of Δ4 and Δ6/Δ5 Fad and LC-PUFA biosynthetic ability, than fish fed a fish oil diet or reared at 32 ppt, respectively. These results suggested that Lxr and Srebp1 are involved in regulation of LC-PUFA biosynthesis probably by promoting the expression of two Fads in rabbitfish liver, which, to our knowledge, is the first report in marine teleosts

In Vivo Gene Knockdown in Rat Dorsal Root Ganglia Mediated by Self-Complementary Adeno-Associated Virus Serotype 5 Following Intrathecal Delivery

We report here in adult rat viral vector mediate-gene knockdown in the primary sensory neurons and the associated cellular and behavior consequences. Self-complementary adeno-associated virus serotype 5 (AAV5) was constructed to express green fluorescent protein (GFP) and a small interfering RNA (siRNA) targeting mammalian target of rapamycin (mTOR). The AAV vectors were injected via an intrathecal catheter. We observed profound GFP expression in lumbar DRG neurons beginning at 2-week post-injection. Of those neurons, over 85% were large to medium-diameter and co-labeled with NF200, a marker for myelinated fibers. Western blotting of mTOR revealed an 80% reduction in the lumbar DRGs (L4–L6) of rats treated with the active siRNA vectors compared to the control siRNA vector. Gene knockdown became apparent as early as 7-day post-injection and lasted for at least 5 weeks. Importantly, mTOR knockdown occurred in large (NF200) and small-diameter neurons (nociceptors). The viral administration induced an increase of Iba1 immunoreactivity in the DRGs, which was likely attributed to the expression of GFP but not siRNA. Rats with mTOR knockdown in DRG neurons showed normal general behavior and unaltered responses to noxious stimuli. In conclusion, intrathecal AAV5 is a highly efficient vehicle to deliver siRNA and generate gene knockdown in DRG neurons. This will be valuable for both basic research and clinic intervention of diseases involving primary sensory neurons