8 research outputs found

    Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling

    Full text link
    Uncertainty decomposition refers to the task of decomposing the total uncertainty of a model into data (aleatoric) uncertainty, resulting from the inherent complexity or ambiguity of the data, and model (epistemic) uncertainty, resulting from the lack of knowledge in the model. Performing uncertainty decomposition for large language models (LLMs) is an important step toward improving the reliability, trustworthiness, and interpretability of LLMs, but this research task is very challenging and remains unresolved. The existing canonical method, Bayesian Neural Network (BNN), cannot be applied to LLMs, because BNN requires training and ensembling multiple variants of models, which is infeasible or prohibitively expensive for LLMs. In this paper, we introduce an uncertainty decomposition framework for LLMs, called input clarifications ensemble, which bypasses the need to train new models. Rather than ensembling models with different parameters, our approach generates a set of clarifications for the input, feeds them into the fixed LLMs, and ensembles the corresponding predictions. We show that our framework shares a symmetric decomposition structure with BNN. Empirical evaluations demonstrate that the proposed framework provides accurate and reliable uncertainty quantification on various tasks. Code will be made publicly available at https://github.com/UCSB-NLP-Chang/llm_uncertainty .Comment: 15 pages, 3 figure

    Certified Robustness for Large Language Models with Self-Denoising

    Full text link
    Although large language models (LLMs) have achieved great success in vast real-world applications, their vulnerabilities towards noisy inputs have significantly limited their uses, especially in high-stake environments. In these contexts, it is crucial to ensure that every prediction made by large language models is stable, i.e., LLM predictions should be consistent given minor differences in the input. This largely falls into the study of certified robust LLMs, i.e., all predictions of LLM are certified to be correct in a local region around the input. Randomized smoothing has demonstrated great potential in certifying the robustness and prediction stability of LLMs. However, randomized smoothing requires adding noise to the input before model prediction, and its certification performance depends largely on the model's performance on corrupted data. As a result, its direct application to LLMs remains challenging and often results in a small certification radius. To address this issue, we take advantage of the multitasking nature of LLMs and propose to denoise the corrupted inputs with LLMs in a self-denoising manner. Different from previous works like denoised smoothing, which requires training a separate model to robustify LLM, our method enjoys far better efficiency and flexibility. Our experiment results show that our method outperforms the existing certification methods under both certified robustness and empirical robustness. The codes are available at https://github.com/UCSB-NLP-Chang/SelfDenoise

    Improving the efficiency of inverted organic solar cells by introducing ferrocenedicarboxylic acid between an ITO/ZnO interlayer

    No full text
    In this study, ferrocenedicarboxylic acid (FDA) has been introduced between an ITO electrode and ZnO interlayer to improve the performance of inverted polymer solar cells. The highest power conversion efficiency (PCE) of 9.06% is achieved among the measurements. Besides, the average PCE of FDA/ZnO based devices is observed with 11.9% enhancement (8.73% vs. 7.80%) compared to ZnO-only devices. Electrical characterization, surface morphology, wetting properties, as well as exciton generation rate and dissociation probability were investigated to understand the impact of FDA insertion on the interfacial properties. It was found that exciton dissociation efficiency and charge collection efficiency were significantly improved after inserting FDA, while the surface morphology, average roughness and water contact angle of the ZnO film were not changed. It was thought that FDA connected to the ITO electrode and ZnO film because of its carboxyl groups, which lead to a compact interfacial contact and reduced charge recombination. In addition, the devices based on the FDA/ZnO interlayers displayed improved stability in the argon-filled glove box without any encapsulation for about 1000 h compared to the ZnO-only devices. This study provides a new idea to introduce materials with functional groups between ITO/metal oxides interfaces to achieve more efficient charge collection and device performance
    corecore