The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediations

The recent advancements in Large Language Models (LLMs) have garnered widespread acclaim for their remarkable emerging capabilities. However, the issue of hallucination has parallelly emerged as a by-product, posing significant concerns. While some recent endeavors have been made to identify and mitigate different types of hallucination, there has been a limited emphasis on the nuanced categorization of hallucination and associated mitigation methods. To address this gap, we offer a fine-grained discourse on profiling hallucination based on its degree, orientation, and category, along with offering strategies for alleviation. As such, we define two overarching orientations of hallucination: (i) factual mirage (FM) and (ii) silver lining (SL). To provide a more comprehensive understanding, both orientations are further sub-categorized into intrinsic and extrinsic, with three degrees of severity - (i) mild, (ii) moderate, and (iii) alarming. We also meticulously categorize hallucination into six types: (i) acronym ambiguity, (ii) numeric nuisance, (iii) generated golem, (iv) virtual voice, (v) geographic erratum, and (vi) time wrap. Furthermore, we curate HallucInation eLiciTation (HILT), a publicly available dataset comprising of 75,000 samples generated using 15 contemporary LLMs along with human annotations for the aforementioned categories. Finally, to establish a method for quantifying and to offer a comparative spectrum that allows us to evaluate and rank LLMs based on their vulnerability to producing hallucinations, we propose Hallucination Vulnerability Index (HVI). We firmly believe that HVI holds significant value as a tool for the wider NLP community, with the potential to serve as a rubric in AI-related policy-making. In conclusion, we propose two solution strategies for mitigating hallucinations

LIPI at the NTCIR-17 FinArg-1 Task: Using Pre-trained Language Models for Comprehending Financial Arguments

Comprehending arguments from financial texts helps investors in making data driven decisions. The FinArg tasks of NTCIR-17 deal with mining arguments related to finance from Research Reports, Earnings Conference Calls, and Social Media. In this paper, we describe our team's approach to solve the three such problems - Argument Unit Classification, Argument Relation Detection & Classification, and Identifying Attack and Support Argumentative Relations. We obtained best performance using pre-trained language models (like BERT-SEC and FinBERT) and cross-encoder architecture

A novel multi-model estimation of phosphorus in coal and its ash using FTIR spectroscopy

Abstract The level of phosphorus must be carefully monitored for proper and effective utilization of coal and coal ash. The phosphorus content needs to be assessed to optimize combustion efficiency and maintenance costs of power plants, ensure quality, and minimize the environmental impact of coal and coal ash. The detection of low levels of phosphorus in coal and coal ash is a significant challenge due to its complex chemical composition and low concentration levels. Effective monitoring requires accurate and sensitive equipment for the detection of phosphorus in coal and coal ash. X-ray fluorescence (XRF) is a commonly used analytical technique for the determination of phosphorus content in coal and coal ash samples but proves challenging due to their comparatively weak fluorescence intensity. Fourier Transform Infrared spectroscopy (FTIR) emerges as a promising alternative that is simple, rapid, and cost-effective. However, research in this area has been limited. Until now, only a limited number of research studies have outlined the estimation of major elements in coal, predominantly relying on FTIR spectroscopy. In this article, we explore the potential of FTIR spectroscopy combined with machine learning models (piecewise linear regression—PLR, partial least square regression—PLSR, random forest—RF, and support vector regression—SVR) for quantifying the phosphorus content in coal and coal ash. For model development, the methodology employs the mid-infrared absorption peak intensity levels of phosphorus-specific functional groups and anionic groups of phosphate minerals at various working concentration ranges of coal and coal ash. This paper proposes a multi-model estimation (using PLR, PLSR, and RF) approach based on FTIR spectral data to detect and rapidly estimate low levels of phosphorus in coal and its ash (R $$^2$$ 2 of 0.836, RMSE of 0.735 ppm, RMSE (%) of 34.801, MBE of − 0.077 ppm, MBE (%) of 5.499, and MAE of 0.528 ppm in coal samples and R $$^2$$ 2 of 0.803, RMSE of 0.676 ppm, RMSE (%) of 38.050, MBE of − 0.118 ppm, MBE (%) of 4.501, and MAE of 0.474 ppm in coal ash samples). Our findings suggest that FTIR combined with the multi-model approach combining PLR, PLSR, and RF regression models is a reliable tool for rapid and near-real-time measurement of phosphorus in coal and coal ash and can be suitably modified to model phosphorus content in other natural samples such as soil, shale, etc

Iron pyrite, a potential photovoltaic material, increases plant biomass upon seed pretreatment

Iron pyrite (FeS2) is a promising material with plethora of applications ranging from sulfuric acid production to photo-voltaic devices. Interestingly, the proponents of the theory of hydrothermal origin of life on earth argues that FeS2 may have evolved 4.0 billion years ago, and used as an energy source by the earliest evolving life forms on earth. In the present time, bacteria like Thiobacillus ferrooxidans, which survives in the oxygen deficient environments, derives energy solely from FeS2 to maintain its critical biomass. The key question, we addressed in this paper is 'whether higher plants have the ability to derive energy solely from FeS2, just like the way Thiobacillus species does.' To answer this question, we developed a novel, inexpensive, low temperature scheme (< 100 °C) for FeS2 synthesis. We characterize FeS2 using X-ray diffraction (XRD), scanning electron microscopy (SEM) and high-resolution transmission electron microscopy (HRTEM) techniques. Further, we pretreated the chick-pea (Legume) seeds for 12 hours in sterile, double distilled aqueous medium of dispersed FeS2 (80 μg/ml). Following this, we allowed these seeds to grow in sterile, double distilled water for 7 days. At this stage, we observed that FeS2 pre-treated seeds result in significantly healthier plants, with increased dryweight and enhanced sulfur content as compared to the control plants. In summary, a brief FeS2 pre-treatment of the seeds resulted in increased plant biomass. This study has drawn an evolutionary consilience between two diverse life forms in terms of their ability to use a common pre-biotic energy molecule for biomass production. Our results suggest that FeS2 apart from its opto-electronic potential, could also be used as a pro-fertilizer for sustainable agriculture practices