Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimension

We study how to characterize and predict the truthfulness of texts generated from large language models (LLMs), which serves as a crucial step in building trust between humans and LLMs. Although several approaches based on entropy or verbalized uncertainty have been proposed to calibrate model predictions, these methods are often intractable, sensitive to hyperparameters, and less reliable when applied in generative tasks with LLMs. In this paper, we suggest investigating internal activations and quantifying LLM's truthfulness using the local intrinsic dimension (LID) of model activations. Through experiments on four question answering (QA) datasets, we demonstrate the effectiveness ohttps://info.arxiv.org/help/prep#abstractsf our proposed method. Additionally, we study intrinsic dimensions in LLMs and their relations with model layers, autoregressive language modeling, and the training of LLMs, revealing that intrinsic dimensions can be a powerful approach to understanding LLMs.Comment: preprint, 9 pages, 5 figure

Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curation

Instruction tuning has emerged to enhance the capabilities of large language models (LLMs) to comprehend instructions and generate appropriate responses. Existing methods either manually annotate or employ LLM (e.g., GPT-series) to generate data for instruction tuning. However, they often overlook associating instructions with existing annotated datasets. In this paper, we propose Dynosaur, a dynamic growth paradigm for the automatic curation of instruction-tuning data. Based on the metadata of existing datasets, we use LLMs to automatically construct instruction-tuning data by identifying relevant data fields and generating appropriate instructions. By leveraging the existing annotated datasets, Dynosaur offers several advantages: 1) it reduces the API cost for generating instructions (e.g., it costs less than $12 USD by calling GPT-3.5-turbo for generating 800K instruction tuning samples; 2) it provides high-quality data for instruction tuning (e.g., it performs better than Alpaca and Flan on Super-NI and Longform with comparable data sizes); and 3) it supports the continuous improvement of models by generating instruction-tuning data when a new annotated dataset becomes available. We further investigate a continual learning scheme for learning with the ever-growing instruction-tuning dataset, and demonstrate that replaying tasks with diverse instruction embeddings not only helps mitigate forgetting issues but generalizes to unseen tasks better. Code and data are available at https://github.com/WadeYin9712/Dynosaur.Comment: EMNLP 2023. Code and data are available at https://github.com/WadeYin9712/Dynosau

Sustainabilitas Arsitektur Masjid: Evaluasi Konsep “Simple Architecture” sebagai Implementasi Desain Arsitektur Berkelanjutan suatu Kawasan

Makalah ini membahas aspek-aspek “kesederhanaan” (simplicity) sebagai konsep desain bangunan masjid secara berkelanjutan (sustainable) sesuai konteks dengan mengambil studi kasus masjid kawasan Al-Irsyad Satya Kota Baru Parahyangan, Bandung. Masjid sebagai subyek arsitektur dan pusat ibadah menjadi ruang publik yang didesain dari elemen-elemen yang secara ideal mengandung nilai-nilai Islam dan bertujuan mendukung fungsinya. Desain masjid berkonsep simple atau “sederhana” digunakan sebagai alternatif kontemporer untuk mengoptimalisasi fungsi tersebut, meliputi struktur bangunan hingga biaya pemeliharaan (maintenance) sesuai prinsip keberlanjutan. Keterkaitan erat bangunan masjid dengan aktivitas masyarakat berpotensi melibatkan partisipasi masyarakat dan pengelola dalam menerapkan program sustainabilitas sesuai konteks lingkungannya. Metode yang digunakan dalam penelitian ini berbasis pendekatan Grounded Theory secara kualitatif melalui pengumpulan data dari kegiatan observasi, interview dan analisis program keberlanjutan kawasan. Penelitian menemukan keterkaitan konsep “sederhana” yang mendukung sustainabilitas desain sekaligus menggarisbawahi evaluasi konsep desain “sederhana” yang hadir serta faktor pemeliharaan/pengembangan masjid dan kawasan

Smoking, Habitual Tea Drinking and Metabolic Syndrome in Elderly Men Living in Rural Community: The Tianliao Old People (TOP) Study 02

The literature shows an inconsistent relationship between lifestyle behaviors and metabolic syndrome (MetS), especially in the elderly. We designed this study to investigate the interrelationships among cigarette smoking, tea drinking and MetS, and to verify the factors associated with MetS in elderly males dwelling in rural community. In July 2010, with a whole community sampling method, 414 male subjects aged over 65 dwelling in Tianliao township were randomly sampled. The response rate was 60.8%. Each subject completed the structured questionnaires including sociodemographic characteristics, habitual behaviors (including cigarette smoking and tea drinking habits) and medical history. After an overnight fast, the laboratory and anthropometric data were obtained. MetS was confirmed according to the criteria defined by the modified NCEP ATP III for the male Chinese population. Subjects were split into either non-MetS or MetS groups for further analysis. Of the 361 subjects with complete data, 132 (36.6%) elderly men were classified as having MetS. Using binary logistic regression, body mass index, serum uric acid, high sensitivity C-reactive protein, HOMA index, current smokers (OR = 2.72, 95%CI: 1.03 ∼ 7.19), total smoking amount > = 30 (OR = 2.78, 95%CI: 1.31 ∼ 5.90) and more than 20 cigarettes daily (OR = 2.54, 95%CI: 1.24 ∼ 5.18) were positively associated with MetS. Current un- or partial fermented tea drinker (OR = 0.42, 95%CI: 0.22 ∼ 0.84), tea drinking habit for 1–9 years (OR = 0.36, 95%CI: 0.15 ∼ 0.90) and more than 240cc daily (OR = 0.35, 95%CI: 0.17 ∼ 0.72) were negatively associated with MetS. In conclusion, this study suggests that smoking habit was positively associated with MetS, but tea drinking habit was negatively associated with MetS in elderly men dwelling in rural community

Red Teaming Language Model Detectors with Language Models

The prevalence and strong capability of large language models (LLMs) present significant safety and ethical risks if exploited by malicious users. To prevent the potentially deceptive usage of LLMs, recent works have proposed algorithms to detect LLM-generated text and protect LLMs. In this paper, we investigate the robustness and reliability of these LLM detectors under adversarial attacks. We study two types of attack strategies: 1) replacing certain words in an LLM's output with their synonyms given the context; 2) automatically searching for an instructional prompt to alter the writing style of the generation. In both strategies, we leverage an auxiliary LLM to generate the word replacements or the instructional prompt. Different from previous works, we consider a challenging setting where the auxiliary LLM can also be protected by a detector. Experiments reveal that our attacks effectively compromise the performance of all detectors in the study with plausible generations, underscoring the urgent need to improve the robustness of LLM-generated text detection systems.Comment: Preprint. Accepted by TAC

Regulatory Roles of miRNA in the Human Neural Stem Cell Transformation to Glioma Stem Cells

To investigate the expressional alternation of microRNAs (miRNA) during the malignant transformation and development of human glioma, we measured miRNA expression profile as well as mRNA expression profile in normal human neural stem cells (hNSCs) and human glioma stem cells (hGSCs). We found 116 miRNA up‐regulated and 62 miRNA down‐regulated in GSCs. On the other hand, we identified 1,372 mRNA down‐regulated, and 1,501 mRNA up‐regulated in GSCs compared to those in NSCs. We then analyzed the pathways and the predicted target genes of the miRNAs which differ significantly in expression between GSCs and NSCs using the statistical enrichment methods. These target mRNAs are involved in many cancer‐related signaling pathways, such as cell cycle, axon guidance, glioma development, adhesion junction, MAPK and Wnt signaling. Furthermore, we obtained the differently expressed miRNA‐target relationships according to the θ value which is used to calculate the regulation extent of miRNA‐target and using the databases of miRanda, Targetscans and Pictar. Among the top 10 miRNA‐target relationships, hsa‐miR‐198 and its potential targeted gene DCX and NNAT were selected for validation, and NNAT was found to be the direct target of miR‐198. Finally, the functional roles of miR‐155–5p and miR‐124–3p whose expressions altered significantly between GSCs and NSCs were addressed. Our results provide new clues for the potential mechanisms involved in the origin and development of glioma. Clinically, the altered miRNAs may serve as potential targets and diagnostic tools for novel therapeutic strategies of glioblastoma. J. Cell. Biochem. 115: 1368–1380, 2014. © 2014 Wiley Periodicals, Inc.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/107550/1/jcb24786.pd

An Empirical Evaluation Of User Satisfaction With A School Nursing Information System

The adoption of a school nursing information system is considered one of the most efficient ways in which to document health records as well as monitor health conditions electronically. However, despite the importance of computerized health records in school nursing practice, few studies have examined user satisfaction of a school nursing information system. The aim of this study is to investigate the critical factors effecting school nurses’ satisfaction with a school nursing information system Utilizing a survey approach, questionnaires are distributed to nurses working in a primary or high school which introduces a new school nursing information system. The findings show several factors, including perceived usefulness, perceived of ease of use, training and workload are significant with user satisfaction. These results suggest that school nursing information system designers should comprehensively understand users’ demands and perceptions about the system, which will further facilitate user satisfaction, decrease their workload, and ultimately enhance job performance

Childhood abuse and association with adult depressive symptoms among people with cardiovascular disease

BackgroundTo study the association between the total/different types of childhood abuse and adult depressive symptoms in people with cardiovascular disease (CVD).MethodsThe subjects were people with CVD who continuously participated in the China Health and Retirement Longitudinal Study (CHARLS) life history survey and the 2018 wave of the CHARLS national baseline Survey. Multi-level logistic regression models were used to analyze the relationship between emotional neglect, physical neglect, physical abuse and adult depressive symptoms.ResultsA total of 4,823 respondents were included in this study. The incidence of childhood abuse (existed emotional neglect, physical neglect or physical abuse) was 43.58% among people over 45 years old with CVD, which was higher than that of the general population (36.62%, p < 0.05). Adjusted model showed that overall childhood abuse was associated with adult depressive symptoms (OR = 1.230, 95%CI:1.094–1.383). Among different types of childhood abuse, only physical abuse was associated with depressive symptoms in adulthood (OR = 1.345, 95%CI:1.184–1.528).ConclusionCompared with that of the general population, the incidence of childhood abuse in CVD population is higher. Physical abuse in childhood increased the risk of depressive symptoms in adulthood. It suggested that the occurrence of depressive symptoms was the result of related factors in the whole life course. In order to prevent the depressive symptoms, childhood abuse also needs to be considered. It is very important to identify and prevent the continuation of childhood abuse in time