Identifying and Mitigating Vulnerabilities in LLM-Integrated Applications

Large language models (LLMs) are increasingly deployed as the service backend for LLM-integrated applications such as code completion and AI-powered search. LLM-integrated applications serve as middleware to refine users' queries with domain-specific knowledge to better inform LLMs and enhance the responses. Despite numerous opportunities and benefits, LLM-integrated applications also introduce new attack surfaces. Understanding, minimizing, and eliminating these emerging attack surfaces is a new area of research. In this work, we consider a setup where the user and LLM interact via an LLM-integrated application in the middle. We focus on the communication rounds that begin with user's queries and end with LLM-integrated application returning responses to the queries, powered by LLMs at the service backend. For this query-response protocol, we identify potential vulnerabilities that can originate from the malicious application developer or from an outsider threat initiator that is able to control the database access, manipulate and poison data that are high-risk for the user. Successful exploits of the identified vulnerabilities result in the users receiving responses tailored to the intent of a threat initiator. We assess such threats against LLM-integrated applications empowered by OpenAI GPT-3.5 and GPT-4. Our empirical results show that the threats can effectively bypass the restrictions and moderation policies of OpenAI, resulting in users receiving responses that contain bias, toxic content, privacy risk, and disinformation. To mitigate those threats, we identify and define four key properties, namely integrity, source identification, attack detectability, and utility preservation, that need to be satisfied by a safe LLM-integrated application. Based on these properties, we develop a lightweight, threat-agnostic defense that mitigates both insider and outsider threats

MDTD: A Multi Domain Trojan Detector for Deep Neural Networks

Machine learning models that use deep neural networks (DNNs) are vulnerable to backdoor attacks. An adversary carrying out a backdoor attack embeds a predefined perturbation called a trigger into a small subset of input samples and trains the DNN such that the presence of the trigger in the input results in an adversary-desired output class. Such adversarial retraining however needs to ensure that outputs for inputs without the trigger remain unaffected and provide high classification accuracy on clean samples. In this paper, we propose MDTD, a Multi-Domain Trojan Detector for DNNs, which detects inputs containing a Trojan trigger at testing time. MDTD does not require knowledge of trigger-embedding strategy of the attacker and can be applied to a pre-trained DNN model with image, audio, or graph-based inputs. MDTD leverages an insight that input samples containing a Trojan trigger are located relatively farther away from a decision boundary than clean samples. MDTD estimates the distance to a decision boundary using adversarial learning methods and uses this distance to infer whether a test-time input sample is Trojaned or not. We evaluate MDTD against state-of-the-art Trojan detection methods across five widely used image-based datasets: CIFAR100, CIFAR10, GTSRB, SVHN, and Flowers102; four graph-based datasets: AIDS, WinMal, Toxicant, and COLLAB; and the SpeechCommand audio dataset. MDTD effectively identifies samples that contain different types of Trojan triggers. We evaluate MDTD against adaptive attacks where an adversary trains a robust DNN to increase (decrease) distance of benign (Trojan) inputs from a decision boundary.Comment: Accepted to ACM Conference on Computer and Communications Security (ACM CCS) 202

Heavy metal induced ecological risk in the city of Urumqi, NW China

A total of 169 samples of road dust collected in the city of Urumqi, capital of the Xinjiang Uygur Autonomous Region in northwest China, were analyzed by method of inductively coupled plasma-mass spectrometry for 10 elements (i.e., Cd, Cr, Cu, Ni, Pb, Mn, Be, Co, Zn, and U). The possible sources of metals are identified with multivariate analysis such as correlation analysis, principal component analysis, and cluster analysis. Besides, enrichment factors are used to quantitatively evaluate the influences of human activities on heavy metal concentrations. Moreover, the potential ecological risk index is applied to evaluating the ecological risk of heavy metal pollutants. The results indicate that: (1) the concentrations of the heavy metals involved were much higher in urban areas than the background values, except those of Co and U. Mn, U, and Co are mainly of natural origin; Cu, Pb, Zn, and Cr are mainly of traffic sources and are partly of industrial sources; Ni and Be are mainly the results of industrial activities, such as machine shops, firepower plants, tire and rubber factories, cement factories, and textile mills and are partly of the traffic sources; (2) with high "toxic-response" factor and high concentration, Cd has more serious influences on the environment than other heavy metals. Therefore, commercial and industrial areas are usually characterized by higher potential ecological risk when compared with residential areas and new developing urban areas. The results of this study could be helpful for the management of environment in industrial areas

Contamination levels assessment of potential toxic metals in road dust deposited in different types of urban environment

A total of 42 samples of road dust were collected along ring road, city centre, city side, and freeway in Urumqi, China. Total concentrations of Cd, Cr, Cu, Ni, Pb, Mn, Be, Co, Zn, and U were determined by using the inductively coupled plasma-mass spectrometry in order to assess and to compare road dust contamination levels of metals among the four roads. The results show that, among the four categories of roads, mean concentrations of Co and U vary little. City centre locations show strong enrichments of Cd, Cu, Pb, and Be. Along the ring road, the highest mean concentrations were found for Cr, Ni, Mn, and Co. However, the highest concentrations of Zn and U were found along the freeway. The cluster analysis shows that three main groups can be distinguished. Every group may be associated with different main sources and concentrations of the metals. The results of contamination assessment reveal that, among all of the potential toxic metals, Cd, Cu, and Zn pollution were obviously heavier with moderate or high contamination indices for most road dust samples, while Cr, Ni, and Pb contamination were lower along the four categories of roads. Compared with the city side, Cd, Cu, Pb, Ni, and Zn contamination were heavier along the ring road, the city centre, and the freeway with high traffic density. Low Pb contamination or no contamination in all the road dust samples may be related to the increasing usage of lead-free petrol

Spatial distribution and contamination assessment of heavy metals in urban road dusts from Urumqi, NW China

This study reports the spatial distribution pattern and degree of heavy metal pollution (Cd, Cr, Cu, Ni, Pb, Mn, Be, Co, Zn and U) in 169 urban road dust samples from urban area of Urumqi city. The spatial distribution pattern shows that Cu, Pb, Cr and Zn have similar patterns of spatial distribution. Their hot-spot areas were mainly associated with main roads where high traffic density was identified. Ni and Mn show similar spatial distributions coinciding with the industrial areas, while the spatial distribution patterns of Co and U show hot-spot areas were mainly located in the sides of the urban area where the road dust was significantly influenced by natural soils. The spatial distributions of Be and Cd were very different from other metals. The geo-accumulation index suggests that road dust in Urumqi city was uncontaminated to moderately contaminated with Cd, Cu, Ni, Pb, Mn, Be, Zn and U. The integrated pollution index shows IPIs of all road dust samples were higher than 1, suggesting that the road dust quality of Urumqi city has clearly been polluted by anthropogenic emission of heavy metals. Moreover, the spatial distribution pattern of IPIs also shows several distribution trends in the studied region. (C) 2009 Elsevier B.V. All rights reserved

Auxin response factor 6A regulates photosynthesis, sugar accumulation, and fruit development in tomato.

Auxin response factors (ARFs) are involved in auxin-mediated transcriptional regulation in plants. In this study, we performed functional characterization of SlARF6A in tomato. SlARF6A is located in the nucleus and exhibits transcriptional activator activity. Overexpression of SlARF6A increased chlorophyll contents in the fruits and leaves of tomato plants, whereas downregulation of SlARF6A decreased chlorophyll contents compared with those of wild-type (WT) plants. Analysis of chloroplasts using transmission electron microscopy indicated increased sizes of chloroplasts in SlARF6A-overexpressing plants and decreased numbers of chloroplasts in SlARF6A-downregulated plants. Overexpression of SlARF6A increased the photosynthesis rate and accumulation of starch and soluble sugars, whereas knockdown of SlARF6A resulted in opposite phenotypes in tomato leaves and fruits. RNA-sequence analysis showed that regulation of SlARF6A expression altered the expression of genes involved in chlorophyll metabolism, photosynthesis and sugar metabolism. SlARF6A directly bound to the promoters of SlGLK1, CAB, and RbcS genes and positively regulated the expression of these genes. Overexpression of SlARF6A also inhibited fruit ripening and ethylene production, whereas downregulation of SlARF6A increased fruit ripening and ethylene production. SlARF6A directly bound to the SAMS1 promoter and negatively regulated SAMS1 expression. Taken together, these results expand our understanding of ARFs with regard to photosynthesis, sugar accumulation and fruit development and provide a potential target for genetic engineering to improve fruit nutrition in horticulture crops

Observed trends of heating and cooling degree-days in Xinjiang Province, China

Global warming has the potential to impact various aspects of human society such as agriculture, construction, transportation, water resources management, power generation, and phenology. The impact on energy, especially energy consumption for heating and cooling of buildings, is very important. These influences are different in terms of space and time due to spatial and temporal variations of temperature. In this study, daily data of minimum and maximum temperature of 51 stations for 1959-2004 were used to detect annual and seasonal variations of heating and cooling degree-days in Xinjiang, China, by using the Mann-Kendall trend test and linear regression techniques. The results indicate that: (1) taking 18A degrees C as the base temperature, annual mean heating degree-days (HDD) ranged between 2,700 and 7,973A degrees C, and annual mean cooling degree-days (CDD) (the base temperature is 24A degrees C) ranged between 0.4 and 792A degrees C. CDDs are relatively low in Xinjiang; (2) autumn, winter, and annual HDDs show significant decreasing trends. Annual CDD at 23 out of 51 stations present significant increasing trends, while no remarkable positive trends can be observed at the other stations; and (3) with respect to spatial variations, Xinjiang was characterized by significant decreasing annual, winter, and autumn HDDs, and it was particularly true for the northern Xinjiang. The annual and summer CDDs in the western parts of northern Xinjiang (the edges of the Tarim Basin and the Turpan-Hami Basin) were characterized by significant increasing trends. However, no fixed spatial patterns can be identified in the variations of annual and summer CDDs. The results of this study could be useful for energy management in Xinjiang and are also helpful for better understanding of impacts of global warming on energy consumption in other countries of the world

Deficiency of N-linked glycosylation impairs immune function of B7-H6

B7-H6 is a novel immune checkpoint molecule that triggers NK cell cytotoxicity, but the role of N-glycosylation in B7-H6 is poorly understood. We here identified the existence of N-glycosylation of B7-H6 in different cell lines and exogenous expression cells by PNGase F digestion and tunicamycin blockage. Subsequently, we demonstrated that B7-H6 contains 6 functional N-linked glycosylation sites by single site mutation and electrophoresis. Phylogenetical and structural analysis revealed that N43 and N208 glycan are conserved in jawed vertebrates and may thus contribute more to the biological functions. We further demonstrated that N43 and N208 glycosylation are essential for B7-H6 to trigger NK cell activation. Mechanistically, we found that N43 and N208 glycan contributed to the stability and membrane expression of B7-H6 protein. Lack of N208 glycosylation led to membrane B7-H6 shedding, while N43 mutation resulted in impaired B7-H6/NKp30 binding affinity. Together, our findings highlight the significance of N-linked glycosylation in B7-H6 biological functions and suggest potential targets for modulating NK cell-mediated immunity

Myoglobinopathy is an adult-onset autosomal dominant myopathy with characteristic sarcoplasmic inclusions

Myoglobin, encoded by MB, is a small cytoplasmic globular hemoprotein highly expressed in cardiac myocytes and oxidative skeletal myofibers. Myoglobin binds O-2, facilitates its intracellular transport and serves as a controller of nitric oxide and reactive oxygen species. Here, we identify a recurrent c.292C>T ( p.His98Tyr) substitution in MB in fourteen members of six European families suffering from an autosomal dominant progressive myopathy with highly characteristic sarcoplasmic inclusions in skeletal and cardiac muscle. Myoglobinopathy manifests in adulthood with proximal and axial weakness that progresses to involve distal muscles and causes respiratory and cardiac failure. Biochemical characterization reveals that the mutant myoglobin has altered O-2 binding, exhibits a faster heme dissociation rate and has a lower reduction potential compared to wild-type myoglobin. Preliminary studies show that mutant myoglobin may result in elevated superoxide levels at the cellular level. These data define a recognizable muscle disease associated with MB mutation.Peer reviewe