A Systematic Evaluation of Large Language Models on Out-of-Distribution Logical Reasoning Tasks

Large language models (LLMs), such as GPT-3.5 and GPT-4, have greatly advanced the performance of artificial systems on various natural language processing tasks to human-like levels. However, their generalisation and robustness to perform logical reasoning remain under-evaluated. To probe this ability, we propose three new logical reasoning datasets named "ReClor-plus", "LogiQA-plus" and "LogiQAv2-plus", each featuring three subsets: the first with randomly shuffled options, the second with the correct choices replaced by "none of the other options are correct", and a combination of the previous two subsets. We carry out experiments on these datasets with both discriminative and generative LLMs and show that these simple tricks greatly hinder the performance of the language models. Despite their superior performance on the original publicly available datasets, we find that all models struggle to answer our newly constructed datasets. We show that introducing task variations by perturbing a sizable training set can markedly improve the model's generalisation and robustness in logical reasoning tasks. Moreover, applying logic-driven data augmentation for fine-tuning, combined with prompting can enhance the generalisation performance of both discriminative large language models and generative large language models. These results offer insights into assessing and improving the generalisation and robustness of large language models for logical reasoning tasks. We make our source code and data publicly available \url{https://github.com/Strong-AI-Lab/Logical-and-abstract-reasoning}.Comment: Accepted for oral presentation at the LLM@IJCAI 2023 non-archival symposiu

Enhancing Logical Reasoning of Large Language Models through Logic-Driven Data Augmentation

Combining large language models with logical reasoning enhance their capacity to address problems in a robust and reliable manner. Nevertheless, the intricate nature of logical reasoning poses challenges to gathering reliable data from web for building comprehensive training datasets, subsequently affecting the performance on downstream tasks. To address this, we introduce a novel logic-driven data augmentation approach, AMR-LDA. AMR-LDA converts the original text into an Abstract Meaning Representation (AMR) graph, a structured semantic representation that encapsulates the logic structure of the sentence, upon which operations are performed to generate logically modified AMR graphs. The modified AMR graphs are subsequently converted back into texts to create augmented data. Notably, our methodology is architecture-agnostic and enhances generative large language models, such as GPT-3.5 and GPT-4, through prompt augmentation, and fine-tuning discriminative large language models through contrastive learning with logic-driven data augmentation. Empirical evidence underscores the efficacy of our proposed method with improvement in performance across seven downstream tasks, such as logical reasoning reading comprehension, textual entailment, and natural language inference. Furthermore, our method ranked first on the ReClor leaderboard \url{https://eval.ai/web/challenges/challenge-page/503/leaderboard/1347}. The source code and data are publicly available \url{https://github.com/Strong-AI-Lab/Logical-Equivalence-driven-AMR-Data-Augmentation-for-Representation-Learning}.Comment: Accepted for oral presentation at the LLM@IJCAI 2023 non-archival symposiu

Crown ether decorated silicon photonics for safeguarding against lead poisoning

Lead (Pb2+) toxification in society is one of the most concerning public health crisis that remains unaddressed. The exposure to Pb2+ poisoning leads to a multitude of enduring health issues, even at the part-per-billion scale (ppb). Yet, public action dwarfs its impact. Pb2+ poisoning is estimated to account for 1 million deaths per year globally, which is in addition to its chronic impact on children. With their ring-shaped cavities, crown ethers are uniquely capable of selectively binding to specific ions. In this work, for the first time, the synergistic integration of highly-scalable silicon photonics, with crown ether amine conjugation via Fischer esterification in an environmentally-friendly fashion is demonstrated. This realises a photonic platform that enables the in-situ, highly-selective and quantitative detection of various ions. The development dispels the existing notion that Fischer esterification is restricted to organic compounds, laying the ground for subsequent amine conjugation for various crown ethers. In this work, the platform is engineered for Pb2+ detection, demonstrating a large dynamic detection range of 1 - 262000 ppb with high selectivity against a wide range of relevant ions. These results indicate the potential for the pervasive implementation of the technology to safeguard against ubiquitous lead poisoning in our society

Crystal network structure and stability of beeswax-based oleogels with different polyunsaturated fatty acid oils

The effect of different types of oils including camellia oil (CLO), sunflower oil (SFO), corn oil (CO) and linseed oil (LO) on the formation, crystal network structure and mechanical properties of 4%wt beeswax (BW) in oleogel was investigated. BW oleogels containing oils with higher contents of polyunsaturated fatty acids gelled first (1%wt), especially LO with higher contents of linolenic acid rather than CLO with higher contents of monounsaturated fatty acids. In comparison, oils with higher polyunsaturated fatty acid contents exhibited higher Db with more extensive microstructure at different cooling rates, which was related to shorter nucleation induction time of crystal and higher crystallinity. Stronger van der Waals forces were observed in oleogels with higher polyunsaturated fatty acid contents especially for LO oleogel. Rheology also showed that LO oleogel with higher content of linolenic acid had higher crystallinity and lower crystal melting interfacial tension, resulting in the formation of a more stable network structure

Effect of different chain lengths of monoglyceride on the O/W interfacial properties with high-melting and low-melting crystals in low-fat aerated emulsion

The effect of different types of monoglycerides, including monopalmitin, capryl monoglyceride (GMB), and succinylated monoglyceride (GMSA) in combination with palm kernel stearin (PKS) and beeswax (BW), on the formation, crystal network structure, and partial coalescence properties of aerated emulsions (20 % w/w fat) was investigated. The stability of BW and PKS crystals with a 1 % concentration of GMSA and GMB, respectively, in the oil phase was lower than the other crystals. BW-GMSA and PKS-GMB crystals exhibited a lower crystallization rate, higher contact angles and no significant peak shift in the small-angle X-ray scattering results. The BW-GMSA and PKS-GMB emulsions had a lower nucleation rate in the bulk and a higher nucleation rate at the interface, resulting in a higher fraction of crystals adsorbed at the oil/water interface. This reduced the number of interfacial proteins and led to a high degree of partial coalescence and the formation of stable aerated networks

Different hydrophilic polyglycerol fatty acid esters interact with fat crystals and proteins at the interface to co-stabilize highly unsaturated whipped emulsions

This study investigates the impact of different hydrophilic polyglycerol fatty acid esters (SWA-10D, M-7D and M-10D) on the stability of aerated emulsions containing palm oil stearin (solid fat content of 6 w/w) under varying pressures. The study encompasses a comparative analysis of the microstructure of droplets, distribution of fat crystals, and protein at the interface within these distinct emulsions. In addition, the study evaluates the stability of the emulsions after a whipping process. The microstructure of emulsions prepared with M-7D showed discernible evident bright rings that become more pronounced as pressure increased. Furthermore, the droplet size of M-7D emulsion was consistently smaller in comparison to M-10D and SWA-10D emulsions at different pressure levels. The M-7D emulsion exhibited a higher nucleation rate, featuring a greater count of crystal nuclei at the interface. Simultaneously, the interfacial protein content in the M-7D emulsion was lower compared to the other samples, diminishing from 2.5 mg/mL to 0.6 mg/mL as pressure increased. Consequently, the interface accommodated a higher concentration of interfacial fat crystals, while the protein content decreased, resulting in increased partial coalescence. This phenomenon, in turn, promoted the formation of a sharp rosette-shaped aerated structure, leading to a diminutive reduction of less than 10 in height over 6 h. This outcome serves as a clear indicator of the formation of a stable aerated structure

Beeswax crystals form a network structure in highly unsaturated oils and O/W emulsions under supersaturation and cool temperature conditions

Beeswax (BW) is widely used in structured oil to mimic fat crystals, due to its needle-like crystal structure and effective gelling ability. In this study, the crystallization behavior of BW in liquid oil was analyzed at different cooling temperatures and BW concentrations. Results showed that temperature and BW concentrations played a role in reaching the supersaturation state for the system of BW in linseed oil, with 40%BW at 20 °C and 20%BW at 5 °C, respectively. Short nuclear induction time and higher crystallinity of BW were found at lower cooling temperatures, promoting formation of supersaturation state. In addition, the apparent activation energy of the crystallization process indicates that crystallization is inhibited at high BW concentrations. Furthermore, higher supersaturation levels and lower cooling temperatures affected the droplet size and crystal structure of the O/W emulsions; these conditions accelerated the penetration of crystals through the interfacial membrane and led to droplet aggregation and coalescence. High supersaturation level and low cooling temperature promoted emulsion instability and had a remarkable impact on food quality

Weighted Gene Co-expression Network Analysis Identifies FKBP11 as a Key Regulator in Acute Aortic Dissection through a NF-kB Dependent Pathway

Acute aortic dissection (AAD) is a life-threatening disease. Despite the higher risk of mortality, currently there are no effective therapies that can ameliorate AAD development or progression. Identification of meaningful clusters of co-expressed genes or representative biomarkers for AAD may help to identify new pathomechanisms and foster development of new therapies. To this end, we performed a weighted gene co-expression network analysis (WGCNA) and calculated module-trait correlations based on a public microarray dataset (GSE 52093) and discovered 9 modules were found to be related to AAD. The module which has the strongest positive correlation with AAD was further analyzed and the top 10 hub genes SLC20A1, GINS2, CNN1, FAM198B, MAD2L2, UBE2T, FKBP11, SLMAP, CCDC34, and GALK1 were identified. Furthermore, we validated the data by qRT-PCR in an independent sample set originated from our study center. Overall, the qRT-PCR results were consistent with the results of the microarray analysis. Intriguingly, the highest change was found for FKBP11, a protein belongs to the FKBP family of peptidyl-prolyl cis/trans isomerases, which catalyze the folding of proline-containing polypeptides. In congruent with the gene expression analysis, FKBP11 expression was induced in cultured endothelial cells by angiotensin II treatment and endothelium of the dissected aorta. More importantly we show that FKBP11 provokes inflammation in endothelial cells by interacting with NF-kB p65 subunit, resulting in pro-inflammatory cytokines production. Accordingly, siRNA mediated knockdown of FKBP11 in cultured endothelial cells suppressed angiotensin II induced monocyte transmigration through the endothelial monolayer. Based on these data, we hypothesize that pro-inflammatory cytokines elicited by FKBP11 overexpression in the endothelium under AAD condition could facilitate transendothelial migration of the circulating monocytes into the aorta, where they differentiate into active macrophages and secrete MMPs and other extracellular matrix (ECM) degrading proteins, contributing to sustained inflammation and AAD. Taken together, our data identify important role of FKBP11 which can serve as biomarker and/or therapeutic target for AAD