Unlock the Potential of Counterfactually-Augmented Data in Out-Of-Distribution Generalization

Counterfactually-Augmented Data (CAD) -- minimal editing of sentences to flip the corresponding labels -- has the potential to improve the Out-Of-Distribution (OOD) generalization capability of language models, as CAD induces language models to exploit domain-independent causal features and exclude spurious correlations. However, the empirical results of CAD's OOD generalization are not as efficient as anticipated. In this study, we attribute the inefficiency to the myopia phenomenon caused by CAD: language models only focus on causal features that are edited in the augmentation operation and exclude other non-edited causal features. Therefore, the potential of CAD is not fully exploited. To address this issue, we analyze the myopia phenomenon in feature space from the perspective of Fisher's Linear Discriminant, then we introduce two additional constraints based on CAD's structural properties (dataset-level and sentence-level) to help language models extract more complete causal features in CAD, thereby mitigating the myopia phenomenon and improving OOD generalization capability. We evaluate our method on two tasks: Sentiment Analysis and Natural Language Inference, and the experimental results demonstrate that our method could unlock the potential of CAD and improve the OOD generalization performance of language models by 1.0% to 5.9%.Comment: Expert Systems With Applications 2023. arXiv admin note: text overlap with arXiv:2302.0934

Chain-of-Thought Tuning: Masked Language Models can also Think Step By Step in Natural Language Understanding

Chain-of-Thought (CoT) is a technique that guides Large Language Models (LLMs) to decompose complex tasks into multi-step reasoning through intermediate steps in natural language form. Briefly, CoT enables LLMs to think step by step. However, although many Natural Language Understanding (NLU) tasks also require thinking step by step, LLMs perform less well than small-scale Masked Language Models (MLMs). To migrate CoT from LLMs to MLMs, we propose Chain-of-Thought Tuning (CoTT), a two-step reasoning framework based on prompt tuning, to implement step-by-step thinking for MLMs on NLU tasks. From the perspective of CoT, CoTT's two-step framework enables MLMs to implement task decomposition; CoTT's prompt tuning allows intermediate steps to be used in natural language form. Thereby, the success of CoT can be extended to NLU tasks through MLMs. To verify the effectiveness of CoTT, we conduct experiments on two NLU tasks: hierarchical classification and relation extraction, and the results show that CoTT outperforms baselines and achieves state-of-the-art performance.Comment: EMNLP2023 Main Conferenc

Modelling of Mg doped ZnO TFTs

The ever increase in use of ZnO TFTs requires further in depth analysis to obtain the true transport mechanisms. This paper explores the modelling of MgZnO TFTs using a defect state based model based on multiple trapping and release and successfully validates the model with the fitting parameters VFB, To, Nt and σo

ScalAna: Automating Scaling Loss Detection with Graph Analysis

Scaling a parallel program to modern supercomputers is challenging due to inter-process communication, Amdahl's law, and resource contention. Performance analysis tools for finding such scaling bottlenecks either base on profiling or tracing. Profiling incurs low overheads but does not capture detailed dependencies needed for root-cause analysis. Tracing collects all information at prohibitive overheads. In this work, we design ScalAna that uses static analysis techniques to achieve the best of both worlds - it enables the analyzability of traces at a cost similar to profiling. ScalAna first leverages static compiler techniques to build a Program Structure Graph, which records the main computation and communication patterns as well as the program's control structures. At runtime, we adopt lightweight techniques to collect performance data according to the graph structure and generate a Program Performance Graph. With this graph, we propose a novel approach, called backtracking root cause detection, which can automatically and efficiently detect the root cause of scaling loss. We evaluate ScalAna with real applications. Results show that our approach can effectively locate the root cause of scaling loss for real applications and incurs 1.73% overhead on average for up to 2,048 processes. We achieve up to 11.11% performance improvement by fixing the root causes detected by ScalAna on 2,048 processes.Comment: conferenc

TiO2‐Based Schottky Diodes as Bidirectional Switches for Bipolar Resistive Memories

This study presents TiO2-based Schottky diodes designed as bidirectional switches for bipolar resistive memories. The TiO2 films in these Schottky diodes are prepared through an anodization process. The reverse current of these diodes exhibits an exponential increase with rising reverse voltage, ultimately matching the forward current. When two diodes are connected back-to-back, they demonstrate superior current–voltage symmetry and provide a wider off-state voltage range compared to a single diode, reaching up to 3.65 V. The adjustable off-state voltage range (0.40–3.65 V) of the switch, whether utilizing two diodes or a single diode, correlates well with the TiO2 layer thickness and oxygen partial pressure during Pt electrode sputtering. These diodes possess bidirectional switching characteristics and can serve as effective switch elements to address the sneak-path issue in bipolar resistive memories

Preparation of a nano emodin transfersome and study on its anti-obesity mechanism in adipose tissue of diet-induced obese rats

OBJECTIVE: To describe the preparation of nano emodin transfersome (NET) and investigate its effect on mRNA expression of adipose triglyceride lipase (ATGL) and G0/G1 switch gene 2 (G0S2) in adipose tissue of diet-induced obese rats. METHODS: NET was prepared by film-ultrasonic dispersion method. The effects of emodin components at different ratios on encapsulation efficiency were investigated.The NET envelopment rate was determined by ultraviolet spectrophotometry. The particle size and Zeta potential of NET were evaluated by Zetasizer analyzer. Sixty male SD rats were assigned to groups randomly. After 8-week treatment, body weight, wet weight of visceral fat and the percentage of body fat (PBF) were measured. Fasting blood glucose and serum lipid levels were determined. The adipose tissue section was HE stained, and the cellular diameter and quantity of adipocytes were evaluated by light microscopy. The mRNA expression of ATGL and G0S2 from the peri-renal fat tissue was assayed by RT-PCR. RESULTS: The appropriate formulation was deoxycholic acid sodium salt vs. phospholipids 1:8, cholesterol vs. phospholipids 1:3, vitamin Evs. phospholipids 1:20, and emodin vs. phospholipid 1:6. Zeta potential was −15.11 mV, and the particle size was 292.2 nm. The mean encapsulation efficiency was (69.35 ± 0.25)%. Compared with the obese model group, body weight, wet weight of visceral fat, PBF and mRNA expression of G0S2 from peri-renal fat tissue were decreased significantly after NET treatment (all P < 0.05), while high-density lipoprotein cholesterol (HDL-C), the diameter of adipocytes and mRNA expression of ATGL from peri-renal fat tissue were increased significantly (all P < 0.05). CONCLUSION: The preparation method is simple and reasonable. NET with negative electricity was small and uniform in particle size, with high encapsulation efficiency and stability. NET could reduce body weight and adipocyte size, and this effect was associated with the up-regulation of ATGL, down-regulation of G0S2 expression in the adipose tissue, and improved insulin sensitivity

Highly Stable and Conductive Microcapsules for Enhancement of Joule Heating Performance

Nanocarbons show great promise for establishing the next generation of Joule heating systems, but suffer from the limited maximum temperature due to precociously convective heat dissipation from electrothermal system to surrounding environment. Here we introduce a strategy to eliminate such convective heat transfer by inserting highly stable and conductive microcapsules into the electrothermal structures. The microcapsule is composed of encapsulated long-chain alkanes and graphene oxide/carbon nanotube hybrids as core and shell material, respectively. Multiform carbon nanotubes in the microspheres stabilize the capsule shell to resist volume-change-induced rupture during repeated heating/cooling process, and meanwhile enhance the thermal conductance of encapsulated alkanes which facilitates an expeditious heat exchange. The resulting microcapsules can be homogeneously incorporated in the nanocarbon-based electrothermal structures. At a dopant of 5%, the working temperature can be enhanced by 30% even at a low voltage and moderate temperature, which indicates a great value in daily household applications. Therefore, the stable and conductive microcapsule may serve as a versatile and valuable dopant for varieties of heat generation systems