Discover, Explanation, Improvement: Automatic Slice Detection Framework for Natural Language Processing

Current natural language processing (NLP) models such as BERT and RoBERTa have achieved high overall performance, but they often make systematic errors due to bias or certain difficult features to learn. Thus research on slice detection models (SDM) which automatically identifies underperforming groups of datapoints has gradually caught more attention, which aims at both understanding model behaviors and providing insights for future model training and designing. However, there is little systematic research on SDM and quantitative evaluation of its assessment for NLP models. Our paper fills this gap by proposing "Discover, Explanation, Improvement" framework that discovers coherent and underperforming groups of datapoints and unites datapoints of each slice under human-understandable concepts; it also provides comprehensive evaluation tasks and the corresponding quantitative metrics, which enable convenient comparison for future works. Results show that our framework can accurately select error-prone datapoints with informative semantic features that summarize error patterns, based on which it directly boosts model performance by an average of 2.85 points based on trained models without tuning any parameters across multiple datasets.Comment: 15 pages, 5 figure

OpenAGI: When LLM Meets Domain Experts

Human intelligence has the remarkable ability to assemble basic skills into complex ones so as to solve complex tasks. This ability is equally important for Artificial Intelligence (AI), and thus, we assert that in addition to the development of large, comprehensive intelligent models, it is equally crucial to equip such models with the capability to harness various domain-specific expert models for complex task-solving in the pursuit of Artificial General Intelligence (AGI). Recent developments in Large Language Models (LLMs) have demonstrated remarkable learning and reasoning abilities, making them promising as a controller to select, synthesize, and execute external models to solve complex tasks. In this project, we develop OpenAGI, an open-source AGI research platform, specifically designed to offer complex, multi-step tasks and accompanied by task-specific datasets, evaluation metrics, and a diverse range of extensible models. OpenAGI formulates complex tasks as natural language queries, serving as input to the LLM. The LLM subsequently selects, synthesizes, and executes models provided by OpenAGI to address the task. Furthermore, we propose a Reinforcement Learning from Task Feedback (RLTF) mechanism, which uses the task-solving result as feedback to improve the LLM's task-solving ability. Thus, the LLM is responsible for synthesizing various external models for solving complex tasks, while RLTF provides feedback to improve its task-solving ability, enabling a feedback loop for self-improving AI. We believe that the paradigm of LLMs operating various expert models for complex task-solving is a promising approach towards AGI. To facilitate the community's long-term improvement and evaluation of AGI's ability, we open-source the code, benchmark, and evaluation methods of the OpenAGI project at https://github.com/agiresearch/OpenAGI.Comment: 18 pages, 6 figures, 7 table

GenRec: Large Language Model for Generative Recommendation

In recent years, large language models (LLM) have emerged as powerful tools for diverse natural language processing tasks. However, their potential for recommender systems under the generative recommendation paradigm remains relatively unexplored. This paper presents an innovative approach to recommendation systems using large language models (LLMs) based on text data. In this paper, we present a novel LLM for generative recommendation (GenRec) that utilized the expressive power of LLM to directly generate the target item to recommend, rather than calculating ranking score for each candidate item one by one as in traditional discriminative recommendation. GenRec uses LLM's understanding ability to interpret context, learn user preferences, and generate relevant recommendation. Our proposed approach leverages the vast knowledge encoded in large language models to accomplish recommendation tasks. We first we formulate specialized prompts to enhance the ability of LLM to comprehend recommendation tasks. Subsequently, we use these prompts to fine-tune the LLaMA backbone LLM on a dataset of user-item interactions, represented by textual data, to capture user preferences and item characteristics. Our research underscores the potential of LLM-based generative recommendation in revolutionizing the domain of recommendation systems and offers a foundational framework for future explorations in this field. We conduct extensive experiments on benchmark datasets, and the experiments shows that our GenRec has significant better results on large dataset

Inactivation of TRPM2 channels by extracellular divalent copper

Cu2+ is an essential metal ion that plays a critical role in the regulation of a number of ion channels and receptors in addition to acting as a cofactor in a variety of enzymes. Here, we showed that human melastatin transient receptor potential 2 (hTRPM2) channel is sensitive to inhibition by extracellular Cu2+. Cu2 + at concentrations as low as 3 μM inhibited the hTRPM2 channel completely and irreversibly upon washing or using Cu2+ chelators, suggesting channel inactivation. The Cu2+-induced inactivation was similar when the channels conducted inward or outward currents, indicating the permeating ions had little effect on Cu2+-induced inactivation. Furthermore, Cu2+ had no effect on singe channel conductance. Alanine substitution by site-directed mutagenesis of His995 in the pore-forming region strongly attenuated Cu2+-induced channel inactivation, and mutation of several other pore residues to alanine altered the kinetics of channel inactivation by Cu2+. In addition, while introduction of the P1018L mutation is known to result in channel inactivation, exposure to Cu2+ accelerated the inactivation of this mutant channel. In contrast with the hTRPM2, the mouse TRPM2 (mTRPM2) channel, which contains glutamine at the position equivalent to His995, was insensitive to Cu2+. Replacement of His995 with glutamine in the hTRPM2 conferred loss of Cu2+-induced channel inactivation. Taken together, these results suggest that Cu2+ inactivates the hTRPM2 channel by interacting with the outer pore region. Our results also indicate that the amino acid residue difference in this region gives rise to species-dependent effect by Cu2+ on the human and mouse TRPM2 channels

Computational Complexity in Optional Syllabification of Yavapai

In addition to the substance in phonology, a number of researchers have argued that computation also matters in phonology. Using the data in Yavapai (Yuman language), I show that other than an OT analysis focusing mainly on substance, a computational analysis is necessary for explaining the complex syllabification processes and the frequencies of optional surface representations due to different syllabifications. I will use computational complexity encoded in subregular hierarchy as the main technical tool in the computational analysis. Our main hypothesis is that when both SRs are well-formed based on the syllable phonotactics, the one less complex to generate is more frequently attested. The paper shows that the syllabification pattern in Yavapai necessarily requires a computational motivation, which in turn shows that computational property is a crucial factor in phonological transformations

How to Index Item IDs for Recommendation Foundation Models

Recommendation foundation model utilizes large language models (LLM) for recommendation by converting recommendation tasks into natural language tasks. It enables generative recommendation which directly generates the item(s) to recommend rather than calculating a ranking score for each and every candidate item in traditional recommendation models, simplifying the recommendation pipeline from multi-stage filtering to single-stage filtering. To avoid generating excessively long text when deciding which item(s) to recommend, creating LLM-compatible item IDs is essential for recommendation foundation models. In this study, we systematically examine the item indexing problem for recommendation foundation models, using P5 as the representative backbone model and replicating its results with various indexing methods. To emphasize the importance of item indexing, we first discuss the issues of several trivial item indexing methods, such as independent indexing, title indexing, and random indexing. We then propose four simple yet effective solutions, including sequential indexing, collaborative indexing, semantic (content-based) indexing, and hybrid indexing. Our reproducibility study of P5 highlights the significant influence of item indexing methods on the model performance, and our results on real-world datasets validate the effectiveness of our proposed solutions.Comment: 12 pages, 6 figure

Voltage-independent effects by extracellular Cu²⁺ on inward and outward TRPM2 channel currents.

<p>(A) ADPR-induced currents mediated by the hTRPM2 channels at a holding membrane potentials (HP) of +40 mV (outward currents) or −40 mV (inward currents) and the effect of 100 µM Cu²⁺. The dotted lines indicate the baseline. (B–C) Summary of the outward or inward current amplitude before and after exposure to Cu²⁺ and upon washout, as shown in (A). The number of cells examined in each case is 4. ***, p<0.005 compared with the currents before and after exposure to the indicated Cu²⁺. (D) Summary of the time for 90% inhibition at both 40 mV and −40 mV, there is no significant difference between these groups.</p

Effects of external Cu²⁺ on human TRPM2 single channel conductance.

<p>(A) Representative recordings in the outside-out configuration of the effects of 30 µM Cu²⁺ on ADPR-induced TRPM2 currents (in red). Single channel events are clearly visible in the expanded traces illustrated below. (B) The grey and red histograms of single channel events indicated the exposure in ECS and 30 µM Cu²⁺, respectively. The superimposed curve represents a fit of a double­Gaussian function.</p

TRPM2 open channels inactivated by extracellular Cu²⁺.

<p>(A) Representative recordings of the inward currents evoked by 500 µM ADPR at −80 mV, using a 500 ms voltage ramp of −100 mV to +100 mV applied every 5 s, before and after exposure to the indicated Cu²⁺ concentrations. The dotted lines indicate zero currents. (B–C) Summary of the percentage inhibition (B) and time required for inward current amplitude reached 90% inhibition after Cu²⁺ exposure (C). (D) Left panel, the ADPR-induced inward currents when fully inhibited by 100 µM Cu²⁺ were not reversed after treating with 20 µM 2-ME; Right panel, summary of the current recovery during exposure to 2-ME. (E) Left panel, the ADPR-induced inward currents when fully inhibited by 100 µM Cu²⁺ were not reversed after treating with 5 mM EDTA; Right panel, summary of the current recovery during exposure to EDTA. Residual current expressed as the percentage of the currents immediately before exposure to Cu²⁺ is 3.3±1.7% after inactivation by Cu²⁺, which returned to 2.6±0.9% after washing with EDTA. In 2-ME group, residual current changed from 1.8±0.5% to 2.0±0.7%. The number of cells examined in each case is 4–6.</p