Deep Short Text Classification with Knowledge Powered Attention

Short text classification is one of important tasks in Natural Language Processing (NLP). Unlike paragraphs or documents, short texts are more ambiguous since they have not enough contextual information, which poses a great challenge for classification. In this paper, we retrieve knowledge from external knowledge source to enhance the semantic representation of short texts. We take conceptual information as a kind of knowledge and incorporate it into deep neural networks. For the purpose of measuring the importance of knowledge, we introduce attention mechanisms and propose deep Short Text Classification with Knowledge powered Attention (STCKA). We utilize Concept towards Short Text (C- ST) attention and Concept towards Concept Set (C-CS) attention to acquire the weight of concepts from two aspects. And we classify a short text with the help of conceptual information. Unlike traditional approaches, our model acts like a human being who has intrinsic ability to make decisions based on observation (i.e., training data for machines) and pays more attention to important knowledge. We also conduct extensive experiments on four public datasets for different tasks. The experimental results and case studies show that our model outperforms the state-of-the-art methods, justifying the effectiveness of knowledge powered attention

Ensuring Readability and Data-fidelity using Head-modifier Templates in Deep Type Description Generation

A type description is a succinct noun compound which helps human and machines to quickly grasp the informative and distinctive information of an entity. Entities in most knowledge graphs (KGs) still lack such descriptions, thus calling for automatic methods to supplement such information. However, existing generative methods either overlook the grammatical structure or make factual mistakes in generated texts. To solve these problems, we propose a head-modifier template-based method to ensure the readability and data fidelity of generated type descriptions. We also propose a new dataset and two automatic metrics for this task. Experiments show that our method improves substantially compared with baselines and achieves state-of-the-art performance on both datasets.Comment: ACL 201

Polymer Scaffolds for Small-Diameter Vascular Tissue Engineering

To better engineer small-diameter blood vessels, a few types of novel scaffolds are fabricated from biodegradable poly( L -lactic acid) (PLLA) by means of thermally induced phase-separation (TIPS) techniques. By utilizing the differences in thermal conductivities of the mold materials and using benzene as the solvent scaffolds with oriented gradient microtubular structures in the axial or radial direction can be created. The porosity, tubular size, and the orientational direction of the microtubules can be controlled by the polymer concentration, the TIPS temperature, and by utilizing materials of different thermal conductivities. These gradient microtubular structures facilitate cell seeding and mass transfer for cell growth and function. Nanofibrous scaffolds with an oriented and interconnected microtubular pore network are also developed by a one-step TIPS method using a benzene/tetrahydrofuran mixture as the solvent without the need for porogen materials. The structural features of such scaffolds can be conveniently adjusted by varying the solvent ratio, phase-separation temperature, and polymer concentration to mimic the nanofibrous features of an extracellular matrix. These scaffolds were fabricated for the tissue engineering of small-diameter blood vessels by utilizing their advantageous structural features to facilitate blood-vessel regeneration.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/78073/1/2833_ftp.pd

Uniqueness of the critical points of solutions to two kinds of semilinear elliptic equations in higher dimensional domains

In this paper, we provide an affirmative answer to the conjecture A for bounded simple rotationally symmetric domains $\Omega\subset \mathbb{R}^n(n\geq 3)$ along $x_n$ axis. Precisely, we use a new simple argument to study the symmetry of positive solutions for two kinds of semilinear elliptic equations. To do this, when $f(\cdot,s)$ is strictly convex with respect to $s$, we show that the nonnegativity of the first eigenvalue of the corresponding linearized operator in somehow symmetric domains is a sufficient condition for the symmetry of $u$. Moreover, we prove the uniqueness of critical points of a positive solution to semilinear elliptic equation $-\triangle u=f(\cdot,u)$ with zero Dirichlet boundary condition for simple rotationally symmetric domains in $\mathbb{R}^n$ by continuity method and a variety of maximum principles.Comment: 18 page

Sen2Pro: A Probabilistic Perspective to Sentence Embedding from Pre-trained Language Model

Sentence embedding is one of the most fundamental tasks in Natural Language Processing and plays an important role in various tasks. The recent breakthrough in sentence embedding is achieved by pre-trained language models (PLMs). Despite its success, an embedded vector (Sen2Vec) representing a point estimate does not naturally express uncertainty in a taskagnostic way. This paper thereby proposes an efficient framework on probabilistic sentence embedding (Sen2Pro) from PLMs, and it represents a sentence as a probability density distribution in an embedding space to reflect both model uncertainty and data uncertainty (i.e., many-to-one nature) in the sentence representation. The proposed framework performs in a plug-and-play way without retraining PLMs anymore, and it is easy to implement and generally applied on top of any PLM. The superiority of Sen2Pro over Sen2Vec has been theoretically verified and practically illustrated on different NLP tasks.Comment: Accepted to ACL2023 workshop Rep4NL

Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: A Preliminary Empirical Study

Evaluating the quality of generated text is a challenging task in natural language processing. This difficulty arises from the inherent complexity and diversity of text. Recently, OpenAI's ChatGPT, a powerful large language model (LLM), has garnered significant attention due to its impressive performance in various tasks. Therefore, we present this report to investigate the effectiveness of LLMs, especially ChatGPT, and explore ways to optimize their use in assessing text quality. We compared three kinds of reference-free evaluation methods based on ChatGPT or similar LLMs. The experimental results prove that ChatGPT is capable to evaluate text quality effectively from various perspectives without reference and demonstrates superior performance than most existing automatic metrics. In particular, the Explicit Score, which utilizes ChatGPT to generate a numeric score measuring text quality, is the most effective and reliable method among the three exploited approaches. However, directly comparing the quality of two texts using ChatGPT may lead to suboptimal results. We hope this report will provide valuable insights into selecting appropriate methods for evaluating text quality with LLMs such as ChatGPT.Comment: Technical Report, 13 page

TeGit: Generating High-Quality Instruction-Tuning Data with Text-Grounded Task Design

High-quality instruction-tuning data is critical to improving LLM capabilities. Existing data collection methods are limited by unrealistic manual labeling costs or by the hallucination of relying solely on LLM generation. To address the problems, this paper presents a scalable method to automatically collect high-quality instructional adaptation data by training language models to automatically design tasks based on human-written texts. Intuitively, human-written text helps to help the model attenuate illusions during the generation of tasks. Unlike instruction back-translation-based methods that directly take the given text as a response, we require the model to generate the \textit{instruction}, \textit{input}, and \textit{output} simultaneously to filter the noise. The results of the automated and manual evaluation experiments demonstrate the quality of our dataset.Comment: Work in progres

StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving

Most existing chain-of-thought (CoT) prompting methods suffer from the issues of generalizability and consistency, as they often rely on instance-specific solutions that may not be applicable to other cases and lack task-level consistency in their reasoning steps. To address these limitations, we propose a comprehensive framework, StrategyLLM, harnessing the capabilities of LLMs to tackle various tasks. The framework improves generalizability by formulating general problem-solving strategies and enhances consistency by producing consistent solutions using these strategies. StrategyLLM employs four LLM-based agents: strategy generator, executor, optimizer, and evaluator, working together to generate, evaluate, and select promising strategies for a given task automatically. The experimental results demonstrate that StrategyLLM outperforms the competitive baseline CoT-SC that requires human-annotated solutions on 13 datasets across 4 challenging tasks without human involvement, including math reasoning (39.2% $\rightarrow$ 43.3%), commonsense reasoning (70.3% $\rightarrow$ 72.5%), algorithmic reasoning (51.7% $\rightarrow$ 62.0%), and symbolic reasoning (30.0% $\rightarrow$ 79.2%)