76 research outputs found
Simple Smale flows and their templates on
The embedded template is a geometric tool in dynamics being used to model
knots and links as periodic orbits of -dimensional flows. We prove that for
an embedded template in with fixed homeomorphism type, its boundary as a
trivalent spatial graph is a complete isotopic invariant. Moreover, we
construct an invariant of embedded templates by Kauffman's invariant of spatial
graphs, which is a set of knots and links. As application, the isotopic
classification of simple Smale flows on is discussed.Comment: 14 pages, 3 figure
Quantifying and Attributing the Hallucination of Large Language Models via Association Analysis
Although demonstrating superb performance on various NLP tasks, large
language models (LLMs) still suffer from the hallucination problem, which
threatens the reliability of LLMs. To measure the level of hallucination of
LLMs, previous works first categorize the hallucination according to the
phenomenon similarity, then quantify the proportion that model outputs contain
hallucinatory contents. However, such hallucination rates could easily be
distorted by confounders. Moreover, such hallucination rates could not reflect
the reasons for the hallucination, as similar hallucinatory phenomena may
originate from different sources. To address these issues, we propose to
combine the hallucination level quantification and hallucination reason
investigation through an association analysis, which builds the relationship
between the hallucination rate of LLMs with a set of risk factors. In this way,
we are able to observe the hallucination level under each value of each risk
factor, examining the contribution and statistical significance of each risk
factor, meanwhile excluding the confounding effect of other factors.
Additionally, by recognizing the risk factors according to a taxonomy of model
capability, we reveal a set of potential deficiencies in commonsense
memorization, relational reasoning, and instruction following, which may
further provide guidance for the pretraining and supervised fine-tuning process
of LLMs to mitigate the hallucination
Meta-Learning Triplet Network with Adaptive Margins for Few-Shot Named Entity Recognition
Meta-learning methods have been widely used in few-shot named entity
recognition (NER), especially prototype-based methods. However, the Other(O)
class is difficult to be represented by a prototype vector because there are
generally a large number of samples in the class that have miscellaneous
semantics. To solve the problem, we propose MeTNet, which generates prototype
vectors for entity types only but not O-class. We design an improved triplet
network to map samples and prototype vectors into a low-dimensional space that
is easier to be classified and propose an adaptive margin for each entity type.
The margin plays as a radius and controls a region with adaptive size in the
low-dimensional space. Based on the regions, we propose a new inference
procedure to predict the label of a query instance. We conduct extensive
experiments in both in-domain and cross-domain settings to show the superiority
of MeTNet over other state-of-the-art methods. In particular, we release a
Chinese few-shot NER dataset FEW-COMM extracted from a well-known e-commerce
platform. To the best of our knowledge, this is the first Chinese few-shot NER
dataset. All the datasets and codes are provided at
https://github.com/hccngu/MeTNet
Exchanging-based Multimodal Fusion with Transformer
We study the problem of multimodal fusion in this paper. Recent
exchanging-based methods have been proposed for vision-vision fusion, which aim
to exchange embeddings learned from one modality to the other. However, most of
them project inputs of multimodalities into different low-dimensional spaces
and cannot be applied to the sequential input data. To solve these issues, in
this paper, we propose a novel exchanging-based multimodal fusion model MuSE
for text-vision fusion based on Transformer. We first use two encoders to
separately map multimodal inputs into different low-dimensional spaces. Then we
employ two decoders to regularize the embeddings and pull them into the same
space. The two decoders capture the correlations between texts and images with
the image captioning task and the text-to-image generation task, respectively.
Further, based on the regularized embeddings, we present CrossTransformer,
which uses two Transformer encoders with shared parameters as the backbone
model to exchange knowledge between multimodalities. Specifically,
CrossTransformer first learns the global contextual information of the inputs
in the shallow layers. After that, it performs inter-modal exchange by
selecting a proportion of tokens in one modality and replacing their embeddings
with the average of embeddings in the other modality. We conduct extensive
experiments to evaluate the performance of MuSE on the Multimodal Named Entity
Recognition task and the Multimodal Sentiment Analysis task. Our results show
the superiority of MuSE against other competitors. Our code and data are
provided at https://github.com/RecklessRonan/MuSE
FLM-101B: An Open LLM and How to Train It with $100K Budget
Large language models (LLMs) have achieved remarkable success in NLP and
multimodal tasks, among others. Despite these successes, two main challenges
remain in developing LLMs: (i) high computational cost, and (ii) fair and
objective evaluations. In this paper, we report a solution to significantly
reduce LLM training cost through a growth strategy. We demonstrate that a
101B-parameter LLM with 0.31T tokens can be trained with a budget of 100K US
dollars. Inspired by IQ tests, we also consolidate an additional range of
evaluations on top of existing evaluations that focus on knowledge-oriented
abilities. These IQ evaluations include symbolic mapping, rule understanding,
pattern mining, and anti-interference. Such evaluations minimize the potential
impact of memorization. Experimental results show that our model, named
FLM-101B, trained with a budget of 100K US dollars, achieves performance
comparable to powerful and well-known models, e.g., GPT-3 and GLM-130B,
especially on the additional range of IQ evaluations. The checkpoint of
FLM-101B is released at https://huggingface.co/CofeAI/FLM-101B
A novel dilated contextual attention module for breast cancer mitosis cell detection
Background and object: Mitotic count (MC) is a critical histological parameter for accurately assessing the degree of invasiveness in breast cancer, holding significant clinical value for cancer treatment and prognosis. However, accurately identifying mitotic cells poses a challenge due to their morphological and size diversity.Objective: We propose a novel end-to-end deep-learning method for identifying mitotic cells in breast cancer pathological images, with the aim of enhancing the performance of recognizing mitotic cells.Methods: We introduced the Dilated Cascading Network (DilCasNet) composed of detection and classification stages. To enhance the model’s ability to capture distant feature dependencies in mitotic cells, we devised a novel Dilated Contextual Attention Module (DiCoA) that utilizes sparse global attention during the detection. For reclassifying mitotic cell areas localized in the detection stage, we integrate the EfficientNet-B7 and VGG16 pre-trained models (InPreMo) in the classification step.Results: Based on the canine mammary carcinoma (CMC) mitosis dataset, DilCasNet demonstrates superior overall performance compared to the benchmark model. The specific metrics of the model’s performance are as follows: F1 score of 82.9%, Precision of 82.6%, and Recall of 83.2%. With the incorporation of the DiCoA attention module, the model exhibited an improvement of over 3.5% in the F1 during the detection stage.Conclusion: The DilCasNet achieved a favorable detection performance of mitotic cells in breast cancer and provides a solution for detecting mitotic cells in pathological images of other cancers
Inter-patient ECG heartbeat classification for arrhythmia classification: a new approach of multi-layer perceptron with weight capsule and sequence-to-sequence combination
Objective: The objective of this research is to construct a method to alleviate the problem of sample imbalance in classification, especially for arrhythmia classification. This approach can improve the performance of the model without using data enhancement.Methods: In this study, we have developed a new Multi-layer Perceptron (MLP) block and have used a Weight Capsule (WCapsule) network with MLP combined with sequence-to-sequence (Seq2Seq) network to classify arrhythmias. Our work is based on the MIT-BIH arrhythmia database, the original electrocardiogram (ECG) data is classified according to the criteria recommended by the American Association for Medical Instrumentation (AAMI). Also, our method’s performance is further evaluated.Results: The proposed model is evaluated using the inter-patient paradigm. Our proposed method shows an accuracy (ACC) of 99.88% under sample imbalance. For Class N, sensitivity (SEN) is 99.79%, positive predictive value (PPV) is 99.90%, and specificity (SPEC) is 99.19%. For Class S, SEN is 97.66%, PPV is 96.14%, and SPEC is 99.85%. For Class V, SEN is 99.97%, PPV is 99.07%, and SPEC is 99.94%. For Class F, SEN is 97.94%, PPV is 98.70%, and SPEC is 99.99%. When using only half of the training sample, our method shows that the SEN of Class N and V is 0.97% and 5.27% higher than the traditional machine learning algorithm.Conclusion: The proposed method combines MLP, weight capsule network with Seq2seq network, effectively addresses the problem of sample imbalance in arrhythmia classification, and produces good performance. Our method also shows promising potential in less samples
- …