527 research outputs found
DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuning
We propose Multiple Experts Fine-tuning Framework to build a financial large
language model (LLM), DISC-FinLLM. Our methodology improves general LLMs by
endowing them with multi-turn question answering abilities, domain text
processing capabilities, mathematical computation skills, and
retrieval-enhanced generation capabilities. We build a financial
instruction-tuning dataset named DISC-FIN-SFT, including instruction samples of
four categories (consulting, NLP tasks, computing and retrieval-augmented
generation). Evaluations conducted on multiple benchmarks demonstrate that our
model performs better than baseline models in various financial scenarios.
Further resources can be found at https://github.com/FudanDISC/DISC-FinLLM.Comment: 18 pages, 13 figures, 7 table
GenRES: Rethinking Evaluation for Generative Relation Extraction in the Era of Large Language Models
The field of relation extraction (RE) is experiencing a notable shift towards
generative relation extraction (GRE), leveraging the capabilities of large
language models (LLMs). However, we discovered that traditional relation
extraction (RE) metrics like precision and recall fall short in evaluating GRE
methods. This shortfall arises because these metrics rely on exact matching
with human-annotated reference relations, while GRE methods often produce
diverse and semantically accurate relations that differ from the references. To
fill this gap, we introduce GenRES for a multi-dimensional assessment in terms
of the topic similarity, uniqueness, granularity, factualness, and completeness
of the GRE results. With GenRES, we empirically identified that (1)
precision/recall fails to justify the performance of GRE methods; (2)
human-annotated referential relations can be incomplete; (3) prompting LLMs
with a fixed set of relations or entities can cause hallucinations. Next, we
conducted a human evaluation of GRE methods that shows GenRES is consistent
with human preferences for RE quality. Last, we made a comprehensive evaluation
of fourteen leading LLMs using GenRES across document, bag, and sentence level
RE datasets, respectively, to set the benchmark for future research in GR
Spatiotemporally Discriminative Video-Language Pre-Training with Text Grounding
Most of existing video-language pre-training methods focus on instance-level
alignment between video clips and captions via global contrastive learning but
neglect rich fine-grained local information, which is of importance to
downstream tasks requiring temporal localization and semantic reasoning. In
this work, we propose a simple yet effective video-language pre-training
framework, namely G-ViLM, to learn discriminative spatiotemporal features. Two
novel designs involving spatiotemporal grounding and temporal grouping promote
learning local region-noun alignment and temporal-aware features
simultaneously. Specifically, spatiotemporal grounding aggregates semantically
similar video tokens and aligns them with noun phrases extracted from the
caption to promote local region-noun correspondences. Moreover, temporal
grouping leverages cut-and-paste to manually create temporal scene changes and
then learns distinguishable features from different scenes. Comprehensive
evaluations demonstrate that G-ViLM performs favorably against existing
approaches on four representative downstream tasks, covering text-video
retrieval, video question answering, video action recognition and temporal
action localization. G-ViLM performs competitively on all evaluated tasks and
in particular achieves R@10 of 65.1 on zero-shot MSR-VTT retrieval, over 9%
higher than the state-of-the-art method
Discovery of Visual Semantics by Unsupervised and Self-Supervised Representation Learning
The success of deep learning in computer vision is rooted in the ability of
deep networks to scale up model complexity as demanded by challenging visual
tasks. As complexity is increased, so is the need for large amounts of labeled
data to train the model. This is associated with a costly human annotation
effort. To address this concern, with the long-term goal of leveraging the
abundance of cheap unlabeled data, we explore methods of unsupervised
"pre-training." In particular, we propose to use self-supervised automatic
image colorization.
We show that traditional methods for unsupervised learning, such as
layer-wise clustering or autoencoders, remain inferior to supervised
pre-training. In search for an alternative, we develop a fully automatic image
colorization method. Our method sets a new state-of-the-art in revitalizing old
black-and-white photography, without requiring human effort or expertise.
Additionally, it gives us a method for self-supervised representation learning.
In order for the model to appropriately re-color a grayscale object, it must
first be able to identify it. This ability, learned entirely self-supervised,
can be used to improve other visual tasks, such as classification and semantic
segmentation. As a future direction for self-supervision, we investigate if
multiple proxy tasks can be combined to improve generalization. This turns out
to be a challenging open problem. We hope that our contributions to this
endeavor will provide a foundation for future efforts in making
self-supervision compete with supervised pre-training.Comment: Ph.D. thesi
Towards using Cough for Respiratory Disease Diagnosis by leveraging Artificial Intelligence: A Survey
Cough acoustics contain multitudes of vital information about
pathomorphological alterations in the respiratory system. Reliable and accurate
detection of cough events by investigating the underlying cough latent features
and disease diagnosis can play an indispensable role in revitalizing the
healthcare practices. The recent application of Artificial Intelligence (AI)
and advances of ubiquitous computing for respiratory disease prediction has
created an auspicious trend and myriad of future possibilities in the medical
domain. In particular, there is an expeditiously emerging trend of Machine
learning (ML) and Deep Learning (DL)-based diagnostic algorithms exploiting
cough signatures. The enormous body of literature on cough-based AI algorithms
demonstrate that these models can play a significant role for detecting the
onset of a specific respiratory disease. However, it is pertinent to collect
the information from all relevant studies in an exhaustive manner for the
medical experts and AI scientists to analyze the decisive role of AI/ML. This
survey offers a comprehensive overview of the cough data-driven ML/DL detection
and preliminary diagnosis frameworks, along with a detailed list of significant
features. We investigate the mechanism that causes cough and the latent cough
features of the respiratory modalities. We also analyze the customized cough
monitoring application, and their AI-powered recognition algorithms. Challenges
and prospective future research directions to develop practical, robust, and
ubiquitous solutions are also discussed in detail.Comment: 30 pages, 12 figures, 9 table
Knowledge-Informed Machine Learning for Cancer Diagnosis and Prognosis: A review
Cancer remains one of the most challenging diseases to treat in the medical
field. Machine learning has enabled in-depth analysis of rich multi-omics
profiles and medical imaging for cancer diagnosis and prognosis. Despite these
advancements, machine learning models face challenges stemming from limited
labeled sample sizes, the intricate interplay of high-dimensionality data
types, the inherent heterogeneity observed among patients and within tumors,
and concerns about interpretability and consistency with existing biomedical
knowledge. One approach to surmount these challenges is to integrate biomedical
knowledge into data-driven models, which has proven potential to improve the
accuracy, robustness, and interpretability of model results. Here, we review
the state-of-the-art machine learning studies that adopted the fusion of
biomedical knowledge and data, termed knowledge-informed machine learning, for
cancer diagnosis and prognosis. Emphasizing the properties inherent in four
primary data types including clinical, imaging, molecular, and treatment data,
we highlight modeling considerations relevant to these contexts. We provide an
overview of diverse forms of knowledge representation and current strategies of
knowledge integration into machine learning pipelines with concrete examples.
We conclude the review article by discussing future directions to advance
cancer research through knowledge-informed machine learning.Comment: 41 pages, 4 figures, 2 table
- …