188 research outputs found
On the Evaluation of Generative Models in Distributed Learning Tasks
The evaluation of deep generative models including generative adversarial
networks (GANs) and diffusion models has been extensively studied in the
literature. While the existing evaluation methods mainly target a centralized
learning problem with training data stored by a single client, many
applications of generative models concern distributed learning settings, e.g.
the federated learning scenario, where training data are collected by and
distributed among several clients. In this paper, we study the evaluation of
generative models in distributed learning tasks with heterogeneous data
distributions. First, we focus on the Fr\'echet inception distance (FID) and
consider the following FID-based aggregate scores over the clients: 1) FID-avg
as the mean of clients' individual FID scores, 2) FID-all as the FID distance
of the trained model to the collective dataset containing all clients' data. We
prove that the model rankings according to the FID-all and FID-avg scores could
be inconsistent, which can lead to different optimal generative models
according to the two aggregate scores. Next, we consider the kernel inception
distance (KID) and similarly define the KID-avg and KID-all aggregations.
Unlike the FID case, we prove that KID-all and KID-avg result in the same
rankings of generative models. We perform several numerical experiments on
standard image datasets and training schemes to support our theoretical
findings on the evaluation of generative models in distributed learning
problems.Comment: 17 pages, 10 figure
Unlock Multi-Modal Capability of Dense Retrieval via Visual Module Plugin
This paper proposes Multi-modAl Retrieval model via Visual modulE pLugin
(MARVEL) to learn an embedding space for queries and multi-modal documents to
conduct retrieval. MARVEL encodes queries and multi-modal documents with a
unified encoder model, which helps to alleviate the modality gap between images
and texts. Specifically, we enable the image understanding ability of a
well-trained dense retriever, T5-ANCE, by incorporating the image features
encoded by the visual module as its inputs. To facilitate the multi-modal
retrieval tasks, we build the ClueWeb22-MM dataset based on the ClueWeb22
dataset, which regards anchor texts as queries, and exact the related texts and
image documents from anchor linked web pages. Our experiments show that MARVEL
significantly outperforms the state-of-the-art methods on the multi-modal
retrieval dataset WebQA and ClueWeb22-MM. Our further analyses show that the
visual module plugin method is tailored to enable the image understanding
ability for an existing dense retrieval model. Besides, we also show that the
language model has the ability to extract image semantics from image encoders
and adapt the image features in the input space of language models. All codes
are available at https://github.com/OpenMatch/MARVEL
Automated damage diagnosis of concrete jack arch beam using optimized deep stacked autoencoders and multi-sensor fusion
A novel hybrid framework of optimized deep learning models combined with multi-sensor fusion is developed for condition diagnosis of concrete arch beam. The vibration responses of structure are first processed by principal component analysis for dimensionality reduction and noise elimination. Then, the deep network based on stacked autoencoders (SAE) is established at each sensor for initial condition diagnosis, where extracted principal components and corresponding condition categories are inputs and output, respectively. To enhance diagnostic accuracy of proposed deep SAE, an enhanced whale optimization algorithm is proposed to optimize network meta-parameters. Eventually, Dempster-Shafer fusion algorithm is employed to combine initial diagnosis results from each sensor to make a final diagnosis. A miniature structural component of Sydney Harbour Bridge with artificial multiple progressive damages is tested in laboratory. The results demonstrate that the proposed method can detect structural damage accurately, even under the condition of limited sensors and high levels of uncertainties
Modeling User Viewing Flow using Large Language Models for Article Recommendation
This paper proposes the User Viewing Flow Modeling (SINGLE) method for the
article recommendation task, which models the user constant preference and
instant interest from user-clicked articles. Specifically, we employ a user
constant viewing flow modeling method to summarize the user's general interest
to recommend articles. We utilize Large Language Models (LLMs) to capture
constant user preferences from previously clicked articles, such as skills and
positions. Then we design the user instant viewing flow modeling method to
build interactions between user-clicked article history and candidate articles.
It attentively reads the representations of user-clicked articles and aims to
learn the user's different interest views to match the candidate article. Our
experimental results on the Alibaba Technology Association (ATA) website show
the advantage of SINGLE, which achieves 2.4% improvements over previous
baseline models in the online A/B test. Our further analyses illustrate that
SINGLE has the ability to build a more tailored recommendation system by
mimicking different article viewing behaviors of users and recommending more
appropriate and diverse articles to match user interests.Comment: 8 pages
- …