5,162 research outputs found
The Roots of the Uncertainty in the Enterprise Tech-innovation Process under the Net-Environment
This paper applies evolutionary view to investigate the enterprise tech-innovation process and study the uncertainty of the routine, net environment and the innovational technique. In the end the measurement of risk is described as the uncertainty characteristic then induced the coherent characteristic between the risk and the uncertainty, which can be applied in the theory on the innovation management
A Quantitative Review on Language Model Efficiency Research
Language models (LMs) are being scaled and becoming powerful. Improving their
efficiency is one of the core research topics in neural information processing
systems. Tay et al. (2022) provided a comprehensive overview of efficient
Transformers that have become an indispensable staple in the field of NLP.
However, in the section of "On Evaluation", they left an open question "which
fundamental efficient Transformer one should consider," answered by "still a
mystery" because "many research papers select their own benchmarks."
Unfortunately, there was not quantitative analysis about the performances of
Transformers on any benchmarks. Moreover, state space models (SSMs) have
demonstrated their abilities of modeling long-range sequences with
non-attention mechanisms, which were not discussed in the prior review. This
article makes a meta analysis on the results from a set of papers on efficient
Transformers as well as those on SSMs. It provides a quantitative review on LM
efficiency research and gives suggestions for future research.Comment: 29 pages, 24 table
Embedding Mental Health Discourse for Community Recommendation
Our paper investigates the use of discourse embedding techniques to develop a
community recommendation system that focuses on mental health support groups on
social media. Social media platforms provide a means for users to anonymously
connect with communities that cater to their specific interests. However, with
the vast number of online communities available, users may face difficulties in
identifying relevant groups to address their mental health concerns. To address
this challenge, we explore the integration of discourse information from
various subreddit communities using embedding techniques to develop an
effective recommendation system. Our approach involves the use of content-based
and collaborative filtering techniques to enhance the performance of the
recommendation system. Our findings indicate that the proposed approach
outperforms the use of each technique separately and provides interpretability
in the recommendation process.Comment: Accepted to the 4th workshop on Computational Approaches to Discourse
(CODI-2023) at ACL 202
SE(3) Diffusion Model-based Point Cloud Registration for Robust 6D Object Pose Estimation
In this paper, we introduce an SE(3) diffusion model-based point cloud
registration framework for 6D object pose estimation in real-world scenarios.
Our approach formulates the 3D registration task as a denoising diffusion
process, which progressively refines the pose of the source point cloud to
obtain a precise alignment with the model point cloud. Training our framework
involves two operations: An SE(3) diffusion process and an SE(3) reverse
process. The SE(3) diffusion process gradually perturbs the optimal rigid
transformation of a pair of point clouds by continuously injecting noise
(perturbation transformation). By contrast, the SE(3) reverse process focuses
on learning a denoising network that refines the noisy transformation
step-by-step, bringing it closer to the optimal transformation for accurate
pose estimation. Unlike standard diffusion models used in linear Euclidean
spaces, our diffusion model operates on the SE(3) manifold. This requires
exploiting the linear Lie algebra associated with SE(3) to
constrain the transformation transitions during the diffusion and reverse
processes. Additionally, to effectively train our denoising network, we derive
a registration-specific variational lower bound as the optimization objective
for model learning. Furthermore, we show that our denoising network can be
constructed with a surrogate registration model, making our approach applicable
to different deep registration networks. Extensive experiments demonstrate that
our diffusion registration framework presents outstanding pose estimation
performance on the real-world TUD-L, LINEMOD, and Occluded-LINEMOD datasets.Comment: Accepted by NeurIPS-202
FILM: How can Few-Shot Image Classification Benefit from Pre-Trained Language Models?
Few-shot learning aims to train models that can be generalized to novel
classes with only a few samples. Recently, a line of works are proposed to
enhance few-shot learning with accessible semantic information from class
names. However, these works focus on improving existing modules such as visual
prototypes and feature extractors of the standard few-shot learning framework.
This limits the full potential use of semantic information. In this paper, we
propose a novel few-shot learning framework that uses pre-trained language
models based on contrastive learning. To address the challenge of alignment
between visual features and textual embeddings obtained from text-based
pre-trained language model, we carefully design the textual branch of our
framework and introduce a metric module to generalize the cosine similarity.
For better transferability, we let the metric module adapt to different
few-shot tasks and adopt MAML to train the model via bi-level optimization.
Moreover, we conduct extensive experiments on multiple benchmarks to
demonstrate the effectiveness of our method
- …