186 research outputs found
Event-Triggered Adaptive Fuzzy Finite-Time Output Feedback Control for Stochastic Nonlinear Systems With Input and Output Constraints
Fast Chain-of-Thought: A Glance of Future from Parallel Decoding Leads to Answers Faster
In this work, we propose FastCoT, a model-agnostic framework based on
parallel decoding without any further training of an auxiliary model or
modification to the LLM itself. FastCoT uses a size-varying context window
whose size changes with position to conduct parallel decoding and
auto-regressive decoding simultaneously, thus fully utilizing GPU computation
resources. In FastCoT, the parallel decoding part provides the LLM with a quick
glance of the future composed of approximate tokens, which could lead to faster
answers compared to regular autoregressive decoding used by causal
transformers. We also provide an implementation of parallel decoding within
LLM, which supports KV-cache generation and batch processing. Through extensive
experiments, we demonstrate that FastCoT saves inference time by nearly 20%
with only a negligible performance drop compared to the regular approach.
Additionally, we show that the context window size exhibits considerable
robustness for different tasks
FERN: Leveraging Graph Attention Networks for Failure Evaluation and Robust Network Design
Robust network design, which aims to guarantee network availability under
various failure scenarios while optimizing performance/cost objectives, has
received significant attention. Existing approaches often rely on model-based
mixed-integer optimization that is hard to scale or employ deep learning to
solve specific engineering problems yet with limited generalizability. In this
paper, we show that failure evaluation provides a common kernel to improve the
tractability and scalability of existing solutions. By providing a neural
network function approximation of this common kernel using graph attention
networks, we develop a unified learning-based framework, FERN, for scalable
Failure Evaluation and Robust Network design. FERN represents rich problem
inputs as a graph and captures both local and global views by attentively
performing feature extraction from the graph. It enables a broad range of
robust network design problems, including robust network validation, network
upgrade optimization, and fault-tolerant traffic engineering that are discussed
in this paper, to be recasted with respect to the common kernel and thus
computed efficiently using neural networks and over a small set of critical
failure scenarios. Extensive experiments on real-world network topologies show
that FERN can efficiently and accurately identify key failure scenarios for
both OSPF and optimal routing scheme, and generalizes well to different
topologies and input traffic patterns. It can speed up multiple robust network
design problems by more than 80x, 200x, 10x, respectively with negligible
performance gap
Pre-training Graph Transformer with Multimodal Side Information for Recommendation
Side information of items, e.g., images and text description, has shown to be
effective in contributing to accurate recommendations. Inspired by the recent
success of pre-training models on natural language and images, we propose a
pre-training strategy to learn item representations by considering both item
side information and their relationships. We relate items by common user
activities, e.g., co-purchase, and construct a homogeneous item graph. This
graph provides a unified view of item relations and their associated side
information in multimodality. We develop a novel sampling algorithm named
MCNSampling to select contextual neighbors for each item. The proposed
Pre-trained Multimodal Graph Transformer (PMGT) learns item representations
with two objectives: 1) graph structure reconstruction, and 2) masked node
feature reconstruction. Experimental results on real datasets demonstrate that
the proposed PMGT model effectively exploits the multimodality side information
to achieve better accuracies in downstream tasks including item recommendation,
item classification, and click-through ratio prediction. We also report a case
study of testing the proposed PMGT model in an online setting with 600 thousand
users
6DOF Pose Estimation of a 3D Rigid Object based on Edge-enhanced Point Pair Features
The point pair feature (PPF) is widely used for 6D pose estimation. In this
paper, we propose an efficient 6D pose estimation method based on the PPF
framework. We introduce a well-targeted down-sampling strategy that focuses
more on edge area for efficient feature extraction of complex geometry. A pose
hypothesis validation approach is proposed to resolve the symmetric ambiguity
by calculating edge matching degree. We perform evaluations on two challenging
datasets and one real-world collected dataset, demonstrating the superiority of
our method on pose estimation of geometrically complex, occluded, symmetrical
objects. We further validate our method by applying it to simulated punctures.Comment: 16 pages,20 figure
- …