1,150 research outputs found
Differentiable Learning of Generalized Structured Matrices for Efficient Deep Neural Networks
This paper investigates efficient deep neural networks (DNNs) to replace
dense unstructured weight matrices with structured ones that possess desired
properties. The challenge arises because the optimal weight matrix structure in
popular neural network models is obscure in most cases and may vary from layer
to layer even in the same network. Prior structured matrices proposed for
efficient DNNs were mostly hand-crafted without a generalized framework to
systematically learn them. To address this issue, we propose a generalized and
differentiable framework to learn efficient structures of weight matrices by
gradient descent. We first define a new class of structured matrices that
covers a wide range of structured matrices in the literature by adjusting the
structural parameters. Then, the frequency-domain differentiable
parameterization scheme based on the Gaussian-Dirichlet kernel is adopted to
learn the structural parameters by proximal gradient descent. Finally, we
introduce an effective initialization method for the proposed scheme. Our
method learns efficient DNNs with structured matrices, achieving lower
complexity and/or higher performance than prior approaches that employ
low-rank, block-sparse, or block-low-rank matrices
Minimax particle filtering for tracking a highly maneuvering target
Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/152473/1/rnc4785_am.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/152473/2/rnc4785.pd
Autocorrelation and Spectrum Analysis for Variable Symbol Length Communications with Feedback
Variable-length feedback codes can provide advantages over fixed-length
feedback or non-feedback codes. This letter focuses on uncoded
variable-symbol-length feedback communication and analyzes the autocorrelation
and spectrum of the signal. We provide a mathematical expression for the
autocorrelation that can be evaluated numerically. We then numerically evaluate
the autocorrelation and spectrum for the variable-symbol-length signal in a
feedback-based communication system that attains a target reliability for every
symbol by adapting the symbol length to the noise realization. The analysis and
numerical results show that the spectrum changes with SNR when the average
symbol length is fixed, and approaches the fixed-length scheme at high SNR
Improving Prediction Quality in Collaborative Filtering Based on Clustering
In this paper we present the recommender systems that use the k-means clustering method in order to solve the problems associated with neighbor selection. The first method is to solve the problem in which customers belong to different clusters due to the distance-based characteristics despite the fact that they are similar customers, by properly converting data before performing clustering. The second method explains the k-prototype algorithm performing clus-tering by expanding not only the numeric data but also the categorical data. The experimental results show that better prediction quality can be obtained when both methods are used together. 1
Efficient Computation Sharing for Multi-Task Visual Scene Understanding
Solving multiple visual tasks using individual models can be
resource-intensive, while multi-task learning can conserve resources by sharing
knowledge across different tasks. Despite the benefits of multi-task learning,
such techniques can struggle with balancing the loss for each task, leading to
potential performance degradation. We present a novel computation- and
parameter-sharing framework that balances efficiency and accuracy to perform
multiple visual tasks utilizing individually-trained single-task transformers.
Our method is motivated by transfer learning schemes to reduce computational
and parameter storage costs while maintaining the desired performance. Our
approach involves splitting the tasks into a base task and the other sub-tasks,
and sharing a significant portion of activations and parameters/weights between
the base and sub-tasks to decrease inter-task redundancies and enhance
knowledge sharing. The evaluation conducted on NYUD-v2 and PASCAL-context
datasets shows that our method is superior to the state-of-the-art
transformer-based multi-task learning techniques with higher accuracy and
reduced computational resources. Moreover, our method is extended to video
stream inputs, further reducing computational costs by efficiently sharing
information across the temporal domain as well as the task domain. Our codes
and models will be publicly available.Comment: Camera-Ready version. Accepted to ICCV 202
- âŠ