32 research outputs found
FRMDN: Flow-based Recurrent Mixture Density Network
Recurrent Mixture Density Networks (RMDNs) are consisted of two main parts: a
Recurrent Neural Network (RNN) and a Gaussian Mixture Model (GMM), in which a
kind of RNN (almost LSTM) is used to find the parameters of a GMM in every time
step. While available RMDNs have been faced with different difficulties. The
most important of them is highdimensional problems. Since estimating the
covariance matrix for the highdimensional problems is more difficult, due to
existing correlation between dimensions and satisfying the positive definition
condition. Consequently, the available methods have usually used RMDN with a
diagonal covariance matrix for highdimensional problems by supposing
independence among dimensions. Hence, in this paper with inspiring a common
approach in the literature of GMM, we consider a tied configuration for each
precision matrix (inverse of the covariance matrix) in RMDN as (\(\Sigma _k^{
- 1} = U{D_k}U\)) to enrich GMM rather than considering a diagonal form for
it. But due to simplicity, we assume \(U\) be an Identity matrix and
\(D_k\) is a specific diagonal matrix for \(k^{th}\) component. Until now,
we only have a diagonal matrix and it does not differ with available diagonal
RMDNs. Besides, Flowbased neural networks are a new group of generative
models that are able to transform a distribution to a simpler distribution and
vice versa, through a sequence of invertible functions. Therefore, we applied a
diagonal GMM on transformed observations. At every time step, the next
observation, \({y_{t + 1}}\), has been passed through a flowbased neural
network to obtain a much simpler distribution. Experimental results for a
reinforcement learning problem verify the superiority of the proposed method to
the baseline method in terms of Negative LogLikelihood (NLL) for RMDN and
the cumulative reward for a controller with fewer population size
Efficient Relation-aware Neighborhood Aggregation in Graph Neural Networks via Tensor Decomposition
Many Graph Neural Networks (GNNs) are proposed for Knowledge Graph Embedding
(KGE). However, lots of these methods neglect the importance of the information
of relations and combine it with the information of entities inefficiently,
leading to low expressiveness. To address this issue, we introduce a general
knowledge graph encoder incorporating tensor decomposition in the aggregation
function of Relational Graph Convolutional Network (R-GCN). In our model,
neighbor entities are transformed using projection matrices of a low-rank
tensor which are defined by relation types to benefit from multi-task learning
and produce expressive relation-aware representations. Besides, we propose a
low-rank estimation of the core tensor using CP decomposition to compress and
regularize our model. We use a training method inspired by contrastive
learning, which relieves the training limitation of the 1-N method on huge
graphs. We achieve favorably competitive results on FB15k-237 and WN18RR with
embeddings in comparably lower dimensions.Comment: 13 pages, 5 Tables, 2 Figure