648,933 research outputs found
Learnt Topology Gating Artificial Neural Networks
This work combines several established regression and meta-learning techniques to give a holistic regression model
and presents the proposed Learnt Topology Gating Artificial
Neural Networks (LTGANN) model in the context of a general
architecture previously published by the authors. The applied regression techniques are Artificial Neural Networks, which are on one hand used as local experts for the regression modelling and on the other hand as gating networks. The role of the gating networks is to estimate the prediction error of the local experts dependent on the input data samples. This is achieved by relating the input data space to the performance of the local experts, and thus building a performance map, for each of the local experts. The estimation of the prediction error is
then used for the weighting of the local experts predictions. Another advantage of our approach is that the particular neural networks are unconstrained in terms of the number of hidden units. It is only necessary to define the range within which the number of hidden units has to be generated. The model links the topology to the performance, which has been achieved by the network with the given complexity, using a probabilistic approach. As the model was developed in the context of process industry data, it is evaluated using two industrial data sets. The evaluation has shown a clear advantage when using a model combination and meta-learning approach as well as demonstrating the higher performance of LTGANN when compared to a standard combination method
Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment
We present a deep neural network-based approach to image quality assessment
(IQA). The network is trained end-to-end and comprises ten convolutional layers
and five pooling layers for feature extraction, and two fully connected layers
for regression, which makes it significantly deeper than related IQA models.
Unique features of the proposed architecture are that: 1) with slight
adaptations it can be used in a no-reference (NR) as well as in a
full-reference (FR) IQA setting and 2) it allows for joint learning of local
quality and local weights, i.e., relative importance of local quality to the
global quality estimate, in an unified framework. Our approach is purely
data-driven and does not rely on hand-crafted features or other types of prior
domain knowledge about the human visual system or image statistics. We evaluate
the proposed approach on the LIVE, CISQ, and TID2013 databases as well as the
LIVE In the wild image quality challenge database and show superior performance
to state-of-the-art NR and FR IQA methods. Finally, cross-database evaluation
shows a high ability to generalize between different databases, indicating a
high robustness of the learned features
FLCC: Efficient Distributed Federated Learning on IoMT over CSMA/CA
Federated Learning (FL) has emerged as a promising approach for privacy
preservation, allowing sharing of the model parameters between users and the
cloud server rather than the raw local data. FL approaches have been adopted as
a cornerstone of distributed machine learning (ML) to solve several complex use
cases. FL presents an interesting interplay between communication and ML
performance when implemented over distributed wireless nodes. Both the dynamics
of networking and learning play an important role. In this article, we
investigate the performance of FL on an application that might be used to
improve a remote healthcare system over ad hoc networks which employ CSMA/CA to
schedule its transmissions. Our FL over CSMA/CA (FLCC) model is designed to
eliminate untrusted devices and harness frequency reuse and spatial clustering
techniques to improve the throughput required for coordinating a distributed
implementation of FL in the wireless network.
In our proposed model, frequency allocation is performed on the basis of
spatial clustering performed using virtual cells. Each cell assigns a FL server
and dedicated carrier frequencies to exchange the updated model's parameters
within the cell. We present two metrics to evaluate the network performance: 1)
probability of successful transmission while minimizing the interference, and
2) performance of distributed FL model in terms of accuracy and loss while
considering the networking dynamics.
We benchmark the proposed approach using a well-known MNIST dataset for
performance evaluation. We demonstrate that the proposed approach outperforms
the baseline FL algorithms in terms of explicitly defining the chosen users'
criteria and achieving high accuracy in a robust network
Enhanced Graph Neural Networks with Ego-Centric Spectral Subgraph Embeddings Augmentation
Graph Neural Networks (GNNs) have shown remarkable merit in performing
various learning-based tasks in complex networks. The superior performance of
GNNs often correlates with the availability and quality of node-level features
in the input networks. However, for many network applications, such node-level
information may be missing or unreliable, thereby limiting the applicability
and efficacy of GNNs. To address this limitation, we present a novel approach
denoted as Ego-centric Spectral subGraph Embedding Augmentation (ESGEA), which
aims to enhance and design node features, particularly in scenarios where
information is lacking. Our method leverages the topological structure of the
local subgraph to create topology-aware node features. The subgraph features
are generated using an efficient spectral graph embedding technique, and they
serve as node features that capture the local topological organization of the
network. The explicit node features, if present, are then enhanced with the
subgraph embeddings in order to improve the overall performance. ESGEA is
compatible with any GNN-based architecture and is effective even in the absence
of node features. We evaluate the proposed method in a social network graph
classification task where node attributes are unavailable, as well as in a node
classification task where node features are corrupted or even absent. The
evaluation results on seven datasets and eight baseline models indicate up to a
10% improvement in AUC and a 7% improvement in accuracy for graph and node
classification tasks, respectively.Comment: 22nd IEEE International Conference on Machine Learning and
Applications 202
Deep Reinforcement Learning with Swin Transformers
Transformers are neural network models that utilize multiple layers of
self-attention heads and have exhibited enormous potential in natural language
processing tasks. Meanwhile, there have been efforts to adapt transformers to
visual tasks of machine learning, including Vision Transformers and Swin
Transformers. Although some researchers use Vision Transformers for
reinforcement learning tasks, their experiments remain at a small scale due to
the high computational cost. Experiments conducted at a large scale, on the
other hand, have to rely on techniques to cut the costs of Vision Transformers,
which also yield inferior results.
To address this challenge, this article presents the first online
reinforcement learning scheme that is based on Swin Transformers: Swin DQN.
Swin Transformers are promising as a backbone in neural networks by splitting
groups of image pixels into small patches and applying local self-attention
operations inside the (shifted) windows of fixed sizes. They have demonstrated
state-of-the-art performances in benchmarks. In contrast to existing research,
our novel approach is reducing the computational costs, as well as
significantly improving the performance. We demonstrate the superior
performance with experiments on 49 games in the Arcade Learning Environment.
The results show that our approach, using Swin Transformers with Double DQN,
achieves significantly higher maximal evaluation scores than the baseline
method in 45 of all the 49 games ~92%, and higher mean evaluation scores than
the baseline method in 40 of all the 49 games ~82%
Panoptic Out-of-Distribution Segmentation
Deep learning has led to remarkable strides in scene understanding with
panoptic segmentation emerging as a key holistic scene interpretation task.
However, the performance of panoptic segmentation is severely impacted in the
presence of out-of-distribution (OOD) objects i.e. categories of objects that
deviate from the training distribution. To overcome this limitation, we propose
Panoptic Out-of Distribution Segmentation for joint pixel-level semantic
in-distribution and out-of-distribution classification with instance
prediction. We extend two established panoptic segmentation benchmarks,
Cityscapes and BDD100K, with out-of-distribution instance segmentation
annotations, propose suitable evaluation metrics, and present multiple strong
baselines. Importantly, we propose the novel PoDS architecture with a shared
backbone, an OOD contextual module for learning global and local OOD object
cues, and dual symmetrical decoders with task-specific heads that employ our
alignment-mismatch strategy for better OOD generalization. Combined with our
data augmentation strategy, this approach facilitates progressive learning of
out-of-distribution objects while maintaining in-distribution performance. We
perform extensive evaluations that demonstrate that our proposed PoDS network
effectively addresses the main challenges and substantially outperforms the
baselines. We make the dataset, code, and trained models publicly available at
http://pods.cs.uni-freiburg.de
- …