65,595 research outputs found
Unified Probabilistic Neural Architecture and Weight Ensembling Improves Model Robustness
Robust machine learning models with accurately calibrated uncertainties are
crucial for safety-critical applications. Probabilistic machine learning and
especially the Bayesian formalism provide a systematic framework to incorporate
robustness through the distributional estimates and reason about uncertainty.
Recent works have shown that approximate inference approaches that take the
weight space uncertainty of neural networks to generate ensemble prediction are
the state-of-the-art. However, architecture choices have mostly been ad hoc,
which essentially ignores the epistemic uncertainty from the architecture
space. To this end, we propose a Unified probabilistic architecture and weight
ensembling Neural Architecture Search (UraeNAS) that leverages advances in
probabilistic neural architecture search and approximate Bayesian inference to
generate ensembles form the joint distribution of neural network architectures
and weights. The proposed approach showed a significant improvement both with
in-distribution (0.86% in accuracy, 42% in ECE) CIFAR-10 and
out-of-distribution (2.43% in accuracy, 30% in ECE) CIFAR-10-C compared to the
baseline deterministic approach
GraphPNAS: Learning Distribution of Good Neural Architectures via Deep Graph Generative Models
Neural architectures can be naturally viewed as computational graphs.
Motivated by this perspective, we, in this paper, study neural architecture
search (NAS) through the lens of learning random graph models. In contrast to
existing NAS methods which largely focus on searching for a single best
architecture, i.e, point estimation, we propose GraphPNAS a deep graph
generative model that learns a distribution of well-performing architectures.
Relying on graph neural networks (GNNs), our GraphPNAS can better capture
topologies of good neural architectures and relations between operators
therein. Moreover, our graph generator leads to a learnable probabilistic
search method that is more flexible and efficient than the commonly used RNN
generator and random search methods. Finally, we learn our generator via an
efficient reinforcement learning formulation for NAS. To assess the
effectiveness of our GraphPNAS, we conduct extensive experiments on three
search spaces, including the challenging RandWire on TinyImageNet, ENAS on
CIFAR10, and NAS-Bench-101/201. The complexity of RandWire is significantly
larger than other search spaces in the literature. We show that our proposed
graph generator consistently outperforms RNN-based one and achieves better or
comparable performances than state-of-the-art NAS methods
Variational Deep Semantic Hashing for Text Documents
As the amount of textual data has been rapidly increasing over the past
decade, efficient similarity search methods have become a crucial component of
large-scale information retrieval systems. A popular strategy is to represent
original data samples by compact binary codes through hashing. A spectrum of
machine learning methods have been utilized, but they often lack expressiveness
and flexibility in modeling to learn effective representations. The recent
advances of deep learning in a wide range of applications has demonstrated its
capability to learn robust and powerful feature representations for complex
data. Especially, deep generative models naturally combine the expressiveness
of probabilistic generative models with the high capacity of deep neural
networks, which is very suitable for text modeling. However, little work has
leveraged the recent progress in deep learning for text hashing.
In this paper, we propose a series of novel deep document generative models
for text hashing. The first proposed model is unsupervised while the second one
is supervised by utilizing document labels/tags for hashing. The third model
further considers document-specific factors that affect the generation of
words. The probabilistic generative formulation of the proposed models provides
a principled framework for model extension, uncertainty estimation, simulation,
and interpretability. Based on variational inference and reparameterization,
the proposed models can be interpreted as encoder-decoder deep neural networks
and thus they are capable of learning complex nonlinear distributed
representations of the original documents. We conduct a comprehensive set of
experiments on four public testbeds. The experimental results have demonstrated
the effectiveness of the proposed supervised learning models for text hashing.Comment: 11 pages, 4 figure
An agent-driven semantical identifier using radial basis neural networks and reinforcement learning
Due to the huge availability of documents in digital form, and the deception
possibility raise bound to the essence of digital documents and the way they
are spread, the authorship attribution problem has constantly increased its
relevance. Nowadays, authorship attribution,for both information retrieval and
analysis, has gained great importance in the context of security, trust and
copyright preservation. This work proposes an innovative multi-agent driven
machine learning technique that has been developed for authorship attribution.
By means of a preprocessing for word-grouping and time-period related analysis
of the common lexicon, we determine a bias reference level for the recurrence
frequency of the words within analysed texts, and then train a Radial Basis
Neural Networks (RBPNN)-based classifier to identify the correct author. The
main advantage of the proposed approach lies in the generality of the semantic
analysis, which can be applied to different contexts and lexical domains,
without requiring any modification. Moreover, the proposed system is able to
incorporate an external input, meant to tune the classifier, and then
self-adjust by means of continuous learning reinforcement.Comment: Published on: Proceedings of the XV Workshop "Dagli Oggetti agli
Agenti" (WOA 2014), Catania, Italy, Sepember. 25-26, 201
- …