75,783 research outputs found
NFI: a neuro-fuzzy inference method for transductive reasoning
This paper introduces a novel neural fuzzy inference method - NFI for transductive reasoning systems. NFI develops further some ideas from DENFIS - dynamic neuro-fuzzy inference systems for both online and offline time series prediction tasks. While inductive reasoning is concerned with the development of a model (a function) to approximate data in the whole problem space (induction), and consecutively - using this model to predict output values for a new input vector (deduction), in transductive reasoning systems a local model is developed for every new input vector, based on some closest to this vector data from an existing database (also generated from an existing model). NFI is compared with both inductive connectionist systems (e.g., MLP, DENFIS) and transductive reasoning systems (e.g., K-NN) on three case study prediction/identification problems. The first one is a prediction task on Mackey Glass time series; the second one is a classification on Iris data; and the last one is a real medical decision support problem of estimating the level of renal function of a patient, based on measured clinical parameters for the purpose of their personalised treatment. The case studies have demonstrated better accuracy obtained with the use of the NFI transductive reasoning in comparison with the inductive reasoning systems. © 2005 IEEE
Transductive Learning for Textual Few-Shot Classification in API-based Embedding Models
Proprietary and closed APIs are becoming increasingly common to process
natural language, and are impacting the practical applications of natural
language processing, including few-shot classification. Few-shot classification
involves training a model to perform a new classification task with a handful
of labeled data. This paper presents three contributions. First, we introduce a
scenario where the embedding of a pre-trained model is served through a gated
API with compute-cost and data-privacy constraints. Second, we propose a
transductive inference, a learning paradigm that has been overlooked by the NLP
community. Transductive inference, unlike traditional inductive learning,
leverages the statistics of unlabeled data. We also introduce a new
parameter-free transductive regularizer based on the Fisher-Rao loss, which can
be used on top of the gated API embeddings. This method fully utilizes
unlabeled data, does not share any label with the third-party API provider and
could serve as a baseline for future research. Third, we propose an improved
experimental setting and compile a benchmark of eight datasets involving
multiclass classification in four different languages, with up to 151 classes.
We evaluate our methods using eight backbone models, along with an episodic
evaluation over 1,000 episodes, which demonstrate the superiority of
transductive inference over the standard inductive setting.Comment: EMNLP 202
Auto-Encoding Scene Graphs for Image Captioning
We propose Scene Graph Auto-Encoder (SGAE) that incorporates the language
inductive bias into the encoder-decoder image captioning framework for more
human-like captions. Intuitively, we humans use the inductive bias to compose
collocations and contextual inference in discourse. For example, when we see
the relation `person on bike', it is natural to replace `on' with `ride' and
infer `person riding bike on a road' even the `road' is not evident. Therefore,
exploiting such bias as a language prior is expected to help the conventional
encoder-decoder models less likely overfit to the dataset bias and focus on
reasoning. Specifically, we use the scene graph --- a directed graph
() where an object node is connected by adjective nodes and
relationship nodes --- to represent the complex structural layout of both image
() and sentence (). In the textual domain, we use
SGAE to learn a dictionary () that helps to reconstruct sentences
in the pipeline, where encodes the desired language prior;
in the vision-language domain, we use the shared to guide the
encoder-decoder in the pipeline. Thanks to the scene graph
representation and shared dictionary, the inductive bias is transferred across
domains in principle. We validate the effectiveness of SGAE on the challenging
MS-COCO image captioning benchmark, e.g., our SGAE-based single-model achieves
a new state-of-the-art CIDEr-D on the Karpathy split, and a competitive
CIDEr-D (c40) on the official server even compared to other ensemble
models
Changepoint detection versus reinforcement learning: Separable neural substrates approximate different forms of Bayesian inference
Adaptive behavior in even the simplest decision-making tasks requires predicting future events in an environment that is generally nonstationary. As an inductive problem, this prediction requires a commitment to the statistical process underlying environmental change. This challenge can be formalized in a Bayesian framework as a question of choosing a generative model for the task dynamics. Previous learning models assume, implicitly or explicitly, that nonstationarity follows either a continuous diffusion process or a discrete changepoint process. Each approach is slow to adapt when its assumptions are violated. A new mixture of Bayesian experts framework proposes separable brain systems approximating inference under different assumptions regarding the statistical structure of the environment. This model explains data from a laboratory foraging task, in which rats experienced a change in reward contingencies after pharmacological disruption of dorsolateral (DLS) or dorsomedial striatum (DMS). The data and model suggest DLS learns under a diffusion prior whereas DMS learns under a changepoint prior. The combination of these two systems offers a new explanation for how the brain handles inference in an uncertain environment
Sampling-free Inference for Ab-Initio Potential Energy Surface Networks
Recently, it has been shown that neural networks not only approximate the
ground-state wave functions of a single molecular system well but can also
generalize to multiple geometries. While such generalization significantly
speeds up training, each energy evaluation still requires Monte Carlo
integration which limits the evaluation to a few geometries. In this work, we
address the inference shortcomings by proposing the Potential learning from
ab-initio Networks (PlaNet) framework, in which we simultaneously train a
surrogate model in addition to the neural wave function. At inference time, the
surrogate avoids expensive Monte-Carlo integration by directly estimating the
energy, accelerating the process from hours to milliseconds. In this way, we
can accurately model high-resolution multi-dimensional energy surfaces for
larger systems that previously were unobtainable via neural wave functions.
Finally, we explore an additional inductive bias by introducing
physically-motivated restricted neural wave function models. We implement such
a function with several additional improvements in the new PESNet++ model. In
our experimental evaluation, PlaNet accelerates inference by 7 orders of
magnitude for larger molecules like ethanol while preserving accuracy. Compared
to previous energy surface networks, PESNet++ reduces energy errors by up to
74%
The Emergence of Organizing Structure in Conceptual Representation.
Both scientists and children make important structural discoveries, yet their computational underpinnings are not well understood. Structure discovery has previously been formalized as probabilistic inference about the right structural form-where form could be a tree, ring, chain, grid, etc. (Kemp & Tenenbaum, 2008). Although this approach can learn intuitive organizations, including a tree for animals and a ring for the color circle, it assumes a strong inductive bias that considers only these particular forms, and each form is explicitly provided as initial knowledge. Here we introduce a new computational model of how organizing structure can be discovered, utilizing a broad hypothesis space with a preference for sparse connectivity. Given that the inductive bias is more general, the model's initial knowledge shows little qualitative resemblance to some of the discoveries it supports. As a consequence, the model can also learn complex structures for domains that lack intuitive description, as well as predict human property induction judgments without explicit structural forms. By allowing form to emerge from sparsity, our approach clarifies how both the richness and flexibility of human conceptual organization can coexist
- …