262 research outputs found
Probabilistic Inference from Arbitrary Uncertainty using Mixtures of Factorized Generalized Gaussians
This paper presents a general and efficient framework for probabilistic
inference and learning from arbitrary uncertain information. It exploits the
calculation properties of finite mixture models, conjugate families and
factorization. Both the joint probability density of the variables and the
likelihood function of the (objective or subjective) observation are
approximated by a special mixture model, in such a way that any desired
conditional distribution can be directly obtained without numerical
integration. We have developed an extended version of the expectation
maximization (EM) algorithm to estimate the parameters of mixture models from
uncertain training examples (indirect observations). As a consequence, any
piece of exact or uncertain information about both input and output values is
consistently handled in the inference and learning stages. This ability,
extremely useful in certain situations, is not found in most alternative
methods. The proposed framework is formally justified from standard
probabilistic principles and illustrative examples are provided in the fields
of nonparametric pattern classification, nonlinear regression and pattern
completion. Finally, experiments on a real application and comparative results
over standard databases provide empirical evidence of the utility of the method
in a wide range of applications
Simple low cost causal discovery using mutual information and domain knowledge
PhDThis thesis examines causal discovery within datasets, in particular observational datasets where
normal experimental manipulation is not possible. A number of machine learning techniques
are examined in relation to their use of knowledge and the insights they can provide regarding
the situation under study. Their use of prior knowledge and the causal knowledge produced by
the learners are examined. Current causal learning algorithms are discussed in terms of their
strengths and limitations. The main contribution of the thesis is a new causal learner LUMIN
that operates with a polynomial time complexity in both the number of variables and records
examined. It makes no prior assumptions about the form of the relationships and is capable of
making extensive use of available domain information. This learner is compared to a number of
current learning algorithms and it is shown to be competitive with them
Principles of Human Learning
What are the general principles that drive human learning in different situations? I argue that much of human learning can be understood with just three principles. These are generalization, adaptation, and simplicity. To verify this conjecture, I introduce a modeling framework based on the same principles. This framework combines the idea of meta-learning -- also known as learning-to-learn -- with the minimum description length principle. The models that result from this framework capture many aspects of human learning across different domains, including decision-making, associative learning, function learning, multi-task learning, and reinforcement learning. In the context of decision-making, they explain why different heuristic decision-making strategies emerge and how appropriate strategies are selected. The same models furthermore capture order effects found in associative learning, function learning and multi-task learning. In the reinforcement learning context, they resemble individual differences between human exploration strategies and explain empirical data better than any other strategy under consideration. The proposed modeling framework -- together with its accompanying empirical evidence -- may therefore be viewed as a first step towards the identification of a minimal set of principles from which all human behavior derives
Hypothesis Testing with Classifier Systems
This thesis presents a new ML algorithm, HCS, taking
inspiration from Learning Classifier Systems, Decision Trees and
Statistical Hypothesis Testing, aimed at providing clearly
understandable models of medical datasets. Analysis of medical
datasets has some specific requirements not always fulfilled by
standard Machine Learning methods. In particular, heterogeneous
and missing data must be tolerated, the results should be easily
interpretable. Moreover, often the combination of two or more
attributes leads to non-linear effects not detectable for each
attribute on its own. Although it has been designed specifically
for medical datasets, HCS can be applied to a broad range of
data types, making it suitable for many domains. We describe the
details of the algorithm, and test its effectiveness on five
real-world datasets
Decision making under uncertainty
Almost all important decision problems are inevitably subject to some level of uncertainty either about data measurements, the parameters, or predictions describing future evolution. The significance of handling uncertainty is further amplified by the large volume of uncertain data automatically generated by modern data gathering or integration systems. Various types of problems of decision making under uncertainty have been subject to extensive research in computer science, economics and social science. In this dissertation, I study three major problems in this context, ranking, utility maximization, and matching, all involving uncertain datasets.
First, we consider the problem of ranking and top-k query processing over probabilistic datasets. By illustrating the diverse and conflicting behaviors of the prior proposals, we contend that a single, specific ranking function may not suffice for probabilistic datasets. Instead we propose the notion of parameterized ranking functions, that generalize or can approximate many of the previously proposed ranking functions. We present novel exact or approximate algorithms for efficiently ranking large datasets according to these ranking functions, even if the datasets exhibit complex correlations or the probability distributions are continuous.
The second problem concerns with the stochastic versions of a broad class of combinatorial optimization problems. We observe that the expected value is inadequate in capturing different types of risk-averse or risk-prone behaviors, and instead we consider a more general objective which is to maximize the expected utility of the solution for some given utility function. We present a polynomial time approximation algorithm with additive error ε for any ε > 0, under certain conditions. Our result generalizes and improves several prior results on stochastic shortest path, stochastic spanning tree, and stochastic knapsack.
The third is the stochastic matching problem which finds interesting applications in online dating, kidney exchange and online ad assignment. In this problem, the existence of each edge is uncertain and can be only found out by probing the edge. The goal is to design a probing strategy to maximize the expected weight of the matching. We give linear programming based constant-factor approximation algorithms for weighted stochastic matching, which answer an open question raised in prior work
Functional inferences over heterogeneous data
Inference enables an agent to create new knowledge from old or discover implicit
relationships between concepts in a knowledge base (KB), provided that appropriate
techniques are employed to deal with ambiguous, incomplete and sometimes erroneous
data.
The ever-increasing volumes of KBs on the web, available for use by automated
systems, present an opportunity to leverage the available knowledge in order to improve
the inference process in automated query answering systems. This thesis focuses
on the FRANK (Functional Reasoning for Acquiring Novel Knowledge) framework
that responds to queries where no suitable answer is readily contained in any available
data source, using a variety of inference operations.
Most question answering and information retrieval systems assume that answers
to queries are stored in some form in the KB, thereby limiting the range of answers
they can find. We take an approach motivated by rich forms of inference using techniques,
such as regression, for prediction. For instance, FRANK can answer “what
country in Europe will have the largest population in 2021?" by decomposing Europe
geo-spatially, using regression on country population for past years and selecting the
country with the largest predicted value. Our technique, which we refer to as Rich
Inference, combines heuristics, logic and statistical methods to infer novel answers
to queries. It also determines what facts are needed for inference, searches for them,
and then integrates the diverse facts and their formalisms into a local query-specific
inference tree.
Our primary contribution in this thesis is the inference algorithm on which FRANK
works. This includes (1) the process of recursively decomposing queries in way that
allows variables in the query to be instantiated by facts in KBs; (2) the use of aggregate
functions to perform arithmetic and statistical operations (e.g. prediction) to infer new
values from child nodes; and (3) the estimation and propagation of uncertainty values
into the returned answer based on errors introduced by noise in the KBs or errors
introduced by aggregate functions.
We also discuss many of the core concepts and modules that constitute FRANK.
We explain the internal “alist” representation of FRANK that gives it the required
flexibility to tackle different kinds of problems with minimal changes to its internal
representation. We discuss the grammar for a simple query language that allows users
to express queries in a formal way, such that we avoid the complexities of natural
language queries, a problem that falls outside the scope of this thesis. We evaluate the
framework with datasets from open sources
A Hybrid Environment for Syntax-Semantic Tagging
The thesis describes the application of the relaxation labelling algorithm to
NLP disambiguation. Language is modelled through context constraint inspired on
Constraint Grammars. The constraints enable the use of a real value statind
"compatibility". The technique is applied to POS tagging, Shallow Parsing and
Word Sense Disambigation. Experiments and results are reported. The proposed
approach enables the use of multi-feature constraint models, the simultaneous
resolution of several NL disambiguation tasks, and the collaboration of
linguistic and statistical models.Comment: PhD Thesis. 120 page
- …