64 research outputs found
Hypernetwork approach to Bayesian MAML
The main goal of Few-Shot learning algorithms is to enable learning from
small amounts of data. One of the most popular and elegant Few-Shot learning
approaches is Model-Agnostic Meta-Learning (MAML). The main idea behind this
method is to learn the shared universal weights of a meta-model, which are then
adapted for specific tasks. However, the method suffers from over-fitting and
poorly quantifies uncertainty due to limited data size. Bayesian approaches
could, in principle, alleviate these shortcomings by learning weight
distributions in place of point-wise weights. Unfortunately, previous
modifications of MAML are limited due to the simplicity of Gaussian posteriors,
MAML-like gradient-based weight updates, or by the same structure enforced for
universal and adapted weights.
In this paper, we propose a novel framework for Bayesian MAML called
BayesianHMAML, which employs Hypernetworks for weight updates. It learns the
universal weights point-wise, but a probabilistic structure is added when
adapted for specific tasks. In such a framework, we can use simple Gaussian
distributions or more complicated posteriors induced by Continuous Normalizing
Flows.Comment: arXiv admin note: text overlap with arXiv:2205.1574
Scalable Bayesian Meta-Learning through Generalized Implicit Gradients
Meta-learning owns unique effectiveness and swiftness in tackling emerging
tasks with limited data. Its broad applicability is revealed by viewing it as a
bi-level optimization problem. The resultant algorithmic viewpoint however,
faces scalability issues when the inner-level optimization relies on
gradient-based iterations. Implicit differentiation has been considered to
alleviate this challenge, but it is restricted to an isotropic Gaussian prior,
and only favors deterministic meta-learning approaches. This work markedly
mitigates the scalability bottleneck by cross-fertilizing the benefits of
implicit differentiation to probabilistic Bayesian meta-learning. The novel
implicit Bayesian meta-learning (iBaML) method not only broadens the scope of
learnable priors, but also quantifies the associated uncertainty. Furthermore,
the ultimate complexity is well controlled regardless of the inner-level
optimization trajectory. Analytical error bounds are established to demonstrate
the precision and efficiency of the generalized implicit gradient over the
explicit one. Extensive numerical tests are also carried out to empirically
validate the performance of the proposed method.Comment: Accepted as a poster paper in the main track of Proceedings of the
37th AAAI Conference on Artificial Intelligence (AAAI-23
Explain what you see:argumentation-based learning and robotic vision
In this thesis, we have introduced new techniques for the problems of open-ended learning, online incremental learning, and explainable learning. These methods have applications in the classification of tabular data, 3D object category recognition, and 3D object parts segmentation. We have utilized argumentation theory and probability theory to develop these methods. The first proposed open-ended online incremental learning approach is Argumentation-Based online incremental Learning (ABL). ABL works with tabular data and can learn with a small number of learning instances using an abstract argumentation framework and bipolar argumentation framework. It has a higher learning speed than state-of-the-art online incremental techniques. However, it has high computational complexity. We have addressed this problem by introducing Accelerated Argumentation-Based Learning (AABL). AABL uses only an abstract argumentation framework and uses two strategies to accelerate the learning process and reduce the complexity. The second proposed open-ended online incremental learning approach is the Local Hierarchical Dirichlet Process (Local-HDP). Local-HDP aims at addressing two problems of open-ended category recognition of 3D objects and segmenting 3D object parts. We have utilized Local-HDP for the task of object part segmentation in combination with AABL to achieve an interpretable model to explain why a certain 3D object belongs to a certain category. The explanations of this model tell a user that a certain object has specific object parts that look like a set of the typical parts of certain categories. Moreover, integrating AABL and Local-HDP leads to a model that can handle a high degree of occlusion
Arguments, rules and cases in law: Resources for aligning learning and reasoning in structured domains
This paper provides a formal description of two legal domains. In addition, we describe the generation of various artificial datasets from these domains and explain the use of these datasets in previous experiments aligning learning and reasoning. These resources are made available for the further investigation of connections between arguments, cases and rules. The datasets are publicly available at https://github.com/CorSteging/LegalResource
ATHENA Research Book, Volume 2
ATHENA European University is an association of nine higher education institutions with the mission of promoting excellence in research and innovation by enabling international cooperation. The acronym ATHENA stands for Association of Advanced Technologies in Higher Education. Partner institutions are from France, Germany, Greece, Italy, Lithuania, Portugal and Slovenia: University of Orléans, University of Siegen, Hellenic Mediterranean University, Niccolò Cusano University, Vilnius Gediminas Technical University, Polytechnic Institute of Porto and University of Maribor. In 2022, two institutions joined the alliance: the Maria Curie-Skłodowska University from Poland and the University of Vigo from Spain. Also in 2022, an institution from Austria joined the alliance as an associate member: Carinthia University of Applied Sciences. This research book presents a selection of the research activities of ATHENA University's partners. It contains an overview of the research activities of individual members, a selection of the most important bibliographic works of members, peer-reviewed student theses, a descriptive list of ATHENA lectures and reports from individual working sections of the ATHENA project. The ATHENA Research Book provides a platform that encourages collaborative and interdisciplinary research projects by advanced and early career researchers
Arguments, rules and cases in law: Resources for aligning learning and reasoning in structured domains
This paper provides a formal description of two legal domains. In addition, we describe the generation of various artificial datasets from these domains and explain the use of these datasets in previous experiments aligning learning and reasoning. These resources are made available for the further investigation of connections between arguments, cases and rules. The datasets are publicly available at https://github.com/CorSteging/LegalResources.</jats:p
Feature construction using explanations of individual predictions
Feature construction can contribute to comprehensibility and performance of
machine learning models. Unfortunately, it usually requires exhaustive search
in the attribute space or time-consuming human involvement to generate
meaningful features. We propose a novel heuristic approach for reducing the
search space based on aggregation of instance-based explanations of predictive
models. The proposed Explainable Feature Construction (EFC) methodology
identifies groups of co-occurring attributes exposed by popular explanation
methods, such as IME and SHAP. We empirically show that reducing the search to
these groups significantly reduces the time of feature construction using
logical, relational, Cartesian, numerical, and threshold num-of-N and X-of-N
constructive operators. An analysis on 10 transparent synthetic datasets shows
that EFC effectively identifies informative groups of attributes and constructs
relevant features. Using 30 real-world classification datasets, we show
significant improvements in classification accuracy for several classifiers and
demonstrate the feasibility of the proposed feature construction even for large
datasets. Finally, EFC generated interpretable features on a real-world problem
from the financial industry, which were confirmed by a domain expert.Comment: 54 pages, 10 figures, 22 table
Argue to Learn:Accelerated Argumentation-Based Learning
Human agents can acquire knowledge and learn through argumentation. Inspired by this fact, we propose a novel argumentation-based machine learning technique that can be used for online incremental learning scenarios. Existing methods for online incremental learning problems typically do not generalize well from just a few learning instances. Our previous argumentation-based online incremental learning method outperformed state-of-the-art methods in terms of accuracy and learning speed. However, it was neither memory-efficient nor computationally efficient since the algorithm used the power set of the feature values for updating the model. In this paper, we propose an accelerated version of the algorithm, with polynomial instead of exponential complexity, while achieving higher learning accuracy. The proposed method is at least 200 times faster than the original argumentation-based learning method and is more memory-efficient
Evolution, environmental distribution, and engineering of the abyssomicin biosynthetic gene cluster
Ph. D. Thesis.Evolution, environmental distribution, and engineering of the abyssomicin biosynthetic gene
cluster.
Microbial secondary metabolites constitute a great source of pharmaceutically interesting
biomolecules. In particular, the tetronate family of natural products is a structurally and
functionally diverse group of secondary metabolites whose potent bioactivities make them
attractive targets for clinical and industrial exploitation. The abyssomicins are an actively
growing family of small spirotetronate natural products that has been widely studied due to the
unique structural features and bioactivities that some of its members exhibit, including
antimicrobial activity against Gram-positive bacteria such as methicillin- and vancomycinresistant Staphylococcus aureus and different Mycobacteria strains, HIV inhibitory and
reactivator properties and anti-influenza A virus activity.
Abyssomicin C and its atrop- isomer, produced by the slow growing marine Actinobacteria
Micromonospora maris AB-18-032 T
, are type I polyketide antibiotics that inhibit the formation
of p-aminobenzoic acid, a constituent of the folate pathway. Abyssomicin biosynthesis is highly
amenable to reengineering, as the enzymes involved in the synthesis of the tetronate (AbyA1)
and the spiro-tetronate-forming Diels-Alderase (AbyU) are both capable of accepting
structurally diverse substrates. The aim of this project was to set up the grounds for the
discovery and production of novel abyssomicins with applications in the biopharmaceutical
industry. First, in order to understand the environmental distribution and evolution of the
abyssomicin biosynthetic gene clusters (BGCs) present in nature, an analysis of publicly
available genomic and metagenomic data was carried out. The strategy of selecting a pathwayspecific enzyme to direct the mining proved to be an excellent strategy; 74 new Diels–Alderase
homologs were identified and a surprising prevalence of the abyssomicin BGC within terrestrial
habitats, mainly soil and plant-associated, was unveiled. Five complete and 12 partial new
abyssomicin BGCs and 23 new potential abyssomicin BGCs were also identified, suggesting
that a plethora of abyssomicins remain to be discovered. A preliminary study on the
abyssomicin production potential of five of the strains containing potential abyssomicin BGCs
was also carried out although no abyssomicins were found.
After that, with the final goal of producing abyssomicins of various lengths and different
saturation/oxidation patterns, it was necessary to express the aby BGC of M. maris AB-18-032
in a well-established heterologous host. This cluster was successfully moved into E. coli and
various Streptomyces species, the abyssomicin production potential of these strains was
evaluated in various conditions and some of the hosts were promoter engineered to force the
expression of the aby BGC. Active gene expression was demonstrated, but despite the efforts,
none of the heterologous hosts produced abyssomicins. Later analysis unveiled the presence of
several mutations within abyB1, the first polyketide synthase gene in the aby BGC, suggesting
this could be the reason for the lack of production.
Since the approach to heterologously produce abyssomicins was not fruitful, this work then
focused on increasing abyssomicin production in M. maris AB-18-032 and developing genetic
tools for this system. First, through ribosome engineering, a library of M. maris drug-resistant
mutants capable of producing up to 3.4-fold abyssomicin C in comparison to the wild-type
strain was generated. Then, using statistical Design of Experiments (DOE), an efficient
electroporation protocol that could accelerate targeted genetic manipulations in M. maris was
developed. Together, increased abyssomicin production and a quick and easy electroporation
protocol for M. maris, will facilitate future engineering of the aby BGC directly in M. maris to
produce diverse non-natural abyssomicin
- …