64 research outputs found

    Hypernetwork approach to Bayesian MAML

    Full text link
    The main goal of Few-Shot learning algorithms is to enable learning from small amounts of data. One of the most popular and elegant Few-Shot learning approaches is Model-Agnostic Meta-Learning (MAML). The main idea behind this method is to learn the shared universal weights of a meta-model, which are then adapted for specific tasks. However, the method suffers from over-fitting and poorly quantifies uncertainty due to limited data size. Bayesian approaches could, in principle, alleviate these shortcomings by learning weight distributions in place of point-wise weights. Unfortunately, previous modifications of MAML are limited due to the simplicity of Gaussian posteriors, MAML-like gradient-based weight updates, or by the same structure enforced for universal and adapted weights. In this paper, we propose a novel framework for Bayesian MAML called BayesianHMAML, which employs Hypernetworks for weight updates. It learns the universal weights point-wise, but a probabilistic structure is added when adapted for specific tasks. In such a framework, we can use simple Gaussian distributions or more complicated posteriors induced by Continuous Normalizing Flows.Comment: arXiv admin note: text overlap with arXiv:2205.1574

    Scalable Bayesian Meta-Learning through Generalized Implicit Gradients

    Full text link
    Meta-learning owns unique effectiveness and swiftness in tackling emerging tasks with limited data. Its broad applicability is revealed by viewing it as a bi-level optimization problem. The resultant algorithmic viewpoint however, faces scalability issues when the inner-level optimization relies on gradient-based iterations. Implicit differentiation has been considered to alleviate this challenge, but it is restricted to an isotropic Gaussian prior, and only favors deterministic meta-learning approaches. This work markedly mitigates the scalability bottleneck by cross-fertilizing the benefits of implicit differentiation to probabilistic Bayesian meta-learning. The novel implicit Bayesian meta-learning (iBaML) method not only broadens the scope of learnable priors, but also quantifies the associated uncertainty. Furthermore, the ultimate complexity is well controlled regardless of the inner-level optimization trajectory. Analytical error bounds are established to demonstrate the precision and efficiency of the generalized implicit gradient over the explicit one. Extensive numerical tests are also carried out to empirically validate the performance of the proposed method.Comment: Accepted as a poster paper in the main track of Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI-23

    Explain what you see:argumentation-based learning and robotic vision

    Get PDF
    In this thesis, we have introduced new techniques for the problems of open-ended learning, online incremental learning, and explainable learning. These methods have applications in the classification of tabular data, 3D object category recognition, and 3D object parts segmentation. We have utilized argumentation theory and probability theory to develop these methods. The first proposed open-ended online incremental learning approach is Argumentation-Based online incremental Learning (ABL). ABL works with tabular data and can learn with a small number of learning instances using an abstract argumentation framework and bipolar argumentation framework. It has a higher learning speed than state-of-the-art online incremental techniques. However, it has high computational complexity. We have addressed this problem by introducing Accelerated Argumentation-Based Learning (AABL). AABL uses only an abstract argumentation framework and uses two strategies to accelerate the learning process and reduce the complexity. The second proposed open-ended online incremental learning approach is the Local Hierarchical Dirichlet Process (Local-HDP). Local-HDP aims at addressing two problems of open-ended category recognition of 3D objects and segmenting 3D object parts. We have utilized Local-HDP for the task of object part segmentation in combination with AABL to achieve an interpretable model to explain why a certain 3D object belongs to a certain category. The explanations of this model tell a user that a certain object has specific object parts that look like a set of the typical parts of certain categories. Moreover, integrating AABL and Local-HDP leads to a model that can handle a high degree of occlusion

    Arguments, rules and cases in law: Resources for aligning learning and reasoning in structured domains

    Get PDF
    This paper provides a formal description of two legal domains. In addition, we describe the generation of various artificial datasets from these domains and explain the use of these datasets in previous experiments aligning learning and reasoning. These resources are made available for the further investigation of connections between arguments, cases and rules. The datasets are publicly available at https://github.com/CorSteging/LegalResource

    ATHENA Research Book, Volume 2

    Get PDF
    ATHENA European University is an association of nine higher education institutions with the mission of promoting excellence in research and innovation by enabling international cooperation. The acronym ATHENA stands for Association of Advanced Technologies in Higher Education. Partner institutions are from France, Germany, Greece, Italy, Lithuania, Portugal and Slovenia: University of Orléans, University of Siegen, Hellenic Mediterranean University, Niccolò Cusano University, Vilnius Gediminas Technical University, Polytechnic Institute of Porto and University of Maribor. In 2022, two institutions joined the alliance: the Maria Curie-Skłodowska University from Poland and the University of Vigo from Spain. Also in 2022, an institution from Austria joined the alliance as an associate member: Carinthia University of Applied Sciences. This research book presents a selection of the research activities of ATHENA University's partners. It contains an overview of the research activities of individual members, a selection of the most important bibliographic works of members, peer-reviewed student theses, a descriptive list of ATHENA lectures and reports from individual working sections of the ATHENA project. The ATHENA Research Book provides a platform that encourages collaborative and interdisciplinary research projects by advanced and early career researchers

    Arguments, rules and cases in law: Resources for aligning learning and reasoning in structured domains

    Get PDF
    This paper provides a formal description of two legal domains. In addition, we describe the generation of various artificial datasets from these domains and explain the use of these datasets in previous experiments aligning learning and reasoning. These resources are made available for the further investigation of connections between arguments, cases and rules. The datasets are publicly available at https://github.com/CorSteging/LegalResources.</jats:p

    Feature construction using explanations of individual predictions

    Full text link
    Feature construction can contribute to comprehensibility and performance of machine learning models. Unfortunately, it usually requires exhaustive search in the attribute space or time-consuming human involvement to generate meaningful features. We propose a novel heuristic approach for reducing the search space based on aggregation of instance-based explanations of predictive models. The proposed Explainable Feature Construction (EFC) methodology identifies groups of co-occurring attributes exposed by popular explanation methods, such as IME and SHAP. We empirically show that reducing the search to these groups significantly reduces the time of feature construction using logical, relational, Cartesian, numerical, and threshold num-of-N and X-of-N constructive operators. An analysis on 10 transparent synthetic datasets shows that EFC effectively identifies informative groups of attributes and constructs relevant features. Using 30 real-world classification datasets, we show significant improvements in classification accuracy for several classifiers and demonstrate the feasibility of the proposed feature construction even for large datasets. Finally, EFC generated interpretable features on a real-world problem from the financial industry, which were confirmed by a domain expert.Comment: 54 pages, 10 figures, 22 table

    Argue to Learn:Accelerated Argumentation-Based Learning

    Get PDF
    Human agents can acquire knowledge and learn through argumentation. Inspired by this fact, we propose a novel argumentation-based machine learning technique that can be used for online incremental learning scenarios. Existing methods for online incremental learning problems typically do not generalize well from just a few learning instances. Our previous argumentation-based online incremental learning method outperformed state-of-the-art methods in terms of accuracy and learning speed. However, it was neither memory-efficient nor computationally efficient since the algorithm used the power set of the feature values for updating the model. In this paper, we propose an accelerated version of the algorithm, with polynomial instead of exponential complexity, while achieving higher learning accuracy. The proposed method is at least 200 times faster than the original argumentation-based learning method and is more memory-efficient

    Evolution, environmental distribution, and engineering of the abyssomicin biosynthetic gene cluster

    Get PDF
    Ph. D. Thesis.Evolution, environmental distribution, and engineering of the abyssomicin biosynthetic gene cluster. Microbial secondary metabolites constitute a great source of pharmaceutically interesting biomolecules. In particular, the tetronate family of natural products is a structurally and functionally diverse group of secondary metabolites whose potent bioactivities make them attractive targets for clinical and industrial exploitation. The abyssomicins are an actively growing family of small spirotetronate natural products that has been widely studied due to the unique structural features and bioactivities that some of its members exhibit, including antimicrobial activity against Gram-positive bacteria such as methicillin- and vancomycinresistant Staphylococcus aureus and different Mycobacteria strains, HIV inhibitory and reactivator properties and anti-influenza A virus activity. Abyssomicin C and its atrop- isomer, produced by the slow growing marine Actinobacteria Micromonospora maris AB-18-032 T , are type I polyketide antibiotics that inhibit the formation of p-aminobenzoic acid, a constituent of the folate pathway. Abyssomicin biosynthesis is highly amenable to reengineering, as the enzymes involved in the synthesis of the tetronate (AbyA1) and the spiro-tetronate-forming Diels-Alderase (AbyU) are both capable of accepting structurally diverse substrates. The aim of this project was to set up the grounds for the discovery and production of novel abyssomicins with applications in the biopharmaceutical industry. First, in order to understand the environmental distribution and evolution of the abyssomicin biosynthetic gene clusters (BGCs) present in nature, an analysis of publicly available genomic and metagenomic data was carried out. The strategy of selecting a pathwayspecific enzyme to direct the mining proved to be an excellent strategy; 74 new Diels–Alderase homologs were identified and a surprising prevalence of the abyssomicin BGC within terrestrial habitats, mainly soil and plant-associated, was unveiled. Five complete and 12 partial new abyssomicin BGCs and 23 new potential abyssomicin BGCs were also identified, suggesting that a plethora of abyssomicins remain to be discovered. A preliminary study on the abyssomicin production potential of five of the strains containing potential abyssomicin BGCs was also carried out although no abyssomicins were found. After that, with the final goal of producing abyssomicins of various lengths and different saturation/oxidation patterns, it was necessary to express the aby BGC of M. maris AB-18-032 in a well-established heterologous host. This cluster was successfully moved into E. coli and various Streptomyces species, the abyssomicin production potential of these strains was evaluated in various conditions and some of the hosts were promoter engineered to force the expression of the aby BGC. Active gene expression was demonstrated, but despite the efforts, none of the heterologous hosts produced abyssomicins. Later analysis unveiled the presence of several mutations within abyB1, the first polyketide synthase gene in the aby BGC, suggesting this could be the reason for the lack of production. Since the approach to heterologously produce abyssomicins was not fruitful, this work then focused on increasing abyssomicin production in M. maris AB-18-032 and developing genetic tools for this system. First, through ribosome engineering, a library of M. maris drug-resistant mutants capable of producing up to 3.4-fold abyssomicin C in comparison to the wild-type strain was generated. Then, using statistical Design of Experiments (DOE), an efficient electroporation protocol that could accelerate targeted genetic manipulations in M. maris was developed. Together, increased abyssomicin production and a quick and easy electroporation protocol for M. maris, will facilitate future engineering of the aby BGC directly in M. maris to produce diverse non-natural abyssomicin
    corecore