7,423 research outputs found

    Modular lifelong machine learning

    Get PDF
    Deep learning has drastically improved the state-of-the-art in many important fields, including computer vision and natural language processing (LeCun et al., 2015). However, it is expensive to train a deep neural network on a machine learning problem. The overall training cost further increases when one wants to solve additional problems. Lifelong machine learning (LML) develops algorithms that aim to efficiently learn to solve a sequence of problems, which become available one at a time. New problems are solved with less resources by transferring previously learned knowledge. At the same time, an LML algorithm needs to retain good performance on all encountered problems, thus avoiding catastrophic forgetting. Current approaches do not possess all the desired properties of an LML algorithm. First, they primarily focus on preventing catastrophic forgetting (Diaz-Rodriguez et al., 2018; Delange et al., 2021). As a result, they neglect some knowledge transfer properties. Furthermore, they assume that all problems in a sequence share the same input space. Finally, scaling these methods to a large sequence of problems remains a challenge. Modular approaches to deep learning decompose a deep neural network into sub-networks, referred to as modules. Each module can then be trained to perform an atomic transformation, specialised in processing a distinct subset of inputs. This modular approach to storing knowledge makes it easy to only reuse the subset of modules which are useful for the task at hand. This thesis introduces a line of research which demonstrates the merits of a modular approach to lifelong machine learning, and its ability to address the aforementioned shortcomings of other methods. Compared to previous work, we show that a modular approach can be used to achieve more LML properties than previously demonstrated. Furthermore, we develop tools which allow modular LML algorithms to scale in order to retain said properties on longer sequences of problems. First, we introduce HOUDINI, a neurosymbolic framework for modular LML. HOUDINI represents modular deep neural networks as functional programs and accumulates a library of pre-trained modules over a sequence of problems. Given a new problem, we use program synthesis to select a suitable neural architecture, as well as a high-performing combination of pre-trained and new modules. We show that our approach has most of the properties desired from an LML algorithm. Notably, it can perform forward transfer, avoid negative transfer and prevent catastrophic forgetting, even across problems with disparate input domains and problems which require different neural architectures. Second, we produce a modular LML algorithm which retains the properties of HOUDINI but can also scale to longer sequences of problems. To this end, we fix the choice of a neural architecture and introduce a probabilistic search framework, PICLE, for searching through different module combinations. To apply PICLE, we introduce two probabilistic models over neural modules which allows us to efficiently identify promising module combinations. Third, we phrase the search over module combinations in modular LML as black-box optimisation, which allows one to make use of methods from the setting of hyperparameter optimisation (HPO). We then develop a new HPO method which marries a multi-fidelity approach with model-based optimisation. We demonstrate that this leads to improvement in anytime performance in the HPO setting and discuss how this can in turn be used to augment modular LML methods. Overall, this thesis identifies a number of important LML properties, which have not all been attained in past methods, and presents an LML algorithm which can achieve all of them, apart from backward transfer

    Modelling, Monitoring, Control and Optimization for Complex Industrial Processes

    Get PDF
    This reprint includes 22 research papers and an editorial, collected from the Special Issue "Modelling, Monitoring, Control and Optimization for Complex Industrial Processes", highlighting recent research advances and emerging research directions in complex industrial processes. This reprint aims to promote the research field and benefit the readers from both academic communities and industrial sectors

    Dynamic Vector Bin Packing for Online Resource Allocation in the Cloud

    Full text link
    Several cloud-based applications, such as cloud gaming, rent servers to execute jobs which arrive in an online fashion. Each job has a resource demand and must be dispatched to a cloud server which has enough resources to execute the job, which departs after its completion. Under the `pay-as-you-go' billing model, the server rental cost is proportional to the total time that servers are actively running jobs. The problem of efficiently allocating a sequence of online jobs to servers without exceeding the resource capacity of any server while minimizing total server usage time can be modelled as a variant of the dynamic bin packing problem (DBP), called MinUsageTime DBP. In this work, we initiate the study of the problem with multi-dimensional resource demands (e.g. CPU/GPU usage, memory requirement, bandwidth usage, etc.), called MinUsageTime Dynamic Vector Bin Packing (DVBP). We study the competitive ratio (CR) of Any Fit packing algorithms for this problem. We show almost-tight bounds on the CR of three specific Any Fit packing algorithms, namely First Fit, Next Fit, and Move To Front. We prove that the CR of Move To Front is at most (2Ό+1)d+1(2\mu+1)d +1, where Ό\mu is the ratio of the max/min item durations. For d=1d=1, this significantly improves the previously known upper bound of 6Ό+76\mu+7 (Kamali & Lopez-Ortiz, 2015). We then prove the CR of First Fit and Next Fit are bounded by (Ό+2)d+1(\mu+2)d+1 and 2Όd+12\mu d+1, respectively. Next, we prove a lower bound of (Ό+1)d(\mu+1)d on the CR of any Any Fit packing algorithm, an improved lower bound of 2Όd2\mu d for Next Fit, and a lower bound of 2Ό2\mu for Move To Front in the 1-D case. All our bounds improve or match the best-known bounds for the 1-D case. Finally, we experimentally study the average-case performance of these algorithms on randomly generated synthetic data, and observe that Move To Front outperforms other Any Fit packing algorithms.Comment: 24 pages, to appear at SPAA 202

    Academic integrity : a call to research and action

    Get PDF
    Originally published in French:L'urgence de l'intĂ©gritĂ© acadĂ©mique, Éditions EMS, Management & société, Caen, 2021 (ISBN 978-2-37687-472-0).The urgency of doing complements the urgency of knowing. Urgency here is not the inconsequential injunction of irrational immediacy. It arises in various contexts for good reasons, when there is a threat to the human existence and harms to others. Today, our knowledge based civilization is at risk both by new production models of knowledge and by the shamelessness of knowledge delinquents, exposing the greatest number to important risks. Swiftly, the editors respond to the diagnostic by setting up a reference tool for academic integrity. Across multiple dialogues between the twenty-five chapters and five major themes, the ethical response shapes pragmatic horizons for action, on a range of disciplinary competencies: from science to international diplomacy. An interdisciplinary work indispensable for teachers, students and university researchers and administrators

    Patenting Genetic Information

    Get PDF
    The U.S. biotechnology industry got its start and grew to maturity over roughly three decades, beginning in the 1980s. During this period genes were patentable, and many gene patents were granted. University researchers performed basic research— often funded by the government—and then patented the genes they discovered with the encouragement of the Bayh-Dole Act, which sought to encourage practical applications of basic research by allowing patents on federally funded inventions and discoveries. At that time, when a researcher discovered the function of a gene, she could patent it such that no one else could work with that gene in the laboratory without a license. She had no right, however, to control genes in nature, including in human bodies. Universities licensed their researchers’ patents to industry, which brought in significant revenue for further research. University researchers also used gene patents as the basis for obtaining funding for start-up enterprises spun out of university labs. It was in this environment that many of today’s biotechnology companies started. In 2013, the Supreme Court held that naturally occurring genes could no longer be patented. This followed a 2012 decision that disallowed patents on many diagnostic processes. These decisions significantly changed the intellectual property protections in the biotechnology industry. Nevertheless, the industry has continued to grow and thrive. This Article investigates two questions. First, if some form of exclusive rights still applied to genes, would the biotech industry be even more robust, with more new entrants in addition to thriving, well-established companies? Second, does the current lack of protection for gene discoveries incentivize keeping such discoveries secret for the many years that it can take to develop a therapeutic based thereon—to the detriment of patients who could benefit from knowledge of the genetic associations, even before a treatment is developed? The Article concludes by analyzing what protection for discovering genetic associations, if any, will most increase social welfare

    Program and Proceedings: The Nebraska Academy of Sciences 1880-2023. 142th Anniversary Year. One Hundred-Thirty-Third Annual Meeting April 21, 2023. Hybrid Meeting: Nebraska Wesleyan University & Online, Lincoln, Nebraska

    Get PDF
    AERONAUTICS & SPACE SCIENCE Chairperson(s): Dr. Scott Tarry & Michaela Lucas HUMANS PAST AND PRESENT Chairperson(s): Phil R. Geib & Allegra Ward APPLIED SCIENCE & TECHNOLOGY SECTION Chairperson(s): Mary Ettel BIOLOGY Chairpersons: Lauren Gillespie, Steve Heinisch, and Paul Davis BIOMEDICAL SCIENCES Chairperson(s): Annemarie Shibata, Kimberly Carlson, Joseph Dolence, Alexis Hobbs, James Fletcher, Paul Denton CHEM Section Chairperson(s): Nathanael Fackler EARTH SCIENCES Chairpersons: Irina Filina, Jon Schueth, Ross Dixon, Michael Leite ENVIRONMENTAL SCIENCE Chairperson: Mark Hammer PHYSICS Chairperson(s): Dr. Adam Davis SCIENCE EDUCATION Chairperson: Christine Gustafson 2023 Maiben Lecturer: Jason Bartz 2023 FRIEND OF SCIENCE AWARD TO: Ray Ward and Jim Lewi

    Essays on Business Analytics and Game Theory

    Get PDF
    Ph.D

    Social Justice Based on Religious Forms of Prosociality in Russia

    Get PDF
    This article shows the social and psychological aspects of the prosociality in Russia which help to see the ways to social justice forming. In Russia, under the influence of Christianity forms an approach to prosocial behavior as a mandatory element of public life. Objective of study is an identification of the peculiarities of prosocial manifestation in Russian people with different levels of religiosity in modern social and cultural conditions. This study is conducted on the base of the complex of methods, namely, The Scale of Altruism (SRA); Social Norms of Prosocial Behavior (SNPB); Index of Core Spiritual Experiences (INSPIRIT); Religious Orientation Scale (RSO). The sample consists of 221 people living in various Russian cities (38% of men, 62% of women) aged 20 to 66 years (M-39.8). As a result, the collected data and their evaluation and discussion help to support the idea that spirituality and citizenship have a regulatory influence on the prosocial motives of mercy, tolerance, and altruism
    • 

    corecore