2,030 research outputs found
LIPIcs, Volume 251, ITCS 2023, Complete Volume
LIPIcs, Volume 251, ITCS 2023, Complete Volum
Modular lifelong machine learning
Deep learning has drastically improved the state-of-the-art in many important fields, including computer vision and natural language processing (LeCun et al., 2015). However, it is expensive to train a deep neural network on a machine learning problem. The overall training cost further increases when one wants to solve additional problems. Lifelong machine learning (LML) develops algorithms that aim to efficiently learn to solve a sequence of problems, which become available one at a time. New problems are solved with less resources by transferring previously learned knowledge. At the same time, an LML algorithm needs to retain good performance on all encountered problems, thus avoiding catastrophic forgetting. Current approaches do not possess all the desired properties of an LML algorithm. First, they primarily focus on preventing catastrophic forgetting (Diaz-Rodriguez et al., 2018; Delange et al., 2021). As a result, they neglect some knowledge transfer properties. Furthermore, they assume that all problems in a sequence share the same input space. Finally, scaling these methods to a large sequence of problems remains a challenge.
Modular approaches to deep learning decompose a deep neural network into sub-networks, referred to as modules. Each module can then be trained to perform an atomic transformation, specialised in processing a distinct subset of inputs. This modular approach to storing knowledge makes it easy to only reuse the subset of modules which are useful for the task at hand.
This thesis introduces a line of research which demonstrates the merits of a modular approach to lifelong machine learning, and its ability to address the aforementioned shortcomings of other methods. Compared to previous work, we show that a modular approach can be used to achieve more LML properties than previously demonstrated. Furthermore, we develop tools which allow modular LML algorithms to scale in order to retain said properties on longer sequences of problems.
First, we introduce HOUDINI, a neurosymbolic framework for modular LML. HOUDINI represents modular deep neural networks as functional programs and accumulates a library of pre-trained modules over a sequence of problems. Given a new problem, we use program synthesis to select a suitable neural architecture, as well as a high-performing combination of pre-trained and new modules. We show that our approach has most of the properties desired from an LML algorithm. Notably, it can perform forward transfer, avoid negative transfer and prevent catastrophic forgetting, even across problems with disparate input domains and problems which require different neural architectures.
Second, we produce a modular LML algorithm which retains the properties of HOUDINI but can also scale to longer sequences of problems. To this end, we fix the choice of a neural architecture and introduce a probabilistic search framework, PICLE, for searching through different module combinations. To apply PICLE, we introduce two probabilistic models over neural modules which allows us to efficiently identify promising module combinations.
Third, we phrase the search over module combinations in modular LML as black-box optimisation, which allows one to make use of methods from the setting of hyperparameter optimisation (HPO). We then develop a new HPO method which marries a multi-fidelity approach with model-based optimisation. We demonstrate that this leads to improvement in anytime performance in the HPO setting and discuss how this can in turn be used to augment modular LML methods.
Overall, this thesis identifies a number of important LML properties, which have not all been attained in past methods, and presents an LML algorithm which can achieve all of them, apart from backward transfer
Anwendungen maschinellen Lernens für datengetriebene Prävention auf Populationsebene
Healthcare costs are systematically rising, and current therapy-focused healthcare systems are not sustainable in the long run. While disease prevention is a viable instrument for reducing costs and suffering, it requires risk modeling to stratify populations, identify high- risk individuals and enable personalized interventions. In current clinical practice, however, systematic risk stratification is limited: on the one hand, for the vast majority of endpoints, no risk models exist. On the other hand, available models focus on predicting a single disease at a time, rendering predictor collection burdensome. At the same time, the den- sity of individual patient data is constantly increasing. Especially complex data modalities, such as -omics measurements or images, may contain systemic information on future health trajectories relevant for multiple endpoints simultaneously. However, to date, this data is inaccessible for risk modeling as no dedicated methods exist to extract clinically relevant information. This study built on recent advances in machine learning to investigate the ap- plicability of four distinct data modalities not yet leveraged for risk modeling in primary prevention. For each data modality, a neural network-based survival model was developed to extract predictive information, scrutinize performance gains over commonly collected covariates, and pinpoint potential clinical utility. Notably, the developed methodology was able to integrate polygenic risk scores for cardiovascular prevention, outperforming existing approaches and identifying benefiting subpopulations. Investigating NMR metabolomics, the developed methodology allowed the prediction of future disease onset for many common diseases at once, indicating potential applicability as a drop-in replacement for commonly collected covariates. Extending the methodology to phenome-wide risk modeling, elec- tronic health records were found to be a general source of predictive information with high systemic relevance for thousands of endpoints. Assessing retinal fundus photographs, the developed methodology identified diseases where retinal information most impacted health trajectories. In summary, the results demonstrate the capability of neural survival models to integrate complex data modalities for multi-disease risk modeling in primary prevention and illustrate the tremendous potential of machine learning models to disrupt medical practice toward data-driven prevention at population scale.Die Kosten im Gesundheitswesen steigen systematisch und derzeitige therapieorientierte Gesundheitssysteme sind nicht nachhaltig. Angesichts vieler verhinderbarer Krankheiten stellt die Prävention ein veritables Instrument zur Verringerung von Kosten und Leiden dar. Risikostratifizierung ist die grundlegende Voraussetzung für ein präventionszentri- ertes Gesundheitswesen um Personen mit hohem Risiko zu identifizieren und Maßnah- men einzuleiten. Heute ist eine systematische Risikostratifizierung jedoch nur begrenzt möglich, da für die meisten Krankheiten keine Risikomodelle existieren und sich verfüg- bare Modelle auf einzelne Krankheiten beschränken. Weil für deren Berechnung jeweils spezielle Sets an Prädiktoren zu erheben sind werden in Praxis oft nur wenige Modelle angewandt. Gleichzeitig versprechen komplexe Datenmodalitäten, wie Bilder oder -omics- Messungen, systemische Informationen über zukünftige Gesundheitsverläufe, mit poten- tieller Relevanz für viele Endpunkte gleichzeitig. Da es an dedizierten Methoden zur Ex- traktion klinisch relevanter Informationen fehlt, sind diese Daten jedoch für die Risikomod- ellierung unzugänglich, und ihr Potenzial blieb bislang unbewertet. Diese Studie nutzt ma- chinelles Lernen, um die Anwendbarkeit von vier Datenmodalitäten in der Primärpräven- tion zu untersuchen: polygene Risikoscores für die kardiovaskuläre Prävention, NMR Meta- bolomicsdaten, elektronische Gesundheitsakten und Netzhautfundusfotos. Pro Datenmodal- ität wurde ein neuronales Risikomodell entwickelt, um relevante Informationen zu extra- hieren, additive Information gegenüber üblicherweise erfassten Kovariaten zu quantifizieren und den potenziellen klinischen Nutzen der Datenmodalität zu ermitteln. Die entwickelte Me-thodik konnte polygene Risikoscores für die kardiovaskuläre Prävention integrieren. Im Falle der NMR-Metabolomik erschloss die entwickelte Methodik wertvolle Informa- tionen über den zukünftigen Ausbruch von Krankheiten. Unter Einsatz einer phänomen- weiten Risikomodellierung erwiesen sich elektronische Gesundheitsakten als Quelle prädik- tiver Information mit hoher systemischer Relevanz. Bei der Analyse von Fundusfotografien der Netzhaut wurden Krankheiten identifiziert für deren Vorhersage Netzhautinformationen genutzt werden könnten. Zusammengefasst zeigten die Ergebnisse das Potential neuronaler Risikomodelle die medizinische Praxis in Richtung einer datengesteuerten, präventionsori- entierten Medizin zu verändern
Integration of substance compliance and a product lifecycle management system in case organization
Abstract. Substance compliance is the field of identifying applicable product material regulations and managing the product composition to match those regulations. As the various regulations expand and new standards are added, manufacturers must take increased precautions to ensure their products are in line with the latest regulations and standards for example by developing system integrations to ensure better management processes.
This thesis aims to study the development and implementation of an integration of a substance compliance management system with a product lifecycle management (PLM) system in a case organization. The perspective is on identifying how an integration of substance compliance and a PLM system can be conducted and what to take into consideration when introducing such an interface to current operations. The research methods used were two sets of semi-structured interviews and participatory observations.
The findings of this study indicate that substance compliance has connections to data quality. In the case organization in particular, in order to fully grasp the benefits of the integration, special care should be put into completing a three-step plan focused on improving data quality management, using change management to introduce the integration, and utilizing an early and proactive approach to substance compliance. The study largely focuses on giving actionable improvement recommendations, but it also contributes to the substance compliance literature by conducting a brief literature study on the topic and showing the connection of product data quality with the field of study.Aineiden vaatimustenmukaisuuden hallitsemisen ja tuotteen elinkaaren hallintajärjestelmän yhdistäminen kohdeyrityksessä. Tiivistelmä. Aineiden vaatimustenmukaisuuden hallitseminen on ala, jossa tunnistetaan tuotemateriaalien lainsäädännöllisiä vaatimuksia ja varmistetaan, että tuote ei sisällä vaatimustenvastaisia aineita. Tuotemateriaaleja koskevien säädösten määrän kasvaessa elektroniikkavalmistajien on huolehdittava entistä tarkemmin, että heidän tuotteensa noudattavat viimeisimpiä lainsäädäntöjä ja standardeja. Yksi tapa tehdä näin on esimerkiksi panostaa systeemien yhdistämiseen, joka takaa paremmat hallintaprosessit.
Tämän diplomityön tarkoitus on tutkia aineiden vaatimustenmukaisuuden ja tuotteen elinkaaren hallintajärjestelmän yhdistämistä kohdeyrityksessä. Pääpaino työssä on tunnistaa, miten kahden järjestelmän yhdistäminen voidaan toteuttaa, sekä mitä tulisi ottaa huomioon yhdistetyn järjestelmän käyttöönotossa. Diplomityössä käytettiin kahta eri puolistrukturoitua haastattelua sekä osallistuvia havainnointeja tutkimusmenetelminä.
Tutkimustulokset osoittavat, että aineiden vaatimustenmukaisuudella on yhteys tuotedatan laatuun. Jotta kohdeyrityksessä voitaisiin ottaa täysi hyöty yhdistetystä järjestelmästä, tulisi yrityksen toteuttaa kolmiaskeleinen parannussuunnitelma. Suunnitelman tavoite on parantaa tuotedatanlaadun hallintaa, hyödyntää muutosjohtamisen oppeja järjestelmän kehittämiseen ja käyttöönottoon, ja edesauttaa kohdeyritystä ennakoivaan aineiden vaatimustenmukaisuuteen. Työ keskittyy suurimmaksi osaksi kohdeyrityksen parannusehdotusten antamiseen, mutta se myös edistää aineiden vaatimustenmukaisuuteen kohdistuvaa kirjallisuutta pienellä kirjallisuuskatsauksella ja esittämällä linkin tuotedatan laadun kanssa
Developmental Bootstrapping of AIs
Although some current AIs surpass human abilities in closed artificial worlds
such as board games, their abilities in the real world are limited. They make
strange mistakes and do not notice them. They cannot be instructed easily, fail
to use common sense, and lack curiosity. They do not make good collaborators.
Mainstream approaches for creating AIs are the traditional manually-constructed
symbolic AI approach and generative and deep learning AI approaches including
large language models (LLMs). These systems are not well suited for creating
robust and trustworthy AIs. Although it is outside of the mainstream, the
developmental bootstrapping approach has more potential. In developmental
bootstrapping, AIs develop competences like human children do. They start with
innate competences. They interact with the environment and learn from their
interactions. They incrementally extend their innate competences with
self-developed competences. They interact and learn from people and establish
perceptual, cognitive, and common grounding. They acquire the competences they
need through bootstrapping. However, developmental robotics has not yet
produced AIs with robust adult-level competences. Projects have typically
stopped at the Toddler Barrier corresponding to human infant development at
about two years of age, before their speech is fluent. They also do not bridge
the Reading Barrier, to skillfully and skeptically draw on the socially
developed information resources that power current LLMs. The next competences
in human cognitive development involve intrinsic motivation, imitation
learning, imagination, coordination, and communication. This position paper
lays out the logic, prospects, gaps, and challenges for extending the practice
of developmental bootstrapping to acquire further competences and create
robust, resilient, and human-compatible AIs.Comment: 102 pages, 29 figure
Elephants in a landscape of risk: spatial, temporal, and behavioural responses to anthropogenic risk in African savannah elephants
African savanna elephant (Loxodonta africana) populations have declined due to poaching for the ivory trade. Elephants and humans also increasingly share ranges and resources. This thesis investigates whether and how human-mediated risk influences elephant space use, activity patterns, resource use, grouping patterns, and sex differences in responses to risk, in the Ruaha-Rungwa ecosystem, Tanzania. This area experienced multiple poaching surges and has increasing levels of human activity.
I applied occupancy models to elephant occurrence data to investigate space use in relation to risk and environmental factors. Elephant occurrence was negatively associated with human population densities and conversion to agriculture, as well as elephant carcass occurrence (a proxy for poaching risk) and illegal human use.
Using camera trap data I compared active periods, grouping patterns, and use of roads and water sources at one low-risk site and three high-risk sites. Male and female elephants were more nocturnal in high-risk versus low-risk sites, including use of water sources; this was more pronounced for cow-calf groups than for lone males. In the high-risk versus low-risk sites, elephants were active for less time overall, avoided movement on roads, and male elephants associated more with males and cow-calf groups.
I assessed how risk influences elephant use of water sources using camera trap data. Elephant use of a high-risk resource was driven by seasonal variation in water availability, and use of high-risk water sources was more nocturnal than use of low-risk water sources. Males, but not females, adjusted group size in relation to risk.
I discuss costs associated with risk-induced behavioural shifts, including a reduction in total active time and effects on body condition, and show that the consequences of elephant poaching in Ruaha-Rungwa extend beyond effects on population size and structure. I suggest that risk-avoidance behaviour may enable elephants to persist in increasingly human-dominated landscapes
Structural optimization in steel structures, algorithms and applications
L'abstract è presente nell'allegato / the abstract is in the attachmen
Implementation of a real time Hough transform using FPGA technology
This thesis is concerned with the modelling, design and implementation of efficient architectures for performing the Hough Transform (HT) on mega-pixel resolution real-time images using Field Programmable Gate Array (FPGA) technology. Although the HT has been around for many years and a number of algorithms have been developed it still remains a significant bottleneck in many image processing applications.
Even though, the basic idea of the HT is to locate curves in an image that can be parameterized: e.g. straight lines, polynomials or circles, in a suitable parameter space, the research presented in this thesis will focus only on location of straight lines on binary images. The HT algorithm uses an accumulator array (accumulator bins) to detect the existence of a straight line on an image. As the image needs to be binarized, a novel generic synchronization circuit for windowing operations was designed to perform edge detection. An edge detection method of special interest, the canny method, is used and the design and implementation of it in hardware is achieved in this thesis.
As each image pixel can be implemented independently, parallel processing can be performed. However, the main disadvantage of the HT is the large storage and computational requirements. This thesis presents new and state-of-the-art hardware implementations for the minimization of the computational cost, using the Hybrid-Logarithmic Number System (Hybrid-LNS) for calculating the HT for fixed bit-width architectures. It is shown that using the Hybrid-LNS the computational cost is minimized, while the precision of the HT algorithm is maintained.
Advances in FPGA technology now make it possible to implement functions as the HT in reconfigurable fabrics. Methods for storing large arrays on FPGA’s are presented, where data from a 1024 x 1024 pixel camera at a rate of up to 25 frames per second are processed
- …