4,011 research outputs found

    Statistical analysis of grouped text documents

    Get PDF
    L'argomento di questa tesi sono i modelli statistici per l'analisi dei dati testuali, con particolare attenzione ai contesti in cui i campioni di testo sono raggruppati. Quando si ha a che fare con dati testuali, il primo problema è quello di elaborarli, per renderli compatibili dal punto di vista computazionale e metodologico con i metodi matematici e statistici prodotti e continuamente sviluppati dalla comunità scientifica. Per questo motivo, la tesi passa in rassegna i metodi esistenti per la rappresentazione analitica e l'elaborazione di campioni di dati testuali, compresi i "Vector Space Models", le "rappresentazioni distribuite" di parole e documenti e i "contextualized embeddings". Questa rassegna comporta la standardizzazione di una notazione che, anche all'interno dello stesso approccio di rappresentazione, appare molto eterogenea in letteratura. Vengono poi esplorati due domini di applicazione: i social media e il turismo culturale. Per quanto riguarda il primo, viene proposto uno studio sull'autodescrizione di gruppi diversi di individui sulla piattaforma StockTwits, dove i mercati finanziari sono gli argomenti dominanti. La metodologia proposta ha integrato diversi tipi di dati, sia testuali che variabili categoriche. Questo studio ha agevolato la comprensione sul modo in cui le persone si presentano online e ha trovato stutture di comportamento ricorrenti all'interno di gruppi di utenti. Per quanto riguarda il turismo culturale, la tesi approfondisce uno studio condotto nell'ambito del progetto "Data Science for Brescia - Arts and Cultural Places", in cui è stato addestrato un modello linguistico per classificare le recensioni online scritte in italiano in quattro aree semantiche distinte relative alle attrazioni culturali della città di Brescia. Il modello proposto permette di identificare le attrazioni nei documenti di testo, anche quando non sono esplicitamente menzionate nei metadati del documento, aprendo così la possibilità di espandere il database relativo a queste attrazioni culturali con nuove fonti, come piattaforme di social media, forum e altri spazi online. Infine, la tesi presenta uno studio metodologico che esamina la specificità di gruppo delle parole, analizzando diversi stimatori di specificità di gruppo proposti in letteratura. Lo studio ha preso in considerazione documenti testuali raggruppati con variabile di "outcome" e variabile di gruppo. Il suo contributo consiste nella proposta di modellare il corpus di documenti come una distribuzione multivariata, consentendo la simulazione di corpora di documenti di testo con caratteristiche predefinite. La simulazione ha fornito preziose indicazioni sulla relazione tra gruppi di documenti e parole. Inoltre, tutti i risultati possono essere liberamente esplorati attraverso un'applicazione web, i cui componenti sono altresì descritti in questo manoscritto. In conclusione, questa tesi è stata concepita come una raccolta di studi, ognuno dei quali suggerisce percorsi di ricerca futuri per affrontare le sfide dell'analisi dei dati testuali raggruppati.The topic of this thesis is statistical models for the analysis of textual data, emphasizing contexts in which text samples are grouped. When dealing with text data, the first issue is to process it, making it computationally and methodologically compatible with the existing mathematical and statistical methods produced and continually developed by the scientific community. Therefore, the thesis firstly reviews existing methods for analytically representing and processing textual datasets, including Vector Space Models, distributed representations of words and documents, and contextualized embeddings. It realizes this review by standardizing a notation that, even within the same representation approach, appears highly heterogeneous in the literature. Then, two domains of application are explored: social media and cultural tourism. About the former, a study is proposed about self-presentation among diverse groups of individuals on the StockTwits platform, where finance and stock markets are the dominant topics. The methodology proposed integrated various types of data, including textual and categorical data. This study revealed insights into how people present themselves online and found recurring patterns within groups of users. About the latter, the thesis delves into a study conducted as part of the "Data Science for Brescia - Arts and Cultural Places" Project, where a language model was trained to classify Italian-written online reviews into four distinct semantic areas related to cultural attractions in the Italian city of Brescia. The model proposed allows for the identification of attractions in text documents, even when not explicitly mentioned in document metadata, thus opening possibilities for expanding the database related to these cultural attractions with new sources, such as social media platforms, forums, and other online spaces. Lastly, the thesis presents a methodological study examining the group-specificity of words, analyzing various group-specificity estimators proposed in the literature. The study considered grouped text documents with both outcome and group variables. Its contribution consists of the proposal of modeling the corpus of documents as a multivariate distribution, enabling the simulation of corpora of text documents with predefined characteristics. The simulation provided valuable insights into the relationship between groups of documents and words. Furthermore, all its results can be freely explored through a web application, whose components are also described in this manuscript. In conclusion, this thesis has been conceived as a collection of papers. It aimed to contribute to the field with both applications and methodological proposals, and each study presented here suggests paths for future research to address the challenges in the analysis of grouped textual data

    UMSL Bulletin 2023-2024

    Get PDF
    The 2023-2024 Bulletin and Course Catalog for the University of Missouri St. Louis.https://irl.umsl.edu/bulletin/1088/thumbnail.jp

    Multidisciplinary perspectives on Artificial Intelligence and the law

    Get PDF
    This open access book presents an interdisciplinary, multi-authored, edited collection of chapters on Artificial Intelligence (‘AI’) and the Law. AI technology has come to play a central role in the modern data economy. Through a combination of increased computing power, the growing availability of data and the advancement of algorithms, AI has now become an umbrella term for some of the most transformational technological breakthroughs of this age. The importance of AI stems from both the opportunities that it offers and the challenges that it entails. While AI applications hold the promise of economic growth and efficiency gains, they also create significant risks and uncertainty. The potential and perils of AI have thus come to dominate modern discussions of technology and ethics – and although AI was initially allowed to largely develop without guidelines or rules, few would deny that the law is set to play a fundamental role in shaping the future of AI. As the debate over AI is far from over, the need for rigorous analysis has never been greater. This book thus brings together contributors from different fields and backgrounds to explore how the law might provide answers to some of the most pressing questions raised by AI. An outcome of the Católica Research Centre for the Future of Law and its interdisciplinary working group on Law and Artificial Intelligence, it includes contributions by leading scholars in the fields of technology, ethics and the law.info:eu-repo/semantics/publishedVersio

    LIPIcs, Volume 251, ITCS 2023, Complete Volume

    Get PDF
    LIPIcs, Volume 251, ITCS 2023, Complete Volum

    UMSL Bulletin 2022-2023

    Get PDF
    The 2022-2023 Bulletin and Course Catalog for the University of Missouri St. Louis.https://irl.umsl.edu/bulletin/1087/thumbnail.jp

    Investigating the learning potential of the Second Quantum Revolution: development of an approach for secondary school students

    Get PDF
    In recent years we have witnessed important changes: the Second Quantum Revolution is in the spotlight of many countries, and it is creating a new generation of technologies. To unlock the potential of the Second Quantum Revolution, several countries have launched strategic plans and research programs that finance and set the pace of research and development of these new technologies (like the Quantum Flagship, the National Quantum Initiative Act and so on). The increasing pace of technological changes is also challenging science education and institutional systems, requiring them to help to prepare new generations of experts. This work is placed within physics education research and contributes to the challenge by developing an approach and a course about the Second Quantum Revolution. The aims are to promote quantum literacy and, in particular, to value from a cultural and educational perspective the Second Revolution. The dissertation is articulated in two parts. In the first, we unpack the Second Quantum Revolution from a cultural perspective and shed light on the main revolutionary aspects that are elevated to the rank of principles implemented in the design of a course for secondary school students, prospective and in-service teachers. The design process and the educational reconstruction of the activities are presented as well as the results of a pilot study conducted to investigate the impact of the approach on students' understanding and to gather feedback to refine and improve the instructional materials. The second part consists of the exploration of the Second Quantum Revolution as a context to introduce some basic concepts of quantum physics. We present the results of an implementation with secondary school students to investigate if and to what extent external representations could play any role to promote students’ understanding and acceptance of quantum physics as a personal reliable description of the world

    Finding an effective problem-solving heuristic instructional approach for circle geometry

    Get PDF
    Abstracts in English, Afrikaans and ZuluThis research study carried out an investigation into finding a contemporary problem- solving instructional approach that will be effective for teaching and learning of mathematics in South African schools, with specific focus on circle geometry. Prior to conducting this study, a retrospection was done into the mathematical practices implemented, in schools in South Africa, by researchers, educational practitioners and stakeholders such as Non-Governmental Organisations. The aforementioned unanimously identified the instructional approaches for teaching and learning of mathematics, particularly, the traditional teaching and learning approach, as problematic and counter-productive, and this might be contributing to poor learners’ performances. In a bid to replace the obsolete traditional approach, the researcher in this study recommended: “teaching thinking skills” and “teaching effective problem-solving instructional approaches” as more appropriate. With regards to teaching thinking skills, the infusion approach (teaching thinking skills, along with content instructions), was highlighted. For teaching effective problem-solving, Polya’s Problem-Solving Model, was investigated. To ensure an effective design and implementation of the proposed problem-solving instructional approach, the APOS theory (ACE teaching cycle) was adopted. Also, the teaching and learning of circle geometry was carried out in a collaborative classroom setting. This proposed instructional approach was tentatively, labelled as “IPAC mathematics problem-solving instructional model’’ or simply, the “IPAC model”. This was an acronym for the four elements of this new approach, namely - the infusion approach, Polya’s approach, and APOS theory in a collaborative learning classroom. Two groups of Grade 11 mathematics learners served as participants for this study: group 1 - 11A had 30 learners (the control group) and group 2- 11B had 32 learners (the experimental group). Data collected methods for this study were: observations of participants in their natural classroom settings, recorded videos, questionnaires, photograph of participants’ work (classwork/homework and standardized tests). This study followed a mixed-method research design, hence, both quantitative and qualitative data analyses procedures were implemented. The quantitative data was analysed by implementing inferential statistics and descriptive statistics, while the APOS theory analysis was used to analyse the qualitative facet of the collected data. During the APOS theory analysis, content analysis was done on participants’ written responses to each of the four standardized tests’ data. The content analysis was carried out on the written responses of participants, from both the control and the experimental groups. The research findings that emanated from this study were the following: that this new method of teaching and learning is valid, practical and effective; there was a statistically significant improvement in the test scores of participants who were taught by the new instructional approach; participants’ conceptual understanding, procedural fluency, strategic competence and mathematical reasoning skills were enhanced; participants’ problem-solving competence improved, during and after the intervention; the IPAC model guided the majority of the participants to operate at the object and schema levels in relation to the APOS theory mental conceptions. Lastly, the ACE teaching instructional approach significantly guided and enhanced participants’ cognitive engagement and development, which ultimately, optimized their problem-solving competence. Based on these research findings, the researcher recommended among others, that the new instructional approach - the IPAC model, should be implemented for teaching and learning of circle geometry in South African schools. The researcher also recommended that cultivation of thinking skills and implementation of effective problem-solving instructional approaches should be prioritized in mathematics classrooms in South Africa. The researcher established from this study that the developed IPAC model will serve as an effective and a reliable pedagogical tool which can address some of the teaching and learning challenges teachers and learners encounter in mathematics classrooms.Hierdie navorsingstudie het 'n ondersoek gedoen na die vind van 'n kontemporêre probleemoplossende onderrigbenadering wat effektief sal wees vir onderrig en leer van wiskunde in Suid-Afrikaanse skole, met spesifieke fokus op sirkelmeetkunde. Voor die uitvoering van hierdie studie is 'n terugblik gedoen na die wiskundige praktyke wat in skole in Suid-Afrika geïmplementeer is deur navorsers, opvoedkundige praktisyns en belanghebbendes soos nie-regeringsorganisasies. Die instruksionele benaderings vir onderrig en leer van wiskunde, veral die tradisionele onderrig-en-leerbenadering, is eenparig geïdentifiseer as problematies en teenproduktief, en dit kan dalk bydra tot swak leerders se prestasies. In 'n poging om die uitgediende tradisionele benadering te vervang, het die navorser in hierdie studie aanbeveel: "onderrig van denkvaardighede" en "onderrig van effektiewe probleemoplossende onderrigbenaderings" as meer gepas. Met betrekking tot die onderrig van denkvaardig hede, is die infusiebenadering (onderrig van denkvaardighede, tesame met inhoudsinstruksies), uitgelig. Vir die onderrig van effektiewe probleemoplossing is Polya se probleemoplossingsmodel ondersoek. Om 'n effektiewe ontwerp en implementering van die voorgestelde probleemoplossende onderrigbenadering te verseker, is die APOS-teorie (GOS-onderrigsiklus) aanvaar. Die onderrig en leer van sirkelmeetkunde is ook in 'n samewerkende klaskameropset uitgevoer. Hierdie voorgestelde onderrigbenadering is voorlopig, gemerk as "IPAC wiskunde probleemoplossing instruksionele model" of eenvoudig die "IPAC model". Dit was 'n akroniem vir die vier elemente van hierdie nuwe benadering, naamlik - die infusiebenadering, Polya se benadering en APOS-teorie in 'n samewerkende leerklaskamer. Twee groepe graad 11-wiskunde-leerders het as deelnemers vir hierdie studie gedien: groep 1 - 11A het 30 leerders (die kontrolegroep) en groep 2- 11B het 32 leerders (die eksperimentele groep). Data wat ingesamel is metodes vir hierdie studie was: waarnemings van deelnemers in hul natuurlike klaskamerinstellings, opgeneemde video's, vraelyste, foto van deelnemers se werk (klaswerk/huiswerk en gestandaardiseerde toetse). Hierdie studie het 'n gemengde-metode navorsingsontwerp gevolg, dus is beide kwantitatiewe en kwalitatiewe data-ontledingsprosedures geïmplementeer. Die kwantitatiewe data is ontleed deur inferensiële statistiek en beskrywende statistiek te implementeer, terwyl die APOS teorie-analise gebruik is om te analiseer die kwalitatiewe faset van die versamelde data. Tydens die APOS-teorie-analise is inhoudsontleding gedoen op deelnemers se geskrewe antwoorde op elk van die vier gestandaardiseerde toetse se data. Die inhoudsanalise is uitgevoer op die geskrewe reaksie van deelnemers, van beide die kontrole- en die eksperimentele groepe. Die navorsingsbevindinge wat uit hierdie studie voortgespruit het, was die volgende: dat hierdie nuwe metode van onderrig en leer geldig, prakties en effektief is; daar was 'n statisties beduidende verbetering in die toetstellings van deelnemers wat deur die nuwe onderrigbenadering onderrig is; deelnemers se konseptuele begrip, prosedurele vlotheid, strategiese bevoegdheid en wiskundige redenasievaardighede is verbeter; deelnemers se probleemoplossingsbevoegdheid het verbeter, tydens en na die intervensie; die IPAC-model het die meerderheid van die deelnemers gelei om op die objek- en skemavlakke te werk in verhouding tot die APOS-teorie se verstandelike opvattings. Laastens het die GOS-onderrigbenadering die deelnemers se kognitiewe betrokkenheid en ontwikkeling aansienlik gelei en verbeter, wat uiteindelik hul probleemoplossingsbevoegdheid geoptimaliseer het. Op grond van hierdie navorsingsbevindinge het die navorser onder andere aanbeveel dat die nuwe onderrigbenadering - die IPAC-model, geïmplementeer moet word vir onderrig en leer van sirkelmeetkunde in Suid-Afrikaanse skole. Die navorser het ook aanbeveel dat die kweek van denkvaardighede en implementering van effektiewe probleemoplossende onderrigbenaderings in wiskundeklaskamers in Suid-Afrika geprioritiseer moet word. Die navorser het uit hierdie studie vasgestel dat die ontwikkelde IPAC-model sal dien as 'n effektiewe en betroubare pedagogiese hulpmiddel wat sommige van die onderrig- en leeruitdagings wat onderwysers en leerders in wiskundeklaskamers ondervind, kan aanspreek.Lolu cwaningo luqukethe uphenyo mayelana nokuthola ikhambi elingaxazulula ekutholeni indlela eqondile engaletha imiphumela ewusizo ekufundiseni nasekufundeni kwezibalo ezikoleni zaseMzansi Africa, ezophinde ibhekane ngqo ne circle Geometry. Ngaphambi kokuba kuqale lolu cwaningo, kube nolunye ucwaningo olunzulu olwenziwe ngezinye izindlela esezivele zikhona mayelana nezibalo, ezikoleni zaseMzansi Africa, lwenziwa ngabacwaningi, izifundiswa ezingo ncweti Kanye nezinhlangano ezizimele. Inhlangano ebizwa nge okushiwo ngenhla luhlonze indlela eqondile yokufundisa nokufunda izibalo, ikakhulukazi, indlela ejwayelekile yokwenza, njengezindlela eziyinkinga nezingahambisani, futhi lokhu ngungaba yimbangela ekungenzini kahle kwabafundi. Emkhankasweni wokushintsha lolu hlelo oludala lokwenza olungasasizi, uMhlaziyi kulolu cwaningo uncome ukuthi: “ikhono elufundisa ukuzicabangela” Kanye “nekhono lokufundisa elisebenzayo ukuzixazululela izinkinga” njengendlela okuyiyo efanele. Mayelana nekhono elifundisa ukuzicabangela, indlela eyiqophelo (ikhono elifundisa ukuzicabangela, elihambisana nemigomo equkethwe), luthintiwe. Mayelana nohlelo oluwusizo ekuxazululeni izinkinga, uhlelo luka Polya lokuxazulula izinkinga luphenyiwe. Ukuqinisekisa ukuthi uhlelo olusebenzayo futhi oluzosentsenziswa ekuphakamiseni indlela eqondile enemigomo ekuxazululeni izinkinga yokwenza, i APOS theory (ACE teaching cycle) iyona ekhethiwe. Okunye, uhlelo lokufundisa nokufunda i circle geometry lukhishiwe endleleni ehlanganisayo yokuhlala egunjini lokufunda. Okwamanje Lolu hlelo oluphakanyisiwe lokufundisa, lubekwe njenge “IPAC indlela yezibalo eqondile yokuxazulula izinkinga enemigomo” . Lokhu kuyigama elifinqiwe elakhiwe izinhlamvu ezine kule ndlela entsha ebizwa nge infusion approach, Polya’s approach, Kanye ne APOS theory egunjini lokufunda elihlanganisile. Amaqembu amabili ebanga le shuminanye labafundi bezibalo basentshenzisiwe ukubamba iqhaza kulolu cwaningo: iqembu lokuqala ibanga 11A ebelinabafundi abangu 30 (iqembu labaqondisi) bese iqembu lesibili ibanga 11B ebelinabafundi abangu 32 (iqembu elenzayo). Ucwaningo oluqoqiwe lwalendlela lube kanje: imibono yalaba ebekade bebambe iqhaza egunjini lokufunda obuhleliwe, baqophe amavidiyo, babhala imibuzo, bathatha izithombe zalaba ekade bembambe iqhaza lwalomsebenzi wokubamba iqhaza. (imisebenzi yasegunjini lokufunda/imisebenzi yasekhaya Kanye nokwenza uvivinyo). Lolu phenyo lulandele uhlelo oluxubile okuwuhlelo lokuphenya, yingakho zombili lezi zinhlelo zokuqukethwe nokuseZingeni zokuqoqa uphenyo olwenziwe zisentshenzisiwe. Uhlelo lokuqukethwe lemininingwane lusentshenzisiwe ukuhlaziya ngokusebenzisa uhlelo lokuqoqa okutholakele Kanye nohlelo lokwenza okutholakele, futhi kube kwenziwa ne APOS theory analysis ukuhlaziya okusezingeni eliphezulu zigxenye zonke lwemininingwane eqoqiwe. Ngesikhathi se APOS theory analysis, ukuhlaziywa kokuqukethwe okwenziwe ababambe iqhaza babhale okwenzekile ngesikhathi benza lezi zivivinyo ezine ezibekiwe. Uhlelo lokuhlaziya okuqukethwe lwenziwe labhalwa yilaba kade bebambe iqhaza, kuwo womabili amaqembu , elokuqondisa nelokwenza. Uphenyo olutholakele kulolu hlelo lunje: lolu hlelo lokufundisa nokufunda luyasebenza, luyenzeka, futhi lunomehluko: ngokwezibalo kube nomehluko omkhulu oncono ezibalweni zalabo ekade bebambe iqhaza besebenzisa indlela entsha yemigomo: bonke ekade bebambe iqhaza bathole ithuba lokuthi kuthuthuke amakhono abo ekwazini ukuqonda ukuzicabangela, ekwazini ukwenza izinto ezinomehluko eyinqubomgomo, ukumelana nezindlela eziningi eziphumelelisayo Kanye nekhono lokuqonda izibalo; ikhono lalabo ekade bebambe iqhaza ekuxazululeni izinkinga ngokusezingeni lithuthukile, ngesikhathi nangemuva kokwenza ucwaningo; I IPAC model ukwenzisa abaningi balaba ekade bebambe iqhaza kalula umsebenzi ngokuhlukana kwamazinga kusentsenziswa i APOS theory. Ekugcineni, indlela yokwenza ebizwa nge ACE teaching ikwazile okwenzisa kahle ngokusezingeni eliphezulu futhi yakhuphula labo ebekade bebambe iqhaza yaphinde yabathuthukisa, lokhu okwenze bakwazi ukuba sezingeni lokuphumelela ukuxazulula izinkinga. Ngenxa yalokhu okutholakale kucwaningo, umcwaningi uncome ukuthi kokunye, indlela entsha yokwenza ngemigomo – i-IPAC, kumele isentshenziswe ekufundiseni nasekufundeni i circle geometry ezikoleni zaseMzansi Africa. Umcwaningi uphinde waphakamisa ukuthi ukuthuthukisa ikhono lokuzicabangela nokwenziwa kwezindlela ezisebenzayo zokuxazulula izinkinga kumele zibekwe phambili emagunjini okufunda izibalo eMzansi Africa. Umcwaningi ubeke indlela eseqophelweni eliphezulu eyisisekelo kusukela kwisifundo esenziwe yokuthi i IPAC model iyona esebenza njenge ndlela eyithuluzi elibonakalayo futhi elinemiphumela emihle ethembekile, engakwazi ukubhekana nezinkinga futhi ixazulule izinqinamba zokufundisa nokufunda ezikoleni, lezi othisha nabafundi ababhekana nazo egunjini lokufundela izibaloEducational StudiesD. Phil. (Education

    NEMISA Digital Skills Conference (Colloquium) 2023

    Get PDF
    The purpose of the colloquium and events centred around the central role that data plays today as a desirable commodity that must become an important part of massifying digital skilling efforts. Governments amass even more critical data that, if leveraged, could change the way public services are delivered, and even change the social and economic fortunes of any country. Therefore, smart governments and organisations increasingly require data skills to gain insights and foresight, to secure themselves, and for improved decision making and efficiency. However, data skills are scarce, and even more challenging is the inconsistency of the associated training programs with most curated for the Science, Technology, Engineering, and Mathematics (STEM) disciplines. Nonetheless, the interdisciplinary yet agnostic nature of data means that there is opportunity to expand data skills into the non-STEM disciplines as well.College of Engineering, Science and Technolog

    When Deep Learning Meets Polyhedral Theory: A Survey

    Full text link
    In the past decade, deep learning became the prevalent methodology for predictive modeling thanks to the remarkable accuracy of deep neural networks in tasks such as computer vision and natural language processing. Meanwhile, the structure of neural networks converged back to simpler representations based on piecewise constant and piecewise linear functions such as the Rectified Linear Unit (ReLU), which became the most commonly used type of activation function in neural networks. That made certain types of network structure \unicode{x2014}such as the typical fully-connected feedforward neural network\unicode{x2014} amenable to analysis through polyhedral theory and to the application of methodologies such as Linear Programming (LP) and Mixed-Integer Linear Programming (MILP) for a variety of purposes. In this paper, we survey the main topics emerging from this fast-paced area of work, which bring a fresh perspective to understanding neural networks in more detail as well as to applying linear optimization techniques to train, verify, and reduce the size of such networks

    Modular lifelong machine learning

    Get PDF
    Deep learning has drastically improved the state-of-the-art in many important fields, including computer vision and natural language processing (LeCun et al., 2015). However, it is expensive to train a deep neural network on a machine learning problem. The overall training cost further increases when one wants to solve additional problems. Lifelong machine learning (LML) develops algorithms that aim to efficiently learn to solve a sequence of problems, which become available one at a time. New problems are solved with less resources by transferring previously learned knowledge. At the same time, an LML algorithm needs to retain good performance on all encountered problems, thus avoiding catastrophic forgetting. Current approaches do not possess all the desired properties of an LML algorithm. First, they primarily focus on preventing catastrophic forgetting (Diaz-Rodriguez et al., 2018; Delange et al., 2021). As a result, they neglect some knowledge transfer properties. Furthermore, they assume that all problems in a sequence share the same input space. Finally, scaling these methods to a large sequence of problems remains a challenge. Modular approaches to deep learning decompose a deep neural network into sub-networks, referred to as modules. Each module can then be trained to perform an atomic transformation, specialised in processing a distinct subset of inputs. This modular approach to storing knowledge makes it easy to only reuse the subset of modules which are useful for the task at hand. This thesis introduces a line of research which demonstrates the merits of a modular approach to lifelong machine learning, and its ability to address the aforementioned shortcomings of other methods. Compared to previous work, we show that a modular approach can be used to achieve more LML properties than previously demonstrated. Furthermore, we develop tools which allow modular LML algorithms to scale in order to retain said properties on longer sequences of problems. First, we introduce HOUDINI, a neurosymbolic framework for modular LML. HOUDINI represents modular deep neural networks as functional programs and accumulates a library of pre-trained modules over a sequence of problems. Given a new problem, we use program synthesis to select a suitable neural architecture, as well as a high-performing combination of pre-trained and new modules. We show that our approach has most of the properties desired from an LML algorithm. Notably, it can perform forward transfer, avoid negative transfer and prevent catastrophic forgetting, even across problems with disparate input domains and problems which require different neural architectures. Second, we produce a modular LML algorithm which retains the properties of HOUDINI but can also scale to longer sequences of problems. To this end, we fix the choice of a neural architecture and introduce a probabilistic search framework, PICLE, for searching through different module combinations. To apply PICLE, we introduce two probabilistic models over neural modules which allows us to efficiently identify promising module combinations. Third, we phrase the search over module combinations in modular LML as black-box optimisation, which allows one to make use of methods from the setting of hyperparameter optimisation (HPO). We then develop a new HPO method which marries a multi-fidelity approach with model-based optimisation. We demonstrate that this leads to improvement in anytime performance in the HPO setting and discuss how this can in turn be used to augment modular LML methods. Overall, this thesis identifies a number of important LML properties, which have not all been attained in past methods, and presents an LML algorithm which can achieve all of them, apart from backward transfer
    corecore