Incremental learning algorithms and applications

International audienceIncremental learning refers to learning from streaming data, which arrive over time, with limited memory resources and, ideally, without sacrificing model accuracy. This setting fits different application scenarios where lifelong learning is relevant, e.g. due to changing environments , and it offers an elegant scheme for big data processing by means of its sequential treatment. In this contribution, we formalise the concept of incremental learning, we discuss particular challenges which arise in this setting, and we give an overview about popular approaches, its theoretical foundations, and applications which emerged in the last years

Simple but Not Simplistic: Reducing the Complexity of Machine Learning Methods

Programa Oficial de Doutoramento en Computación . 5009V01[Resumo] A chegada do Big Data e a explosión do Internet das cousas supuxeron un gran reto para os investigadores en Aprendizaxe Automática, facendo que o proceso de aprendizaxe sexa mesmo roáis complexo. No mundo real, os problemas da aprendizaxe automática xeralmente teñen complexidades inherentes, como poden ser as características intrínsecas dos datos, o gran número de mostras, a alta dimensión dos datos de entrada, os cambios na distribución entre o conxunto de adestramento e test, etc. Todos estes aspectos son importantes, e requiren novoS modelos que poi dan facer fronte a estas situacións. Nesta tese, abordáronse todos estes problemas, tratando de simplificar o proceso de aprendizaxe automática no escenario actual. En primeiro lugar, realízase unha análise de complexidade para observar como inflúe esta na tarefa de clasificación, e se é posible que a aplicación dun proceso previo de selección de características reduza esta complexidade. Logo, abórdase o proceso de simplificación da fase de aprendizaxe automática mediante a filosofía divide e vencerás, usando un enfoque distribuído. Seguidamente, aplicamos esa mesma filosofía sobre o proceso de selección de características. Finalmente, optamos por un enfoque diferente seguindo a filosofía do Edge Computing, a cal permite que os datos producidos polos dispositivos do Internet das cousas se procesen máis preto de onde se crearon. Os enfoques propostos demostraron a súa capacidade para reducir a complexidade dos métodos de aprendizaxe automática tradicionais e, polo tanto, espérase que a contribución desta tese abra as portas ao desenvolvemento de novos métodos de aprendizaxe máquina máis simples, máis robustos, e máis eficientes computacionalmente.[Resumen] La llegada del Big Data y la explosión del Internet de las cosas han supuesto un gran reto para los investigadores en Aprendizaje Automático, haciendo que el proceso de aprendizaje sea incluso más complejo. En el mundo real, los problemas de aprendizaje automático generalmente tienen complejidades inherentes) como pueden ser las características intrínsecas de los datos, el gran número de muestras, la alta dimensión de los datos de entrada, los cambios en la distribución entre el conjunto de entrenamiento y test, etc. Todos estos aspectos son importantes, y requieren nuevos modelos que puedan hacer frente a estas situaciones. En esta tesis, se han abordado todos estos problemas, tratando de simplificar el proceso de aprendizaje automático en el escenario actual. En primer lugar, se realiza un análisis de complejidad para observar cómo influye ésta en la tarea de clasificación1 y si es posible que la aplicación de un proceso previo de selección de características reduzca esta complejidad. Luego, se aborda el proceso de simplificación de la fase de aprendizaje automático mediante la filosofía divide y vencerás, usando un enfoque distribuido. A continuación, aplicamos esa misma filosofía sobre el proceso de selección de características. Finalmente, optamos por un enfoque diferente siguiendo la filosofía del Edge Computing, la cual permite que los datos producidos por los dispositivos del Internet de las cosas se procesen más cerca de donde se crearon. Los enfoques propuestos han demostrado su capacidad para reducir la complejidad de los métodos de aprendizaje automático tnidicionales y, por lo tanto, se espera que la contribución de esta tesis abra las puertas al desarrollo de nuevos métodos de aprendizaje máquina más simples, más robustos, y más eficientes computacionalmente.[Abstract] The advent of Big Data and the explosion of the Internet of Things, has brought unprecedented challenges to Machine Learning researchers, making the learning task more complexo Real-world machine learning problems usually have inherent complexities, such as the intrinsic characteristics of the data, large number of instauces, high input dimensionality, dataset shift, etc. AH these aspects matter, and can fOI new models that can confront these situations. Thus, in this thesis, we have addressed aH these issues) simplifying the machine learning process in the current scenario. First, we carry out a complexity analysis to see how it inftuences the classification models, and if it is possible that feature selection might result in a deerease of that eomplexity. Then, we address the proeess of simplifying learning with the divide-and-conquer philosophy of the distributed approaeh. Later, we aim to reduce the complexity of the feature seleetion preprocessing through the same philosophy. FinallYl we opt for a different approaeh following the eurrent philosophy Edge eomputing, whieh allows the data produeed by Internet of Things deviees to be proeessed closer to where they were ereated. The proposed approaehes have demonstrated their eapability to reduce the complexity of traditional maehine learning algorithms, and thus it is expeeted that the eontribution of this thesis will open the doors to the development of new maehine learning methods that are simpler, more robust, and more eomputationally efficient

Benefit maximizing classification using feature intervals

Cataloged from PDF version of article.For a long time, classification algorithms have focused on minimizing the quantity of prediction errors by assuming that each possible error has identical consequences. However, in many real-world situations, this assumption is not convenient. For instance, in a medical diagnosis domain, misdiagnosing a sick patient as healthy is much more serious than its opposite. For this reason, there is a great need for new classification methods that can handle asymmetric cost and benefit constraints of classifications. In this thesis, we discuss cost-sensitive classification concepts and propose a new classification algorithm called Benefit Maximization with Feature Intervals (BMFI) that uses the feature projection based knowledge representation. In the framework of BMFI, we introduce five different voting methods that are shown to be effective over different domains. A number of generalization and pruning methodologies based on benefits of classification are implemented and experimented. Empirical evaluation of the methods has shown that BMFI exhibits promising performance results compared to recent wrapper cost-sensitive algorithms, despite the fact that classifier performance is highly dependent on the benefit constraints and class distributions in the domain. In order to evaluate costsensitive classification techniques, we describe a new metric, namely benefit accuracy which computes the relative accuracy of the total benefit obtained with respect to the maximum possible benefit achievable in the domain.İkizler, NazlıM.S

Determining the jet energy scale for ATLAS in the Z+Jet channel

This thesis presents a determination of the jet energy scale for the ATLAS detector using in-situ measurements. This calibration is critical, as jets are found in many analyses, and the energy measurement of jets contributes significantly to the uncertainty in numerous ATLAS results. The energy of the jet is initially taken to be the detector measurement, but this is lower than the true energy because the detector is calibrated for electromagnetic particles, not jets. One can find a correction to this energy by balancing the jet\u27s transverse momentum against a well-measured reference object. Directly calibrating the calorimeter-level jet to the particle-level is called Direct Balance; here, a different method called the Missing ET Projection Fraction (MPF) method is used instead, which balances the pt of the recoiling system against the reference object. The MPF\u27s pile-up resistant nature makes it more suitable to use in the ATLAS environment. Results for the MPF method in the Z+Jet channel are presented. A relative calibration of data to Monte Carlo simulation is provided, including a complete systematic uncertainty analysis. The uncertainty on the in-situ calibration is reduced to around 1% for most transverse momenta

MIST: a portable and efficient toolkit for molecular dynamics integration algorithm development

The main contribution of this thesis is MIST, the Molecular Integration Simula- tion Toolkit, a lightweight and efficient software library written in C++ which provides an abstract interface to common Molecular Dynamics codes, enabling rapid and portable development of new integration schemes for Molecular Dynamics. The initial release provides plug-in interfaces to NAMD-Lite, GROMACS, Amber and LAMMPS and includes several standard integration schemes, a constraint solver, temperature control using Langevin Dynamics, temperature and pressure control using Nosé-Hoover chains, and five advanced sampling schemes. I describe the architecture, functionality and internal details of the library and the C and Fortran APIs which can be used to interface additional MD codes to MIST. As an example to future developers, each of the existing plug-ins and the integrators that are included with MIST are described. Brief instructions for compilation and use of the library are also given as a reference to users. The library is designed to be expressive, portable and performant, and I show via a range of test systems that MIST introduces negligible overheads for serial, parallel, and GPU-accelerated cases, except for Amber where the native integrators run directly on the GPU itself, but only run on the CPU in MIST. The capabilities of MIST for production-quality simulations are demonstrated through the use of a simulated tempering simulation to study the free energy landscape of Alanine-12 in both vacuum and detailed solvent conditions. I also present the evaluation and application of force-field and ab initio Molecular Dynamics to study the structural properties and behaviour of olivine melts. Three existing classical potentials for fayalite are tested and found to give lattice parameters and Radial Distribution Functions in good agreement with experimental data. For forsterite, lattice parameters at ambient pressure and temperature are slightly over-predicted by simulation (similar to other reported results in the literature). Likewise, higher-than expected thermal expansion coefficients and heat capacities are obtained from both ab initio and classical methods. The structure of both the crystal and melt are found to be in good agreement with experimental data. Several methodological improvements which could improve the accuracy of melting point determination and the thermal expansion coefficients are discussed

Computation in Complex Networks

Complex networks are one of the most challenging research focuses of disciplines, including physics, mathematics, biology, medicine, engineering, and computer science, among others. The interest in complex networks is increasingly growing, due to their ability to model several daily life systems, such as technology networks, the Internet, and communication, chemical, neural, social, political and financial networks. The Special Issue “Computation in Complex Networks" of Entropy offers a multidisciplinary view on how some complex systems behave, providing a collection of original and high-quality papers within the research fields of: • Community detection • Complex network modelling • Complex network analysis • Node classification • Information spreading and control • Network robustness • Social networks • Network medicin

The application of Machine Learning for Early Detection of At -Risk Learners in Massive Open Online Courses

With the rapid improvement of digital technology, Massive Open Online Courses (MOOCs) have emerged as powerful open educational learning platforms. MOOCs have been experiencing increased use and popularity in highly ranked universities in recent years. The opportunity to access high-quality courseware content within such platforms, while eliminating the burden of educational, financial and geographical obstacles has led to a growth in participant numbers. Despite the increasing participation in online courses, the low completion rate has raised major concerns in the literature. Identifying those students who are at-risk of dropping out could be a promising solution in solving the low completion rate in the online setting. Flagging at-risk students could assist the course instructors to bolster the struggling students and provide more learning resources. Although many prior studies have considered the dropout issue in the form of a sequence classification problem, such works only address a limited set of retention factors. They typically consider the learners’ activities as a sequence of weekly intervals, neglecting important learning trajectories. In this PhD thesis, my goal is to investigate retention factors. More specifically, the project seeks to explore the association of motivational trajectories, performance trajectories, engagement levels and latent engagement with the withdrawal rate. To achieve this goal, the first objective is to derive learners’ motivations based on Incentive Motivation theory. The Learning Analytic is utilised to classify student motivation into three main categories; Intrinsically motivated, Extrinsically motivated and Amotivation. Machine learning has been employed to detect the lack of motivation at early stages of the courses. The findings reveal that machine learning provides solutions that are capable of automatically identifying the students’ motivational status according to behaviourism theory. As the second and third objectives, three temporal dropout prediction models are proposed in this research work. The models provide dynamic assessment of the influence of the following factors; motivational trajectories, performance trajectories and latent engagement on students and the subsequent risk of them leaving the course. The models could assist the instructor in delivering more intensive intervention support to at-risk students. Supervised machine learning algorithms have been utilised in each model to identify the students who are in danger of dropping out in a timely manner. The results demonstrate that motivational trajectories and engagement levels are significant factors, which might influence the students’ withdrawal in online settings. On the other hand, the findings indicate that performance trajectories and latent engagement might not prevent students from completing online courses

Measurement of Detector-Corrected Cross-Sections in Events with Large Missing Transverse Momentum in Association with Jets

This thesis presents an analysis of events with large transverse missing momentum in association with jets using 139 fb−1 of proton-proton collisions at a centre of mass energy of 13 TeV , that was delivered at the Large Hadron Collider and recorded using the ATLAS detector from 2015 − 2018. The dominant process which contributes to these events is the Z boson decaying to two neutrinos followed closely by the contribution from W bosons decaying leptonically, in which the charged lepton is outside the detector acceptance. The similarity of these processes to Z and W bosons decaying leptonically can be exploited by measuring the one lepton and two lepton regions, and treating the leptons as invisible in order to constrain modelling along with the experimental and theoretical uncertainties. These lepton regions are known as the auxiliary regions. This analysis was performed using three different phase-spaces that are sensitive to different Dark Matter production channels. These three phase-spaces require the ≥ 1 jet , ≥ 2 jet and VBF topologies. The total yield is measured at the detector level and the differential cross section is measured at the particle level as a function of the missing transverse momentum, the dijet invariant mass and the dijet azimuthal angle. Ratios of these cross sections are also presented at the particle level to facilitate comparisons between regions and minimise the systematic uncertainties. These results are interpreted using a likelihood fit and the agree- ment between the modelling and the data is quantified using a χ2 test and p-value. Constraints are made on the axial-vector and pseu- doscalar simplified Dark Matter models and are shown to be competitive with results from the recent dedicated monojet-like search in ATLAS