Continual learning from stationary and non-stationary data

Continual learning aims at developing models that are capable of working on constantly evolving problems over a long-time horizon. In such environments, we can distinguish three essential aspects of training and maintaining machine learning models - incorporating new knowledge, retaining it and reacting to changes. Each of them poses its own challenges, constituting a compound problem with multiple goals. Remembering previously incorporated concepts is the main property of a model that is required when dealing with stationary distributions. In non-stationary environments, models should be capable of selectively forgetting outdated decision boundaries and adapting to new concepts. Finally, a significant difficulty can be found in combining these two abilities within a single learning algorithm, since, in such scenarios, we have to balance remembering and forgetting instead of focusing only on one aspect. The presented dissertation addressed these problems in an exploratory way. Its main goal was to grasp the continual learning paradigm as a whole, analyze its different branches and tackle identified issues covering various aspects of learning from sequentially incoming data. By doing so, this work not only filled several gaps in the current continual learning research but also emphasized the complexity and diversity of challenges existing in this domain. Comprehensive experiments conducted for all of the presented contributions have demonstrated their effectiveness and substantiated the validity of the stated claims

Application of Computational Intelligence Techniques to Process Industry Problems

In the last two decades there has been a large progress in the computational intelligence research field. The fruits of the effort spent on the research in the discussed field are powerful techniques for pattern recognition, data mining, data modelling, etc. These techniques achieve high performance on traditional data sets like the UCI machine learning database. Unfortunately, this kind of data sources usually represent clean data without any problems like data outliers, missing values, feature co-linearity, etc. common to real-life industrial data. The presence of faulty data samples can have very harmful effects on the models, for example if presented during the training of the models, it can either cause sub-optimal performance of the trained model or in the worst case destroy the so far learnt knowledge of the model. For these reasons the application of present modelling techniques to industrial problems has developed into a research field on its own. Based on the discussion of the properties and issues of the data and the state-of-the-art modelling techniques in the process industry, in this paper a novel unified approach to the development of predictive models in the process industry is presented

Addressing the balance between preventive and consequences-reducing measures regarding avoiding drifting into failure while increasing resilience in municipalities

On New Year’s Day 2020, Stavanger Municipality incorporated Finnøy and Rennesøy municipalities. With this merger came additional challenges for the risk management of the municipality. A comprehensive risk and vulnerability assessment (CRVA) was conducted to obtain a more accurate risk picture for the newly merged municipality. This assessment is used as a data source for investigating the main topic of this thesis: “Addressing the balance between preventive and consequences-reducing measures regarding avoiding drifting into failure while increasing resilience in municipalities.” In addition to the Stavanger CRVA (2019b), supporting theories and concepts on risk governance, risk management, barrier management strategies, barrier balance, systems thinking, and drifting into failure are described to assist in investigating the main topic. A qualitative research study consisting of two parts is conducted. The first area of interest is the new measures proposed in the Stavanger CRVA (2019b), resulting from their gap analysis, as access to all existing measures is unavailable. The authors have classified these measures for their preventive and consequence-reducing qualities. In addition, the measures have been classified by barrier element type; organizational, operational, technical, and citizen action. These classification results represent the municipality's distribution of measures, departmental accountability, and critical societal functions. The second area is a document search to support this research, including Norwegian laws and regulations, national publications, Stavanger municipality meeting minutes, and budget reports relevant to the Stavanger CRVA (2019b). Results from the data and document search, combined with the theory and concepts, are used to investigate the main topic, and answer the four research questions posed in this thesis: 1. What is the current distribution between the proposed preventive and consequence-reducing measures in Stavanger Municipality? 2. Should the measures balance be different from today, and if so, why? 3. How can the measures balance be adjusted to provide a better fit for Stavanger Municipality? and 4. Is a holistic approach useful for adjusting the balance in complex organizations? It was identified that most of the proposed measures in the Stavanger CRVA (2019b) were preventive. These measures do not represent the overall distribution of measures in Stavanger municipality since an overview of existing measures was unavailable. There is also uncertainty surrounding the implementation of the proposed measures. Most of these measures are identified as organizational using the Barrier Memorandum by the Petroleum Safety Authority of Norway as a guide. While it is challenging to address barrier balance in Stavanger Municipality, for many reasons described in this thesis, relevant observations have been made on the relations between balance, barriers, resilience, systems thinking and drifting into failure. A key finding is that barrier management used in a municipal setting can increase focus on barrier element types, their interactions, and viewing the system holistically. Another key finding is that focusing on the emerging properties of barrier interaction can lead to drifting into failure, but this can be avoided through increased focus on developing resilience in the system

On-the-fly tracing for data-centric computing : parallelization, workflow and applications

As data-centric computing becomes the trend in science and engineering, more and more hardware systems, as well as middleware frameworks, are emerging to handle the intensive computations associated with big data. At the programming level, it is crucial to have corresponding programming paradigms for dealing with big data. Although MapReduce is now a known programming model for data-centric computing where parallelization is completely replaced by partitioning the computing task through data, not all programs particularly those using statistical computing and data mining algorithms with interdependence can be re-factorized in such a fashion. On the other hand, many traditional automatic parallelization methods put an emphasis on formalism and may not achieve optimal performance with the given limited computing resources. In this work we propose a cross-platform programming paradigm, called on-the-fly data tracing , to provide source-to-source transformation where the same framework also provides the functionality of workflow optimization on larger applications. Using a big-data approximation computations related to large-scale data input are identified in the code and workflow and a simplified core dependence graph is built based on the computational load taking in to account big data. The code can then be partitioned into sections for efficient parallelization; and at the workflow level, optimization can be performed by adjusting the scheduling for big-data considerations, including the I/O performance of the machine. Regarding each unit in both source code and workflow as a model, this framework enables model-based parallel programming that matches the available computing resources. The techniques used in model-based parallel programming as well as the design of the software framework for both parallelization and workflow optimization as well as its implementations with multiple programming languages are presented in the dissertation. Then, the following experiments are performed to validate the framework: i) the benchmarking of parallelization speed-up using typical examples in data analysis and machine learning (e.g. naive Bayes, k-means) and ii) three real-world applications in data-centric computing with the framework are also described to illustrate the efficiency: pattern detection from hurricane and storm surge simulations, road traffic flow prediction and text mining from social media data. In the applications, it illustrates how to build scalable workflows with the framework along with performance enhancements

Finding common ground when experts disagree: robust portfolio decision analysis

We address the problem of decision making under “deep uncertainty,” introducing an approach we call Robust Portfolio Decision Analysis. We introduce the idea of Belief Dominance as a prescriptive operationalization of a concept that has appeared in the literature under a number of names. We use this concept to derive a set of non-dominated portfolios; and then identify robust individual alternatives from the non-dominated portfolios. The Belief Dominance concept allows us to synthesize multiple conflicting sources of information by uncovering the range of alternatives that are intelligent responses to the range of beliefs. This goes beyond solutions that are optimal for any specific set of beliefs to uncover defensible solutions that may not otherwise be revealed. We illustrate our approach using a problem in the climate change and energy policy context: choosing among clean energy technology R&D portfolios. We demonstrate how the Belief Dominance concept can uncover portfolios that would otherwise remain hidden and identify robust individual investments