6,844 research outputs found
Modular lifelong machine learning
Deep learning has drastically improved the state-of-the-art in many important fields, including computer vision and natural language processing (LeCun et al., 2015). However, it is expensive to train a deep neural network on a machine learning problem. The overall training cost further increases when one wants to solve additional problems. Lifelong machine learning (LML) develops algorithms that aim to efficiently learn to solve a sequence of problems, which become available one at a time. New problems are solved with less resources by transferring previously learned knowledge. At the same time, an LML algorithm needs to retain good performance on all encountered problems, thus avoiding catastrophic forgetting. Current approaches do not possess all the desired properties of an LML algorithm. First, they primarily focus on preventing catastrophic forgetting (Diaz-Rodriguez et al., 2018; Delange et al., 2021). As a result, they neglect some knowledge transfer properties. Furthermore, they assume that all problems in a sequence share the same input space. Finally, scaling these methods to a large sequence of problems remains a challenge.
Modular approaches to deep learning decompose a deep neural network into sub-networks, referred to as modules. Each module can then be trained to perform an atomic transformation, specialised in processing a distinct subset of inputs. This modular approach to storing knowledge makes it easy to only reuse the subset of modules which are useful for the task at hand.
This thesis introduces a line of research which demonstrates the merits of a modular approach to lifelong machine learning, and its ability to address the aforementioned shortcomings of other methods. Compared to previous work, we show that a modular approach can be used to achieve more LML properties than previously demonstrated. Furthermore, we develop tools which allow modular LML algorithms to scale in order to retain said properties on longer sequences of problems.
First, we introduce HOUDINI, a neurosymbolic framework for modular LML. HOUDINI represents modular deep neural networks as functional programs and accumulates a library of pre-trained modules over a sequence of problems. Given a new problem, we use program synthesis to select a suitable neural architecture, as well as a high-performing combination of pre-trained and new modules. We show that our approach has most of the properties desired from an LML algorithm. Notably, it can perform forward transfer, avoid negative transfer and prevent catastrophic forgetting, even across problems with disparate input domains and problems which require different neural architectures.
Second, we produce a modular LML algorithm which retains the properties of HOUDINI but can also scale to longer sequences of problems. To this end, we fix the choice of a neural architecture and introduce a probabilistic search framework, PICLE, for searching through different module combinations. To apply PICLE, we introduce two probabilistic models over neural modules which allows us to efficiently identify promising module combinations.
Third, we phrase the search over module combinations in modular LML as black-box optimisation, which allows one to make use of methods from the setting of hyperparameter optimisation (HPO). We then develop a new HPO method which marries a multi-fidelity approach with model-based optimisation. We demonstrate that this leads to improvement in anytime performance in the HPO setting and discuss how this can in turn be used to augment modular LML methods.
Overall, this thesis identifies a number of important LML properties, which have not all been attained in past methods, and presents an LML algorithm which can achieve all of them, apart from backward transfer
Improving diagnostic procedures for epilepsy through automated recording and analysis of patients’ history
Transient loss of consciousness (TLOC) is a time-limited state of profound cognitive impairment characterised by amnesia, abnormal motor control, loss of responsiveness, a short duration and complete recovery. Most instances of TLOC are caused by one of three health conditions: epilepsy, functional (dissociative) seizures (FDS), or syncope. There is often a delay before the correct diagnosis is made and 10-20% of individuals initially receive an incorrect diagnosis. Clinical decision tools based on the endorsement of TLOC symptom lists have been limited to distinguishing between two causes of TLOC. The Initial Paroxysmal Event Profile (iPEP) has shown promise but was demonstrated to have greater accuracy in distinguishing between syncope and epilepsy or FDS than between epilepsy and FDS. The objective of this thesis was to investigate whether interactional, linguistic, and communicative differences in how people with epilepsy and people with FDS describe their experiences of TLOC can improve the predictive performance of the iPEP. An online web application was designed that collected information about TLOC symptoms and medical history from patients and witnesses using a binary questionnaire and verbal interaction with a virtual agent. We explored potential methods of automatically detecting these communicative differences, whether the differences were present during an interaction with a VA, to what extent these automatically detectable communicative differences improve the performance of the iPEP, and the acceptability of the application from the perspective of patients and witnesses. The two feature sets that were applied to previous doctor-patient interactions, features designed to measure formulation effort or detect semantic differences between the two groups, were able to predict the diagnosis with an accuracy of 71% and 81%, respectively. Individuals with epilepsy or FDS provided descriptions of TLOC to the VA that were qualitatively like those observed in previous research. Both feature sets were effective predictors of the diagnosis when applied to the web application recordings (85.7% and 85.7%). Overall, the accuracy of machine learning models trained for the threeway classification between epilepsy, FDS, and syncope using the iPEP responses from patients that were collected through the web application was worse than the performance observed in previous research (65.8% vs 78.3%), but the performance was increased by the inclusion of features extracted from the spoken descriptions on TLOC (85.5%). Finally, most participants who provided feedback reported that the online application was acceptable. These findings suggest that it is feasible to differentiate between people with epilepsy and people with FDS using an automated analysis of spoken seizure descriptions. Furthermore, incorporating these features into a clinical decision tool for TLOC can improve the predictive performance by improving the differential diagnosis between these two health conditions. Future research should use the feedback to improve the design of the application and increase perceived acceptability of the approach
Modelling, Monitoring, Control and Optimization for Complex Industrial Processes
This reprint includes 22 research papers and an editorial, collected from the Special Issue "Modelling, Monitoring, Control and Optimization for Complex Industrial Processes", highlighting recent research advances and emerging research directions in complex industrial processes. This reprint aims to promote the research field and benefit the readers from both academic communities and industrial sectors
A Decision Support System for Economic Viability and Environmental Impact Assessment of Vertical Farms
Vertical farming (VF) is the practice of growing crops or animals using the vertical dimension via multi-tier racks or vertically inclined surfaces. In this thesis, I focus on the emerging industry of plant-specific VF. Vertical plant farming (VPF) is a promising and relatively novel practice that can be conducted in buildings with environmental control and artificial lighting. However, the nascent sector has experienced challenges in economic viability, standardisation, and environmental sustainability. Practitioners and academics call for a comprehensive financial analysis of VPF, but efforts are stifled by a lack of valid and available data.
A review of economic estimation and horticultural software identifies a need for a decision support system (DSS) that facilitates risk-empowered business planning for vertical farmers. This thesis proposes an open-source DSS framework to evaluate business sustainability through financial risk and environmental impact assessments. Data from the literature, alongside lessons learned from industry practitioners, would be centralised in the proposed DSS using imprecise data techniques. These techniques have been applied in engineering but are seldom used in financial forecasting. This could benefit complex sectors which only have scarce data to predict business viability.
To begin the execution of the DSS framework, VPF practitioners were interviewed using a mixed-methods approach. Learnings from over 19 shuttered and operational VPF projects provide insights into the barriers inhibiting scalability and identifying risks to form a risk taxonomy. Labour was the most commonly reported top challenge. Therefore, research was conducted to explore lean principles to improve productivity.
A probabilistic model representing a spectrum of variables and their associated uncertainty was built according to the DSS framework to evaluate the financial risk for VF projects. This enabled flexible computation without precise production or financial data to improve economic estimation accuracy. The model assessed two VPF cases (one in the UK and another in Japan), demonstrating the first risk and uncertainty quantification of VPF business models in the literature. The results highlighted measures to improve economic viability and the viability of the UK and Japan case.
The environmental impact assessment model was developed, allowing VPF operators to evaluate their carbon footprint compared to traditional agriculture using life-cycle assessment. I explore strategies for net-zero carbon production through sensitivity analysis. Renewable energies, especially solar, geothermal, and tidal power, show promise for reducing the carbon emissions of indoor VPF. Results show that renewably-powered VPF can reduce carbon emissions compared to field-based agriculture when considering the land-use change.
The drivers for DSS adoption have been researched, showing a pathway of compliance and design thinking to overcome the ‘problem of implementation’ and enable commercialisation. Further work is suggested to standardise VF equipment, collect benchmarking data, and characterise risks. This work will reduce risk and uncertainty and accelerate the sector’s emergence
Decoding spatial location of attended audio-visual stimulus with EEG and fNIRS
When analyzing complex scenes, humans often focus their attention on an object at a particular spatial location in the presence of background noises and irrelevant visual objects. The ability to decode the attended spatial location would facilitate brain computer interfaces (BCI) for complex scene analysis. Here, we tested two different neuroimaging technologies and investigated their capability to decode audio-visual spatial attention in the presence of competing stimuli from multiple locations. For functional near-infrared spectroscopy (fNIRS), we targeted dorsal frontoparietal network including frontal eye field (FEF) and intra-parietal sulcus (IPS) as well as superior temporal gyrus/planum temporal (STG/PT). They all were shown in previous functional magnetic resonance imaging (fMRI) studies to be activated by auditory, visual, or audio-visual spatial tasks. We found that fNIRS provides robust decoding of attended spatial locations for most participants and correlates with behavioral performance. Moreover, we found that FEF makes a large contribution to decoding performance. Surprisingly, the performance was significantly above chance level 1s after cue onset, which is well before the peak of the fNIRS response.
For electroencephalography (EEG), while there are several successful EEG-based algorithms, to date, all of them focused exclusively on auditory modality where eye-related artifacts are minimized or controlled. Successful integration into a more ecological typical usage requires careful consideration for eye-related artifacts which are inevitable. We showed that fast and reliable decoding can be done with or without ocular-removal algorithm. Our results show that EEG and fNIRS are promising platforms for compact, wearable technologies that could be applied to decode attended spatial location and reveal contributions of specific brain regions during complex scene analysis
Aerial Network Assistance Systems for Post-Disaster Scenarios : Topology Monitoring and Communication Support in Infrastructure-Independent Networks
Communication anytime and anywhere is necessary for our modern society to function. However, the critical network infrastructure quickly fails in the face of a disaster and leaves the affected population without means of communication. This lack can be overcome by smartphone-based emergency communication systems, based on infrastructure-independent networks like Delay-Tolerant Networks (DTNs). DTNs, however, suffer from short device-to-device link distances and, thus, require multi-hop routing or data ferries between disjunct parts of the network. In disaster scenarios, this fragmentation is particularly severe because of the highly clustered human mobility behavior. Nevertheless, aerial communication support systems can connect local network clusters by utilizing Unmanned Aerial Vehicles (UAVs) as data ferries. To facilitate situation-aware and adaptive communication support, knowledge of the network topology, the identification of missing communication links, and the constant reassessment of dynamic disasters are required. These requirements are usually neglected, despite existing approaches to aerial monitoring systems capable of detecting devices and networks.
In this dissertation, we, therefore, facilitate the coexistence of aerial topology monitoring and communications support mechanisms in an autonomous Aerial Network Assistance System for infrastructure-independent networks as our first contribution. To enable system adaptations to unknown and dynamic disaster situations, our second contribution addresses the collection, processing, and utilization of topology information. For one thing, we introduce cooperative monitoring approaches to include the DTN in the monitoring process. Furthermore, we apply novel approaches for data aggregation and network cluster estimation to facilitate the continuous assessment of topology information and an appropriate system adaptation. Based on this, we introduce an adaptive topology-aware routing approach to reroute UAVs and increase the coverage of disconnected nodes outside clusters.
We generalize our contributions by integrating them into a simulation framework, creating an evaluation platform for autonomous aerial systems as our third contribution. We further increase the expressiveness of our aerial system evaluation, by adding movement models for multicopter aircraft combined with power consumption models based on real-world measurements. Additionally, we improve the disaster simulation by generalizing civilian disaster mobility based on a real-world field test. With a prototypical system implementation, we extensively evaluate our contributions and show the significant benefits of cooperative monitoring and topology-aware routing, respectively. We highlight the importance of continuous and integrated topology monitoring for aerial communications support and demonstrate its necessity for an adaptive and long-term disaster deployment. In conclusion, the contributions of this dissertation enable the usage of autonomous Aerial Network Assistance Systems and their adaptability in dynamic disaster scenarios
The place where curses are manufactured : four poets of the Vietnam War
The Vietnam War was unique among American wars. To pinpoint its uniqueness, it was necessary to look for a non-American voice that would enable me to articulate its distinctiveness and explore the American character as observed by an Asian. Takeshi Kaiko proved to be most helpful. From his novel, Into a Black Sun, I was able to establish a working pair of 'bookends' from which to approach the poetry of Walter McDonald, Bruce Weigl, Basil T. Paquet and Steve Mason. Chapter One is devoted to those seemingly mismatched 'bookends,' Walt Whitman and General William C. Westmoreland, and their respective anthropocentric and technocentric visions of progress and the peculiarly American concept of the "open road" as they manifest themselves in Vietnam. In Chapter, Two, I analyze the war poems of Walter McDonald. As a pilot, writing primarily about flying, his poetry manifests General Westmoreland's technocentric vision of the 'road' as determined by and manifest through technology. Chapter Three focuses on the poems of Bruce Weigl. The poems analyzed portray the literal and metaphorical descent from the technocentric, 'numbed' distance of aerial warfare to the world of ground warfare, and the initiation of a 'fucking new guy,' who discovers the contours of the self's interior through a set of experiences that lead from from aerial insertion into the jungle to the degradation of burning human
feces. Chapter Four, devoted to the thirteen poems of Basil T. Paquet, focuses on the continuation of the descent begun in Chapter Two. In his capacity as a medic, Paquet's entire body of poems details his quotidian tasks which entail tending the maimed, the mortally wounded and the dead. The final chapter deals with Steve Mason's JohnnY's Song, and his depiction of the plight of Vietnam veterans back in "The World" who are still trapped inside the interior landscape of their individual "ghettoes" of the soul created by their war-time experiences
Large-Scale Landslide Susceptibility Mapping Using an Integrated Machine Learning Model: A Case Study in the Lvliang Mountains of China
Integration of different models may improve the performance of landslide susceptibility assessment, but few studies have tested it. The present study aims at exploring the way to integrating different models and comparing the results among integrated and individual models. Our objective is to answer this question: Will the integrated model have higher accuracy compared with individual model? The Lvliang mountains area, a landslide-prone area in China, was taken as the study area, and ten factors were considered in the influencing factors system. Three basic machine learning models (the back propagation (BP), support vector machine (SVM), and random forest (RF) models) were integrated by an objective function where the weight coefficients among different models were computed by the gray wolf optimization (GWO) algorithm. 80 and 20% of the landslide data were randomly selected as the training and testing samples, respectively, and different landslide susceptibility maps were generated based on the GIS platform. The results illustrated that the accuracy expressed by the area under the receiver operating characteristic curve (AUC) of the BP-SVM-RF integrated model was the highest (0.7898), which was better than that of the BP (0.6929), SVM (0.6582), RF (0.7258), BP-SVM (0.7360), BP-RF (0.7569), and SVM-RF models (0.7298). The experimental results authenticated the effectiveness of the BP-SVM-RF method, which can be a reliable model for the regional landslide susceptibility assessment of the study area. Moreover, the proposed procedure can be a good option to integrate different models to seek an "optimal" result. Keywords: landslide susceptibility, random forest, integrated model, causal factor, GI
A hybrid model using data mining and multi-criteria decision-making methods for landslide risk mapping at Golestan Province, Iran
The accurate modeling of landslide risk is essential pre-requisite for the development of reliable landslide control and mitigation strategies. However, landslide risk depends on the poorly known environmental and socio-economic factors for regional patterns of landslide occurrence probability and vulnerability, which constitute still a matter of research. Here, a hybrid model is described that couples data mining and multi-criteria decision-making methods for hazard and vulnerability mapping and presents its application to landslide risk assessment in Golestan Province, Northeastern Iran. To this end, landslide probability is mapped using three state-of-the-art machine learning (ML) algorithms—Maximum Entropy, Support Vector Machine and Genetic Algorithm for Rule Set Production—and combine the results with Fuzzy Analytical Hierarchy Process computations of vulnerability to obtain the landslide risk map. Based on obtained results, a discussion is presented on landslide probability as a function of the main relevant human-environmental conditioning factors in Golestan Province. In particular, from the response curves of the machine learning algorithms, it can be found that the probability p of landslide occurrence decreases nearly exponentially with the distance x to the next road, fault, or river. Specifically, the results indicated that p≈exp(−λx) where the length scale λ is about 0.0797 km−1 for road, 0.108 km−1 for fault, and 0.734 km−1 0.734 km−1 for river. Furthermore, according to the results, p follows, approximately, a lognormal function of elevation, while the equation p=p0−K(θ−θ0)2 fits well the dependence of landslide modeling on the slope-angle θ, with p0≈0.64,θ0≈25.6∘and|K|≈6.6×10−4. However, the highest predicted landslide risk levels in Golestan Province are located in the south and southwest areas surrounding Gorgan City, owing to the combined effect of dense local human occupation and strongly landslide-prone environmental conditions. Obtained results provide insights for quantitative modeling of landslide risk, as well as for priority planning in landslide risk management
Computing Interpretable Representations of Cell Morphodynamics
Shape changes (morphodynamics) are one of the principal ways cells interact with their environments and perform key intrinsic behaviours like division. These dynamics arise from a myriad of complex signalling pathways that often organise with emergent simplicity to carry out critical functions including predation, collaboration and migration. A powerful method for analysis can therefore be to quantify this emergent structure, bypassing the low-level complexity. Enormous image datasets are now available to mine. However, it can be difficult to uncover interpretable representations of the global organisation of these heterogeneous dynamic processes. Here, such representations were developed for interpreting morphodynamics in two key areas: mode of action (MoA) comparison for drug discovery (developed using the economically devastating Asian soybean rust crop pathogen) and 3D migration of immune system T cells through extracellular matrices (ECMs). For MoA comparison, population development over a 2D space of shapes (morphospace) was described using two models with condition-dependent parameters: a top-down model of diffusive development over Waddington-type landscapes, and a bottom-up model of tip growth. A variety of landscapes were discovered, describing phenotype transitions during growth, and possible perturbations in the tip growth machinery that cause this variation were identified. For interpreting T cell migration, a new 3D shape descriptor that incorporates key polarisation information was developed, revealing low-dimensionality of shape, and the distinct morphodynamics of run-and-stop modes that emerge at minute timescales were mapped. Periodically oscillating morphodynamics that include retrograde deformation flows were found to underlie active translocation (run mode). Overall, it was found that highly interpretable representations could be uncovered while still leveraging the enormous discovery power of deep learning algorithms. The results show that whole-cell morphodynamics can be a convenient and powerful place to search for structure, with potentially life-saving applications in medicine and biocide discovery as well as immunotherapeutics.Open Acces
- …