9,532 research outputs found
Analysing spatial point patterns in digital pathology: immune cells in high-grade serous ovarian carcinomas
Multiplex immunofluorescence (mIF) imaging technology facilitates the study
of the tumour microenvironment in cancer patients. Due to the capabilities of
this emerging bioimaging technique, it is possible to statistically analyse,
for example, the co-varying location and functions of multiple different types
of immune cells. Complex spatial relationships between different immune cells
have been shown to correlate with patient outcomes and may reveal new pathways
for targeted immunotherapy treatments.
This tutorial reviews methods and procedures relating to spatial point
patterns for complex data analysis. We consider tissue cells as a realisation
of a spatial point process for each patient. We focus on proper functional
descriptors for each observation and techniques that allow us to obtain
information about inter-patient variation.
Ovarian cancer is the deadliest gynaecological malignancy and can resist
chemotherapy treatment effective in cancers. We use a dataset of high-grade
serous ovarian cancer samples from 51 patients. We examine the immune cell
composition (T cells, B cells, macrophages) within tumours and additional
information such as cell classification (tumour or stroma) and other patient
clinical characteristics. Our analyses, supported by reproducible software,
apply to other digital pathology datasets
Modular lifelong machine learning
Deep learning has drastically improved the state-of-the-art in many important fields, including computer vision and natural language processing (LeCun et al., 2015). However, it is expensive to train a deep neural network on a machine learning problem. The overall training cost further increases when one wants to solve additional problems. Lifelong machine learning (LML) develops algorithms that aim to efficiently learn to solve a sequence of problems, which become available one at a time. New problems are solved with less resources by transferring previously learned knowledge. At the same time, an LML algorithm needs to retain good performance on all encountered problems, thus avoiding catastrophic forgetting. Current approaches do not possess all the desired properties of an LML algorithm. First, they primarily focus on preventing catastrophic forgetting (Diaz-Rodriguez et al., 2018; Delange et al., 2021). As a result, they neglect some knowledge transfer properties. Furthermore, they assume that all problems in a sequence share the same input space. Finally, scaling these methods to a large sequence of problems remains a challenge.
Modular approaches to deep learning decompose a deep neural network into sub-networks, referred to as modules. Each module can then be trained to perform an atomic transformation, specialised in processing a distinct subset of inputs. This modular approach to storing knowledge makes it easy to only reuse the subset of modules which are useful for the task at hand.
This thesis introduces a line of research which demonstrates the merits of a modular approach to lifelong machine learning, and its ability to address the aforementioned shortcomings of other methods. Compared to previous work, we show that a modular approach can be used to achieve more LML properties than previously demonstrated. Furthermore, we develop tools which allow modular LML algorithms to scale in order to retain said properties on longer sequences of problems.
First, we introduce HOUDINI, a neurosymbolic framework for modular LML. HOUDINI represents modular deep neural networks as functional programs and accumulates a library of pre-trained modules over a sequence of problems. Given a new problem, we use program synthesis to select a suitable neural architecture, as well as a high-performing combination of pre-trained and new modules. We show that our approach has most of the properties desired from an LML algorithm. Notably, it can perform forward transfer, avoid negative transfer and prevent catastrophic forgetting, even across problems with disparate input domains and problems which require different neural architectures.
Second, we produce a modular LML algorithm which retains the properties of HOUDINI but can also scale to longer sequences of problems. To this end, we fix the choice of a neural architecture and introduce a probabilistic search framework, PICLE, for searching through different module combinations. To apply PICLE, we introduce two probabilistic models over neural modules which allows us to efficiently identify promising module combinations.
Third, we phrase the search over module combinations in modular LML as black-box optimisation, which allows one to make use of methods from the setting of hyperparameter optimisation (HPO). We then develop a new HPO method which marries a multi-fidelity approach with model-based optimisation. We demonstrate that this leads to improvement in anytime performance in the HPO setting and discuss how this can in turn be used to augment modular LML methods.
Overall, this thesis identifies a number of important LML properties, which have not all been attained in past methods, and presents an LML algorithm which can achieve all of them, apart from backward transfer
A generative flow for conditional sampling via optimal transport
Sampling conditional distributions is a fundamental task for Bayesian
inference and density estimation. Generative models, such as normalizing flows
and generative adversarial networks, characterize conditional distributions by
learning a transport map that pushes forward a simple reference (e.g., a
standard Gaussian) to a target distribution. While these approaches
successfully describe many non-Gaussian problems, their performance is often
limited by parametric bias and the reliability of gradient-based (adversarial)
optimizers to learn these transformations. This work proposes a non-parametric
generative model that iteratively maps reference samples to the target. The
model uses block-triangular transport maps, whose components are shown to
characterize conditionals of the target distribution. These maps arise from
solving an optimal transport problem with a weighted cost function,
thereby extending the data-driven approach in [Trigila and Tabak, 2016] for
conditional sampling. The proposed approach is demonstrated on a two
dimensional example and on a parameter inference problem involving nonlinear
ODEs.Comment: 18 pages, 5 figure
Automatic Feature Engineering for Time Series Classification: Evaluation and Discussion
Time Series Classification (TSC) has received much attention in the past two
decades and is still a crucial and challenging problem in data science and
knowledge engineering. Indeed, along with the increasing availability of time
series data, many TSC algorithms have been suggested by the research community
in the literature. Besides state-of-the-art methods based on similarity
measures, intervals, shapelets, dictionaries, deep learning methods or hybrid
ensemble methods, several tools for extracting unsupervised informative summary
statistics, aka features, from time series have been designed in the recent
years. Originally designed for descriptive analysis and visualization of time
series with informative and interpretable features, very few of these feature
engineering tools have been benchmarked for TSC problems and compared with
state-of-the-art TSC algorithms in terms of predictive performance. In this
article, we aim at filling this gap and propose a simple TSC process to
evaluate the potential predictive performance of the feature sets obtained with
existing feature engineering tools. Thus, we present an empirical study of 11
feature engineering tools branched with 9 supervised classifiers over 112 time
series data sets. The analysis of the results of more than 10000 learning
experiments indicate that feature-based methods perform as accurately as
current state-of-the-art TSC algorithms, and thus should rightfully be
considered further in the TSC literature
- …