6,055 research outputs found
Southern Adventist University Undergraduate Catalog 2023-2024
Southern Adventist University\u27s undergraduate catalog for the academic year 2023-2024.https://knowledge.e.southern.edu/undergrad_catalog/1123/thumbnail.jp
Modular lifelong machine learning
Deep learning has drastically improved the state-of-the-art in many important fields, including computer vision and natural language processing (LeCun et al., 2015). However, it is expensive to train a deep neural network on a machine learning problem. The overall training cost further increases when one wants to solve additional problems. Lifelong machine learning (LML) develops algorithms that aim to efficiently learn to solve a sequence of problems, which become available one at a time. New problems are solved with less resources by transferring previously learned knowledge. At the same time, an LML algorithm needs to retain good performance on all encountered problems, thus avoiding catastrophic forgetting. Current approaches do not possess all the desired properties of an LML algorithm. First, they primarily focus on preventing catastrophic forgetting (Diaz-Rodriguez et al., 2018; Delange et al., 2021). As a result, they neglect some knowledge transfer properties. Furthermore, they assume that all problems in a sequence share the same input space. Finally, scaling these methods to a large sequence of problems remains a challenge.
Modular approaches to deep learning decompose a deep neural network into sub-networks, referred to as modules. Each module can then be trained to perform an atomic transformation, specialised in processing a distinct subset of inputs. This modular approach to storing knowledge makes it easy to only reuse the subset of modules which are useful for the task at hand.
This thesis introduces a line of research which demonstrates the merits of a modular approach to lifelong machine learning, and its ability to address the aforementioned shortcomings of other methods. Compared to previous work, we show that a modular approach can be used to achieve more LML properties than previously demonstrated. Furthermore, we develop tools which allow modular LML algorithms to scale in order to retain said properties on longer sequences of problems.
First, we introduce HOUDINI, a neurosymbolic framework for modular LML. HOUDINI represents modular deep neural networks as functional programs and accumulates a library of pre-trained modules over a sequence of problems. Given a new problem, we use program synthesis to select a suitable neural architecture, as well as a high-performing combination of pre-trained and new modules. We show that our approach has most of the properties desired from an LML algorithm. Notably, it can perform forward transfer, avoid negative transfer and prevent catastrophic forgetting, even across problems with disparate input domains and problems which require different neural architectures.
Second, we produce a modular LML algorithm which retains the properties of HOUDINI but can also scale to longer sequences of problems. To this end, we fix the choice of a neural architecture and introduce a probabilistic search framework, PICLE, for searching through different module combinations. To apply PICLE, we introduce two probabilistic models over neural modules which allows us to efficiently identify promising module combinations.
Third, we phrase the search over module combinations in modular LML as black-box optimisation, which allows one to make use of methods from the setting of hyperparameter optimisation (HPO). We then develop a new HPO method which marries a multi-fidelity approach with model-based optimisation. We demonstrate that this leads to improvement in anytime performance in the HPO setting and discuss how this can in turn be used to augment modular LML methods.
Overall, this thesis identifies a number of important LML properties, which have not all been attained in past methods, and presents an LML algorithm which can achieve all of them, apart from backward transfer
Seamless Multimodal Biometrics for Continuous Personalised Wellbeing Monitoring
Artificially intelligent perception is increasingly present in the lives of
every one of us. Vehicles are no exception, (...) In the near future, pattern
recognition will have an even stronger role in vehicles, as self-driving cars
will require automated ways to understand what is happening around (and within)
them and act accordingly. (...) This doctoral work focused on advancing
in-vehicle sensing through the research of novel computer vision and pattern
recognition methodologies for both biometrics and wellbeing monitoring. The
main focus has been on electrocardiogram (ECG) biometrics, a trait well-known
for its potential for seamless driver monitoring. Major efforts were devoted to
achieving improved performance in identification and identity verification in
off-the-person scenarios, well-known for increased noise and variability. Here,
end-to-end deep learning ECG biometric solutions were proposed and important
topics were addressed such as cross-database and long-term performance,
waveform relevance through explainability, and interlead conversion. Face
biometrics, a natural complement to the ECG in seamless unconstrained
scenarios, was also studied in this work. The open challenges of masked face
recognition and interpretability in biometrics were tackled in an effort to
evolve towards algorithms that are more transparent, trustworthy, and robust to
significant occlusions. Within the topic of wellbeing monitoring, improved
solutions to multimodal emotion recognition in groups of people and
activity/violence recognition in in-vehicle scenarios were proposed. At last,
we also proposed a novel way to learn template security within end-to-end
models, dismissing additional separate encryption processes, and a
self-supervised learning approach tailored to sequential data, in order to
ensure data security and optimal performance. (...)Comment: Doctoral thesis presented and approved on the 21st of December 2022
to the University of Port
Analog Photonics Computing for Information Processing, Inference and Optimisation
This review presents an overview of the current state-of-the-art in photonics
computing, which leverages photons, photons coupled with matter, and
optics-related technologies for effective and efficient computational purposes.
It covers the history and development of photonics computing and modern
analogue computing platforms and architectures, focusing on optimization tasks
and neural network implementations. The authors examine special-purpose
optimizers, mathematical descriptions of photonics optimizers, and their
various interconnections. Disparate applications are discussed, including
direct encoding, logistics, finance, phase retrieval, machine learning, neural
networks, probabilistic graphical models, and image processing, among many
others. The main directions of technological advancement and associated
challenges in photonics computing are explored, along with an assessment of its
efficiency. Finally, the paper discusses prospects and the field of optical
quantum computing, providing insights into the potential applications of this
technology.Comment: Invited submission by Journal of Advanced Quantum Technologies;
accepted version 5/06/202
Algorithms for Geometric Facility Location: Centers in a Polygon and Dispersion on a Line
We study three geometric facility location problems in this thesis.
First, we consider the dispersion problem in one dimension. We are given an ordered list
of (possibly overlapping) intervals on a line. We wish to choose exactly one point from
each interval such that their left to right ordering on the line matches the input order.
The aim is to choose the points so that the distance between the closest pair of points is
maximized, i.e., they must be socially distanced while respecting the order. We give a new
linear-time algorithm for this problem that produces a lexicographically optimal solution.
We also consider some generalizations of this problem.
For the next two problems, the domain of interest is a simple polygon with n vertices.
The second problem concerns the visibility center. The convention is to think of a polygon
as the top view of a building (or art gallery) where the polygon boundary represents opaque
walls. Two points in the domain are visible to each other if the line segment joining them
does not intersect the polygon exterior. The distance to visibility from a source point to a
target point is the minimum geodesic distance from the source to a point in the polygon
visible to the target. The question is: Where should a single guard be located within the
polygon to minimize the maximum distance to visibility? For m point sites in the polygon,
we give an O((m + n) log (m + n)) time algorithm to determine their visibility center.
Finally, we address the problem of locating the geodesic edge center of a simple polygon—a
point in the polygon that minimizes the maximum geodesic distance to any edge. For a
triangle, this point coincides with its incenter. The geodesic edge center is a generalization
of the well-studied geodesic center (a point that minimizes the maximum distance to any
vertex). Center problems are closely related to farthest Voronoi diagrams, which are well-
studied for point sites in the plane, and less well-studied for line segment sites in the plane.
When the domain is a polygon rather than the whole plane, only the case of point sites has
been addressed—surprisingly, more general sites (with line segments being the simplest
example) have been largely ignored. En route to our solution, we revisit, correct, and
generalize (sometimes in a non-trivial manner) existing algorithms and structures tailored
to work specifically for point sites. We give an optimal linear-time algorithm for finding
the geodesic edge center of a simple polygon
Swift: A modern highly-parallel gravity and smoothed particle hydrodynamics solver for astrophysical and cosmological applications
Numerical simulations have become one of the key tools used by theorists in
all the fields of astrophysics and cosmology. The development of modern tools
that target the largest existing computing systems and exploit state-of-the-art
numerical methods and algorithms is thus crucial. In this paper, we introduce
the fully open-source highly-parallel, versatile, and modular coupled
hydrodynamics, gravity, cosmology, and galaxy-formation code Swift. The
software package exploits hybrid task-based parallelism, asynchronous
communications, and domain-decomposition algorithms based on balancing the
workload, rather than the data, to efficiently exploit modern high-performance
computing cluster architectures. Gravity is solved for using a
fast-multipole-method, optionally coupled to a particle mesh solver in Fourier
space to handle periodic volumes. For gas evolution, multiple modern flavours
of Smoothed Particle Hydrodynamics are implemented. Swift also evolves
neutrinos using a state-of-the-art particle-based method. Two complementary
networks of sub-grid models for galaxy formation as well as extensions to
simulate planetary physics are also released as part of the code. An extensive
set of output options, including snapshots, light-cones, power spectra, and a
coupling to structure finders are also included. We describe the overall code
architecture, summarize the consistency and accuracy tests that were performed,
and demonstrate the excellent weak-scaling performance of the code using a
representative cosmological hydrodynamical problem with billion
particles. The code is released to the community alongside extensive
documentation for both users and developers, a large selection of example test
problems, and a suite of tools to aid in the analysis of large simulations run
with Swift.Comment: 39 pages, 18 figures, submitted to MNRAS. Code, documentation, and
examples available at www.swiftsim.co
A New Deterministic Algorithm for Fully Dynamic All-Pairs Shortest Paths
We study the fully dynamic All-Pairs Shortest Paths (APSP) problem in
undirected edge-weighted graphs. Given an -vertex graph with
non-negative edge lengths, that undergoes an online sequence of edge insertions
and deletions, the goal is to support approximate distance queries and
shortest-path queries. We provide a deterministic algorithm for this problem,
that, for a given precision parameter , achieves approximation factor
, and has amortized update time
per operation, where is the ratio of longest to
shortest edge length. Query time for distance-query is
, and query time for
shortest-path query is , where is the path that the algorithm returns. To the best of our
knowledge, even allowing any -approximation factor, no adaptive-update
algorithms with better than amortized update time and better than
query time were known prior to this work. We also note that our
guarantees are stronger than the best current guarantees for APSP in
decremental graphs in the adaptive-adversary setting.Comment: arXiv admin note: text overlap with arXiv:2109.0562
- …