Representational Capabilities of Feed-forward and Sequential Neural Architectures

Despite the widespread empirical success of deep neural networks over the past decade, a comprehensive understanding of their mathematical properties remains elusive, which limits the abilities of practitioners to train neural networks in a principled manner. This dissertation provides a representational characterization of a variety of neural network architectures, including fully-connected feed-forward networks and sequential models like transformers. The representational capabilities of neural networks are most famously characterized by the universal approximation theorem, which states that sufficiently large neural networks can closely approximate any well-behaved target function. However, the universal approximation theorem applies exclusively to two-layer neural networks of unbounded size and fails to capture the comparative strengths and weaknesses of different architectures. The thesis addresses these limitations by quantifying the representational consequences of random features, weight regularization, and model depth on feed-forward architectures. It further investigates and contrasts the expressive powers of transformers and other sequential neural architectures. Taken together, these results apply a wide range of theoretical tools—including approximation theory, discrete dynamical systems, and communication complexity—to prove rigorous separations between different neural architectures and scaling regimes

Learning Single-Index Models with Shallow Neural Networks

Single-index models are a class of functions given by an unknown univariate ``link'' function applied to an unknown one-dimensional projection of the input. These models are particularly relevant in high dimension, when the data might present low-dimensional structure that learning algorithms should adapt to. While several statistical aspects of this model, such as the sample complexity of recovering the relevant (one-dimensional) subspace, are well-understood, they rely on tailored algorithms that exploit the specific structure of the target function. In this work, we introduce a natural class of shallow neural networks and study its ability to learn single-index models via gradient flow. More precisely, we consider shallow networks in which biases of the neurons are frozen at random initialization. We show that the corresponding optimization landscape is benign, which in turn leads to generalization guarantees that match the near-optimal sample complexity of dedicated semi-parametric methods.Comment: 76 pages. To appear at NeurIPS 202

Improving the predictions of ML-corrected climate models with novelty detection

While previous works have shown that machine learning (ML) can improve the prediction accuracy of coarse-grid climate models, these ML-augmented methods are more vulnerable to irregular inputs than the traditional physics-based models they rely on. Because ML-predicted corrections feed back into the climate model's base physics, the ML-corrected model regularly produces out of sample data, which can cause model instability and frequent crashes. This work shows that adding semi-supervised novelty detection to identify out-of-sample data and disable the ML-correction accordingly stabilizes simulations and sharply improves the quality of predictions. We design an augmented climate model with a one-class support vector machine (OCSVM) novelty detector that provides better temperature and precipitation forecasts in a year-long simulation than either a baseline (no-ML) or a standard ML-corrected run. By improving the accuracy of coarse-grid climate models, this work helps make accurate climate models accessible to researchers without massive computational resources.Comment: Appearing at Tackling Climate Change with Machine Learning Workshop at NeurIPS 202

Predominant expression of Alzheimer’s disease-associated BIN1 in mature oligodendrocytes and localization to white matter tracts

BIN1 is not expressed in human brain microglial cells. (A) Immunohistochemical staining of adjacent sections of normal human brain cortex with antibodies against BIN1 or Iba1 reveals that BIN1 immunoreactive cells that are morphologically distinct from microglia. The boxed region is shown at a higher magnification on the right. (B) Single and two-color immunostaining of the human brain using antibodies against BIN1 and CD45 reveals that perivenular CD45-positive cells of the hematopoietic lineage do not express BIN1. (TIFF 4392 kb

Mutations in SLC39A14 disrupt manganese homeostasis and cause childhood-onset parkinsonism-dystonia

Although manganese is an essential trace metal, little is known about its transport and homeostatic regulation. Here we have identified a cohort of patients with a novel autosomal recessive manganese transporter defect caused by mutations in SLC39A14. Excessive accumulation of manganese in these patients results in rapidly progressive childhood-onset parkinsonism-dystonia with distinctive brain magnetic resonance imaging appearances and neurodegenerative features on post-mortem examination. We show that mutations in SLC39A14 impair manganese transport in vitro and lead to manganese dyshomeostasis and altered locomotor activity in zebrafish with CRISPR-induced slc39a14 null mutations. Chelation with disodium calcium edetate lowers blood manganese levels in patients and can lead to striking clinical improvement. Our results demonstrate that SLC39A14 functions as a pivotal manganese transporter in vertebrates

Mutations in SLC39A14 disrupt manganese homeostasis and cause childhood-onset parkinsonism-dystonia.

Although manganese is an essential trace metal, little is known about its transport and homeostatic regulation. Here we have identified a cohort of patients with a novel autosomal recessive manganese transporter defect caused by mutations in SLC39A14. Excessive accumulation of manganese in these patients results in rapidly progressive childhood-onset parkinsonism-dystonia with distinctive brain magnetic resonance imaging appearances and neurodegenerative features on post-mortem examination. We show that mutations in SLC39A14 impair manganese transport in vitro and lead to manganese dyshomeostasis and altered locomotor activity in zebrafish with CRISPR-induced slc39a14 null mutations. Chelation with disodium calcium edetate lowers blood manganese levels in patients and can lead to striking clinical improvement. Our results demonstrate that SLC39A14 functions as a pivotal manganese transporter in vertebrates.Action Medical ResearchThis is the final version of the article. It first appeared from Nature Publishing Group via http://dx.doi.org/10.1038/ncomms1160

Service planning and delivery outcomes of home adaptations for ageing in the UK

In response to the impact of demographic change on the healthcare system, ‘ageing in place’ was introduced as a national policy to support elderly people living independently in their homes. Housing adaptation is essential for successful independent living and has been given increased political priority. However, adaptation policies and practice vary regionally, reflecting statutory limits, policy choices and local planning. This study investigated the current status of adaptation provision in different regions in the UK and assessed the effectiveness of local service planning and management. A mixed-methods sequential explanatory research strategy was employed. In the first quantitative phase, a questionnaire survey was carried out involving all 378 local authorities in England, Scotland and Wales. This was followed by a second qualitative phase involving individual interviews with five professionals and two clients and a focus group meeting with six key stakeholders. The study found that the current number of adaptations was relatively small compared with potential demands in most local areas, as was funding for adaptations. On the operational side, the adaptation process was fragmented, involving different service groups in many local authorities. There were disconnections between these groups, which often caused inefficiencies and poor effectiveness. Moving forward, local authorities need to have a clear vision of the overall need for adaptations and allocate sufficient resources. Practical guidelines are also needed for better integrated working and performance management