Universal Approximation with Deep Narrow Networks

The classical Universal Approximation Theorem holds for neural networks of arbitrary width and bounded depth. Here we consider the natural `dual' scenario for networks of bounded width and arbitrary depth. Precisely, let $n$ be the number of inputs neurons, $m$ be the number of output neurons, and let $\rho$ be any nonaffine continuous function, with a continuous nonzero derivative at some point. Then we show that the class of neural networks of arbitrary depth, width $n + m + 2$, and activation function $\rho$, is dense in $C(K; \mathbb{R}^m)$ for $K \subseteq \mathbb{R}^n$ with $K$ compact. This covers every activation function possible to use in practice, and also includes polynomial activation functions, which is unlike the classical version of the theorem, and provides a qualitative difference between deep narrow networks and shallow wide networks. We then consider several extensions of this result. In particular we consider nowhere differentiable activation functions, density in noncompact domains with respect to the $L^p$-norm, and how the width may be reduced to just $n + m + 1$ for `most' activation functions.Comment: Accepted at COLT 202

Photometric monitoring of the blazar 3C 345 for the period 1996 - 2006

We present the results of the blazar 3C 345 monitoring in Johnson-Cousins BVRI bands for the period 1996 - 2006. We have collected 29 V and 43 R data points for this period; the BI light curves contain a few measurements only. The accuracy of our photometry is not better than 0.03 mag in the VR bands. The total amplitude of the variability obtained from our data is 2.06 mag in the V band and 2.25 mag in the R one. 3C 345 showed periods of flaring activity during 1998/99 and 2001: a maximum of the blazar brightness was detected in 2001 February - 15.345 mag in the V band and 14.944 mag in the R one. We confirm that during brighter stages 3C 345 becomes redder; for higher fluxes the colour index seems to be less dependent on the magnitude. The intra-night monitoring of 3C 345 in three consecutive nights in 2001 August revealed no significant intra-night variability; 3C 345 did not show evident flux changes over timescales of weeks around the period of the intra-night monitoring. This result supports the existing facts that intra-night variability is correlated with rapid flux changes rather than with specific flux levels

Modulation of nerve growth factor receptor expression in the urothelium and its relevance to ketmine induced cystitis

Ketamine-induced cystitis (KIC) is a form of bladder pain syndrome (BPS) that usually arises following recreational abuse. Urothelial expression of the p75 low-affinity nerve growth factor receptor (NGFR) has been implicated in the aetiopathology of BPS. The aim of this project was to examine the expression of NGFR in urothelium and KIC and to identify modulators of NGFR expression. From the literature, candidate modulators identified were: ketamine, the transcriptional regulator Early growth response (Egr-1), glucocorticoids, cytokines, brain derived neurotrophic factor (BDNF) and ibuprofen, a non-steroidal anti-inflammatory drug.Immunoperoxidase labelling and semi-quantitative analysis was performed on normal and KIC surgical specimens for NGFR and Egr-1. Cell and organ culture systems were developed from which NGFR transcript and protein expression were assessed following exposure to the candidate modulators.NGFR was expressed by basal cells in all normal urothelial samples and extended suprabasally in KIC samples. A case study revealed widespread urothelial destruction and a non-patent urachal remnant with normal NGFR basal expression, in comparison to NGFR suprabasal extension in Von Brunn’s nests, which were patent with the luminal surface of the bladder. Using an in vitro approach NGFR transcript was most highly expressed by proliferating normal human urothelial cell cultures and was immunolocalised basally in differentiated cell sheets and organ cultures. Glucocorticoids, cytokines, BDNF and Egr-1 did not modulate NGFR expression. Cell death inducing concentrations of ibuprofen, ketamine or G418 up-regulated NGFR transcript and protein in cell and organ cultures.This study supports a urinary-mediated urothelial damage process in KIC and implicates NGFR upregulation in the pathogenesis. Ketamine and other toxins are able to directly up-regulate expression of NGFR in the urothelium and this study has laid the foundation for future exploration of the role of NGFR in the urothelial damage response

Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPU

Signatory is a library for calculating and performing functionality related to the signature and logsignature transforms. The focus is on machine learning, and as such includes features such as CPU parallelism, GPU support, and backpropagation. To our knowledge it is the first GPU-capable library for these operations. Signatory implements new features not available in previous libraries, such as efficient precomputation strategies. Furthermore, several novel algorithmic improvements are introduced, producing substantial real-world speedups even on the CPU without parallelism. The library operates as a Python wrapper around C++, and is compatible with the PyTorch ecosystem. It may be installed directly via \texttt{pip}. Source code, documentation, examples, benchmarks and tests may be found at \texttt{\url{https://github.com/patrick-kidger/signatory}}. The license is Apache-2.0.Comment: Published at ICLR 202

Lineax: unified linear solves and linear least-squares in JAX and Equinox

We introduce Lineax, a library bringing linear solves and linear least-squares to the JAX+Equinox scientific computing ecosystem. Lineax uses general linear operators, and unifies linear solves and least-squares into a single, autodifferentiable API. Solvers and operators are user-extensible, without requiring the user to implement any custom derivative rules to get differentiability. Lineax is available at https://github.com/google/lineax.Comment: 7 pages, 1 figure, NeurIPS 2023 AI for Science worksho

Generalised Interpretable Shapelets for Irregular Time Series

The shapelet transform is a form of feature extraction for time series, in which a time series is described by its similarity to each of a collection of `shapelets'. However it has previously suffered from a number of limitations, such as being limited to regularly-spaced fully-observed time series, and having to choose between efficient training and interpretability. Here, we extend the method to continuous time, and in doing so handle the general case of irregularly-sampled partially-observed multivariate time series. Furthermore, we show that a simple regularisation penalty may be used to train efficiently without sacrificing interpretability. The continuous-time formulation additionally allows for learning the length of each shapelet (previously a discrete object) in a differentiable manner. Finally, we demonstrate that the measure of similarity between time series may be generalised to a learnt pseudometric. We validate our method by demonstrating its performance and interpretability on several datasets; for example we discover (purely from data) that the digits 5 and 6 may be distinguished by the chirality of their bottom loop, and that a kind of spectral gap exists in spoken audio classification

Herschel PACS and SPIRE observations of blazar PKS 1510-089: a case for two blazar zones

We present the results of observations of blazar PKS 1510-089 with the Herschel Space Observatory PACS and SPIRE instruments, together with multiwavelength data from Fermi/LAT, Swift, SMARTS and SMA. The source was found in a quiet state, and its far-infrared spectrum is consistent with a power-law with a spectral index of alpha ~ 0.7. Our Herschel observations were preceded by two 'orphan' gamma-ray flares. The near-infrared data reveal the high-energy cut-off in the main synchrotron component, which cannot be associated with the main gamma-ray component in a one-zone leptonic model. This is because in such a model the luminosity ratio of the External-Compton and synchrotron components is tightly related to the frequency ratio of these components, and in this particular case an unrealistically high energy density of the external radiation would be implied. Therefore, we consider a well-constrained two-zone blazar model to interpret the entire dataset. In this framework, the observed infrared emission is associated with the synchrotron component produced in the hot-dust region at the supra-pc scale, while the gamma-ray emission is associated with the External-Compton component produced in the broad-line region at the sub-pc scale. In addition, the optical/UV emission is associated with the accretion disk thermal emission, with the accretion disk corona likely contributing to the X-ray emission.Comment: 13 pages, 8 figures, 7 tables; accepted for publication in the Astrophysical Journa