Learning Unitary Operators with Help From u(n)

A major challenge in the training of recurrent neural networks is the so-called vanishing or exploding gradient problem. The use of a norm-preserving transition operator can address this issue, but parametrization is challenging. In this work we focus on unitary operators and describe a parametrization using the Lie algebra $\mathfrak{u}(n)$ associated with the Lie group $U(n)$ of $n \times n$ unitary matrices. The exponential map provides a correspondence between these spaces, and allows us to define a unitary matrix using $n^2$ real coefficients relative to a basis of the Lie algebra. The parametrization is closed under additive updates of these coefficients, and thus provides a simple space in which to do gradient descent. We demonstrate the effectiveness of this parametrization on the problem of learning arbitrary unitary operators, comparing to several baselines and outperforming a recently-proposed lower-dimensional parametrization. We additionally use our parametrization to generalize a recently-proposed unitary recurrent neural network to arbitrary unitary matrices, using it to solve standard long-memory tasks.Comment: 9 pages, 3 figures, 5 figures inc. subfigures, to appear at AAAI-1

A Generative Model of Words and Relationships from Multiple Sources

Neural language models are a powerful tool to embed words into semantic vector spaces. However, learning such models generally relies on the availability of abundant and diverse training examples. In highly specialised domains this requirement may not be met due to difficulties in obtaining a large corpus, or the limited range of expression in average use. Such domains may encode prior knowledge about entities in a knowledge base or ontology. We propose a generative model which integrates evidence from diverse data sources, enabling the sharing of semantic information. We achieve this by generalising the concept of co-occurrence from distributional semantics to include other relationships between entities or words, which we model as affine transformations on the embedding space. We demonstrate the effectiveness of this approach by outperforming recent models on a link prediction task and demonstrating its ability to profit from partially or fully unobserved data training labels. We further demonstrate the usefulness of learning from different data sources with overlapping vocabularies.Comment: 8 pages, 5 figures; incorporated feedback from reviewers; to appear in Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence 201

RAD-DINO: Exploring Scalable Medical Image Encoders Beyond Text Supervision

Language-supervised pre-training has proven to be a valuable method for extracting semantically meaningful features from images, serving as a foundational element in multimodal systems within the computer vision and medical imaging domains. However, resulting features are limited by the information contained within the text. This is particularly problematic in medical imaging, where radiologists' written findings focus on specific observations; a challenge compounded by the scarcity of paired imaging-text data due to concerns over leakage of personal health information. In this work, we fundamentally challenge the prevailing reliance on language supervision for learning general purpose biomedical imaging encoders. We introduce RAD-DINO, a biomedical image encoder pre-trained solely on unimodal biomedical imaging data that obtains similar or greater performance than state-of-the-art biomedical language supervised models on a diverse range of benchmarks. Specifically, the quality of learned representations is evaluated on standard imaging tasks (classification and semantic segmentation), and a vision-language alignment task (text report generation from images). To further demonstrate the drawback of language supervision, we show that features from RAD-DINO correlate with other medical records (e.g., sex or age) better than language-supervised models, which are generally not mentioned in radiology reports. Finally, we conduct a series of ablations determining the factors in RAD-DINO's performance; notably, we observe that RAD-DINO's downstream performance scales well with the quantity and diversity of training data, demonstrating that image-only supervision is a scalable approach for training a foundational biomedical image encoder

Striatal Dopamine Transmission Is Subtly Modified in Human A53Tα-Synuclein Overexpressing Mice

Mutations in, or elevated dosage of, SNCA, the gene for α-synuclein (α-syn), cause familial Parkinson's disease (PD). Mouse lines overexpressing the mutant human A53Tα-syn may represent a model of early PD. They display progressive motor deficits, abnormal cellular accumulation of α-syn, and deficits in dopamine-dependent corticostriatal plasticity, which, in the absence of overt nigrostriatal degeneration, suggest there are age-related deficits in striatal dopamine (DA) signalling. In addition A53Tα-syn overexpression in cultured rodent neurons has been reported to inhibit transmitter release. Therefore here we have characterized for the first time DA release in the striatum of mice overexpressing human A53Tα-syn, and explored whether A53Tα-syn overexpression causes deficits in the release of DA. We used fast-scan cyclic voltammetry to detect DA release at carbon-fibre microelectrodes in acute striatal slices from two different lines of A53Tα-syn-overexpressing mice, at up to 24 months. In A53Tα-syn overexpressors, mean DA release evoked by a single stimulus pulse was not different from wild-types, in either dorsal striatum or nucleus accumbens. However the frequency responsiveness of DA release was slightly modified in A53Tα-syn overexpressors, and in particular showed slight deficiency when the confounding effects of striatal ACh acting at presynaptic nicotinic receptors (nAChRs) were antagonized. The re-release of DA was unmodified after single-pulse stimuli, but after prolonged stimulation trains, A53Tα-syn overexpressors showed enhanced recovery of DA release at old age, in keeping with elevated striatal DA content. In summary, A53Tα-syn overexpression in mice causes subtle changes in the regulation of DA release in the striatum. While modest, these modifications may indicate or contribute to striatal dysfunction