Response Characterization for Auditing Cell Dynamics in Long Short-term Memory Networks

In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level. We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods. The ranked contribution of individual cells to the network's output is computed by analyzing a set of interpretable metrics of their decoupled step and sinusoidal responses. As a result, our method is able to uniquely identify neurons with insightful dynamics, quantify relationships between dynamical properties and test accuracy through ablation analysis, and interpret the impact of network capacity on a network's dynamical distribution. Finally, we demonstrate generalizability and scalability of our method by evaluating a series of different benchmark sequential datasets

The Strategic Content of Island Constraints

This issue of Working Papers in Linguistics is a slightly modified version of Alexander Grosu's Ph.D. dissertation from The Ohio State University (submitted to the Graduate School in August, 1972)

PMLR

We propose a neural information processing system obtained by re-purposing the function of a biological neural circuit model to govern simulated and real-world control tasks. Inspired by the structure of the nervous system of the soil-worm, C. elegans, we introduce ordinary neural circuits (ONCs), defined as the model of biological neural circuits reparameterized for the control of alternative tasks. We first demonstrate that ONCs realize networks with higher maximum flow compared to arbitrary wired networks. We then learn instances of ONCs to control a series of robotic tasks, including the autonomous parking of a real-world rover robot. For reconfiguration of the purpose of the neural circuit, we adopt a search-based optimization algorithm. Ordinary neural circuits perform on par and, in some cases, significantly surpass the performance of contemporary deep learning models. ONC networks are compact, 77% sparser than their counterpart neural controllers, and their neural dynamics are fully interpretable at the cell-level

Liquid Time-constant Networks

We introduce a new class of time-continuous recurrent neural network models. Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of linear first-order dynamical systems modulated via nonlinear interlinked gates. The resulting models represent dynamical systems with varying (i.e., liquid) time-constants coupled to their hidden state, with outputs being computed by numerical differential equation solvers. These neural networks exhibit stable and bounded behavior, yield superior expressivity within the family of neural ordinary differential equations, and give rise to improved performance on time-series prediction tasks. To demonstrate these properties, we first take a theoretical approach to find bounds over their dynamics and compute their expressive power by the trajectory length measure in latent trajectory space. We then conduct a series of time-series prediction experiments to manifest the approximation capability of Liquid Time-Constant Networks (LTCs) compared to classical and modern RNNs. Code and data are available at https://github.com/raminmh/liquid_time_constant_networksComment: Accepted to the Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21

Hippocampus-Avoidance Whole-Brain Radiation Therapy Is Efficient in the Long-Term Preservation of Hippocampal Volume

Background and Purpose: With improved life expectancy, preventing neurocognitive decline after cerebral radiotherapy is gaining more importance. Hippocampal damage has been considered the main culprit for cognitive deficits following conventional whole-brain radiation therapy (WBRT). Here, we aimed to determine to which extent hippocampus-avoidance WBRT (HA-WBRT) can prevent hippocampal atrophy compared to conventional WBRT. Methods and Materials: Thirty-five HA-WBRT and 48 WBRT patients were retrospectively selected, comprising a total of 544 contrast-enhanced T1-weighted magnetic resonance imaging studies, longitudinally acquired within 24 months before and 48 months after radiotherapy. HA-WBRT patients were treated analogously to the ongoing HIPPORAD-trial (DRKS00004598) protocol with 30 Gy in 12 fractions and dose to 98% of the hippocampus ≤ 9 Gy and to 2% ≤ 17 Gy. WBRT was mainly performed with 35 Gy in 14 fractions or 30 Gy in 10 fractions. Anatomical images were segmented and the hippocampal volume was quantified using the Computational Anatomy Toolbox (CAT), including neuroradiological expert review of the segmentations. Results: After statistically controlling for confounding variables such as age, gender, and total intracranial volume, hippocampal atrophy was found after both WBRT and HA-WBRT (p Conclusion: HA-WBRT is a therapeutic option for patients with multiple brain metastases, which can effectively and durably minimize hippocampal atrophy compared to conventional WBRT

PSMA-PET based radiotherapy: a review of initial experiences, survey on current practice and future perspectives

Gallium prostate specific membrane antigen (PSMA) ligand positron emission tomography (PET) is an increasingly used imaging modality in prostate cancer, especially in cases of tumor recurrence after curative intended therapy. Owed to the novelty of the PSMA-targeting tracers, clinical evidence on the value of PSMA-PET is moderate but rapidly increasing. State of the art imaging is pivotal for radiotherapy treatment planning as it may affect dose prescription, target delineation and use of concomitant therapy. This review summarizes the evidence on PSMA-PET imaging from a radiation oncologist’s point of view. Additionally a short survey containing twelve examples of patients and 6 additional questions was performed in seven mayor academic centers with experience in PSMA ligand imaging and the findings are reported here

A modal ambiguity in for-infinitival relative clauses

This squib presents two puzzles related to an ambiguity found in for-infinitival relative clauses (FIRs). FIRs invariably receive a modal interpretation even in the absence of any overt modal verb. The modal interpretation seems to come in two distinct types, which can be paraphrased by finite relative clauses employing the modal auxiliaries should and could. The two puzzles presented here arise because the availability of the two readings is constrained by factors that are not otherwise known to affect the interpretation of a relative clause. Specifically, we show, first, that “strong” determiners require the FIR to be interpreted as a SHOULD-relative while “weak” determiners allow both interpretations (the Determiner-Modal Generalization). Secondly, we observe that the COULD-interpretation requires a raising (internally headed) structure for the FIR, while the SHOULD-interpretation is compatible with either a raising or a more standard matching (externally headed) structure (the Raising/Matching Generalization)