Transfer Learning for Neural Semantic Parsing

The goal of semantic parsing is to map natural language to a machine interpretable meaning representation language (MRL). One of the constraints that limits full exploration of deep learning technologies for semantic parsing is the lack of sufficient annotation training data. In this paper, we propose using sequence-to-sequence in a multi-task setup for semantic parsing with a focus on transfer learning. We explore three multi-task architectures for sequence-to-sequence modeling and compare their performance with an independently trained model. Our experiments show that the multi-task setup aids transfer learning from an auxiliary task with large labeled data to a target task with smaller labeled data. We see absolute accuracy gains ranging from 1.0% to 4.4% in our in- house data set, and we also see good gains ranging from 2.5% to 7.0% on the ATIS semantic parsing tasks with syntactic and semantic auxiliary tasks.Comment: Accepted for ACL Repl4NLP 201

Deterministische parallele Verfahren zur numerischen Lösung der räumlich inhomogenen Boltzmann-Gleichung

Das Ziel dieser Arbeit war die Bereitstellung neuer numerischer Methoden für die Boltzmann-Gleichung, die die folgenden Merkmale besitzen: Sie sollten bezüglich der Rechenzeit in einem mit den zur Zeit dominierenden stochastischen Verfahren (Monte Carlo methods) vergleichbaren Rahmen liegen, aber eine wesentlich höhere Genauigkeit aufweisen. Ein paralleles Verfahren für die räumlich inhomogene Boltzmann-Gleichung, das auf einer Operator-Splitting-Technik basiert, wird vorgestellt. Die Transportphase wird durch die bekannten expliziten und impliziten Upwind-Differenzenverfahren und Essentially-Non-Oscillatory-(ENO)- bzw. Weighted-Essentially-Non-Oscillatory-(WENO)-Verfahren in ein und zwei Raumdimensionen modelliert. Der Kollisionsschritt wird mit einer effizienten merischen Berechnung einer Approximation des Kollisionsintegrals auf einem gleichmäßigen Geschwindigkeitsgitter behandelt, welche auf einer Darstellung des Kollisionsintegrals beruht, die in [32] entwickelt wurde. Das Simulationsgebiet ist beschränkt und wir werden die deterministische numerische Approximation verschiedener Randbedingungen, wie z.B. diffuse Reflexion oder Spiegelreflexion, diskutieren. Weiter wird ein neues Beispiel für Streukerne mit Adsorptionszeitabhängigkeit gegeben. Die Operator-Splitting-Technik erlaubt eine effiziente Parallelisierung und die zugehörigen numerischen Ergebnisse, die wir für das Wärmeleitungsproblem zwischen parallelen Platten (in einer und zwei Raumdimensionen) im Falle des Maxwell-Pseudo-Molekül-Kollisionsmodells wie auch für die Simulation eines Schockprofils, wobei hier das BGK-Kollisionsmodell genutzt wird, erhalten, werden vorgestellt.This work was aimed at providing new deterministic numerical methods for the Boltzmann equation equipped with the following features: they should have a computational cost comparable with prevailing stochastic algorithms (Monte Carlo methods) while producing superior numerical results with respect to accuracy. A parallel scheme for the spatially non-homogeneous Boltzmann equation based on an operator splitting method is presented. The transport step is carried out using the well-known explicit and implicit upwind difference schemes and the Essentially Non-Oscillatory (ENO) or Weighted Essentially Non-Oscillatory schemes (WENO) in one or two space dimensions. The collision step is treated by efficent approximate computation of the collision integral on the uniform grid based on the representation of the collision integral developed in [32]. The spatial domain of simulation is bounded and deterministic numerical approximations of different types of boundary conditions, e.g. diffuse or specular reflection, are discussed. Furthermore, a new example for scattering kernels with adsorption time dependence is given. The operator splitting method allows an efficient parallelization and the numerical results obtained for the heat transfer problem between two parallel plates (in one and two space dimensions) in the case of Maxwell pseudo-molecules as well as for the simulation of shock profiles using the BGK collision model are presented

Deterministische parallele Verfahren zur numerischen Lösung der räumlich inhomogenen Boltzmann-Gleichung

Das Ziel dieser Arbeit war die Bereitstellung neuer numerischer Methoden für die Boltzmann-Gleichung, die die folgenden Merkmale besitzen: Sie sollten bezüglich der Rechenzeit in einem mit den zur Zeit dominierenden stochastischen Verfahren (Monte Carlo methods) vergleichbaren Rahmen liegen, aber eine wesentlich höhere Genauigkeit aufweisen. Ein paralleles Verfahren für die räumlich inhomogene Boltzmann-Gleichung, das auf einer Operator-Splitting-Technik basiert, wird vorgestellt. Die Transportphase wird durch die bekannten expliziten und impliziten Upwind-Differenzenverfahren und Essentially-Non-Oscillatory-(ENO)- bzw. Weighted-Essentially-Non-Oscillatory-(WENO)-Verfahren in ein und zwei Raumdimensionen modelliert. Der Kollisionsschritt wird mit einer effizienten merischen Berechnung einer Approximation des Kollisionsintegrals auf einem gleichmäßigen Geschwindigkeitsgitter behandelt, welche auf einer Darstellung des Kollisionsintegrals beruht, die in [32] entwickelt wurde. Das Simulationsgebiet ist beschränkt und wir werden die deterministische numerische Approximation verschiedener Randbedingungen, wie z.B. diffuse Reflexion oder Spiegelreflexion, diskutieren. Weiter wird ein neues Beispiel für Streukerne mit Adsorptionszeitabhängigkeit gegeben. Die Operator-Splitting-Technik erlaubt eine effiziente Parallelisierung und die zugehörigen numerischen Ergebnisse, die wir für das Wärmeleitungsproblem zwischen parallelen Platten (in einer und zwei Raumdimensionen) im Falle des Maxwell-Pseudo-Molekül-Kollisionsmodells wie auch für die Simulation eines Schockprofils, wobei hier das BGK-Kollisionsmodell genutzt wird, erhalten, werden vorgestellt.This work was aimed at providing new deterministic numerical methods for the Boltzmann equation equipped with the following features: they should have a computational cost comparable with prevailing stochastic algorithms (Monte Carlo methods) while producing superior numerical results with respect to accuracy. A parallel scheme for the spatially non-homogeneous Boltzmann equation based on an operator splitting method is presented. The transport step is carried out using the well-known explicit and implicit upwind difference schemes and the Essentially Non-Oscillatory (ENO) or Weighted Essentially Non-Oscillatory schemes (WENO) in one or two space dimensions. The collision step is treated by efficent approximate computation of the collision integral on the uniform grid based on the representation of the collision integral developed in [32]. The spatial domain of simulation is bounded and deterministic numerical approximations of different types of boundary conditions, e.g. diffuse or specular reflection, are discussed. Furthermore, a new example for scattering kernels with adsorption time dependence is given. The operator splitting method allows an efficient parallelization and the numerical results obtained for the heat transfer problem between two parallel plates (in one and two space dimensions) in the case of Maxwell pseudo-molecules as well as for the simulation of shock profiles using the BGK collision model are presented

Advances in Sympathetic Nerve Recording in Humans

In humans, sympathetic activity is commonly assessed by measuring the efferent traffic in the peroneal nerve. The firing activity is the sum of several active neurons, which have the tendency to fire together in a bursting manner. While the estimation of overall sympathetic nervous activity using this multiunit recording approach has advanced our understanding of sympathetic regulation in health and disease no information is gained regarding the underling mechanisms generating the bursts of sympathetic activity. The introduction of single-unit recording has been a major step forward, enabling the examination of specific sympathetic firing patterns in diverse clinical conditions. Disturbances in sympathetic nerve firing, including high firing probabilities, high firing rates or high incidence of multiple firing, or a combination of both may impact on noradrenaline release and effector response, and therefore have clinical implications with regards to the development and progression of target organ damage. Understanding the mechanisms and consequences of specific firing patterns would permit the development of therapeutic strategies targeting these nuances of sympathetic overdrive

Effects of Renal Denervation on Sympathetic Activation, Blood Pressure, and Glucose Metabolism in Patients with Resistant Hypertension

Increased central sympathetic drive is a hallmark of several important clinical conditions including essential hypertension, heart failure, chronic kidney disease, and insulin resistance. Afferent signaling from the kidneys has been identified as an important contributor to elevated central sympathetic drive and increased sympathetic outflow to the kidney and other organs is crucially involved in cardiovascular control. While the resultant effects on renal hemodynamic parameters, sodium and water retention, and renin release are particularly relevant for both acute and long term regulation of blood pressure, increased sympathetic outflow to other vascular beds may facilitate further adverse consequences of sustained sympathetic activation such as insulin resistance, which is commonly associated with hypertension. Recent clinical studies using catheter-based radiofrequency ablation technology to achieve functional renal denervation in patients with resistant hypertension have identified the renal nerves as therapeutic target and have helped to further expose the sympathetic link between hypertension and insulin resistance. Initial data from two clinical trials and several smaller mechanistic clinical studies indicate that this novel approach may indeed provide a safe and effective treatment alternative for resistant hypertension and some of its adverse consequences

Change in Sympathetic Nerve Firing Pattern Associated with Dietary Weight Loss in the Metabolic Syndrome

Sympathetic activation in subjects with the metabolic syndrome (MS) plays a role in the pathogenesis of cardiovascular disease development. Diet-induced weight loss decreases sympathetic outflow. However the mechanisms that account for sympathetic inhibition are not known. We sought to provide a detailed description of the sympathetic response to diet by analyzing the firing behavior of single-unit sympathetic nerve fibers. Fourteen subjects (57 ± 2 years, nine men, five females) fulfilling ATP III criteria for the MS underwent a 3-month low calorie diet. Metabolic profile, hemodynamic parameters, and multi-unit and single-unit muscle sympathetic nerve activity (MSNA, microneurography) were assessed prior to and at the end of the diet. Patients’ weight dropped from 96 ± 4 to 88 ± 3 kg (P < 0.001). This was associated with a decrease in systolic and diastolic blood pressure (−12 ± 3 and −5 ± 2 mmHg, P < 0.05), and in heart rate (−7 ± 2 bpm, P < 0.01) and an improvement in all metabolic parameters (fasting glucose: −0.302.1 ± 0.118 mmol/l, total cholesterol: −0.564 ± 0.164 mmol/l, triglycerides: −0.414 ± 0.137 mmol/l, P < 0.05). Multi-unit MSNA decreased from 68 ± 4 to 59 ± 5 bursts/100 heartbeats (P < 0.05). Single-unit MSNA indicated that the firing rate of individual vasoconstrictor fibers decreased from 59 ± 10 to 32 ± 4 spikes/100 heart beats (P < 0.05). The probability of firing decreased from 34 ± 5 to 23 ± 3% of heartbeats (P < 0.05), and the incidence of multiple firing decreased from 14 ± 4 to 6 ± 1% of heartbeats (P < 0.05). Cardiac and sympathetic baroreflex function were significantly improved (cardiac slope: 6.57 ± 0.69 to 9.57 ± 1.20 ms·mmHg−1; sympathetic slope: −3.86 ± 0.34 to −5.05 ± 0.47 bursts/100 heartbeats·mmHg−1, P < 0.05 for both). Hypocaloric diet decreased sympathetic activity and improved hemodynamic and metabolic parameters. The sympathoinhibition associated with weight loss involves marked changes, not only in the rate but also in the firing pattern of active vasoconstrictive fibers

Longitudinal patterns of leukoaraiosis and brain atrophy in symptomatic small vessel disease.

Cerebral small vessel disease is a common condition associated with lacunar stroke, cognitive impairment and significant functional morbidity. White matter hyperintensities and brain atrophy, seen on magnetic resonance imaging, are correlated with increasing disease severity. However, how the two are related remains an open question. To better define the relationship between white matter hyperintensity growth and brain atrophy, we applied a semi-automated magnetic resonance imaging segmentation analysis pipeline to a 3-year longitudinal cohort of 99 subjects with symptomatic small vessel disease, who were followed-up for ≥1 years. Using a novel two-stage warping pipeline with tissue repair step, voxel-by-voxel rate of change maps were calculated for each tissue class (grey matter, white matter, white matter hyperintensities and lacunes) for each individual. These maps capture both the distribution of disease and spatial information showing local rates of growth and atrophy. These were analysed to answer three primary questions: first, is there a relationship between whole brain atrophy and magnetic resonance imaging markers of small vessel disease (white matter hyperintensities or lacune volume)? Second, is there regional variation within the cerebral white matter in the rate of white matter hyperintensity progression? Finally, are there regionally specific relationships between the rates of white matter hyperintensity progression and cortical grey matter atrophy? We demonstrate that the rates of white matter hyperintensity expansion and grey matter atrophy are strongly correlated (Pearson's R = -0.69, P < 1 × 10(-7)), and significant grey matter loss and whole brain atrophy occurs annually (P < 0.05). Additionally, the rate of white matter hyperintensity growth was heterogeneous, occurring more rapidly within long association fasciculi. Using voxel-based quantification (family-wise error corrected P < 0.05), we show the rate of white matter hyperintensity progression is associated with increases in cortical grey matter atrophy rates, in the medial-frontal, orbito-frontal, parietal and occipital regions. Conversely, increased rates of global grey matter atrophy are significantly associated with faster white matter hyperintensity growth in the frontal and parietal regions. Together, these results link the progression of white matter hyperintensities with increasing rates of regional grey matter atrophy, and demonstrate that grey matter atrophy is the major contributor to whole brain atrophy in symptomatic cerebral small vessel disease. These measures provide novel insights into the longitudinal pathogenesis of small vessel disease, and imply that therapies aimed at reducing progression of white matter hyperintensities via end-arteriole damage may protect against secondary brain atrophy and consequent functional morbidity

Diffusion tensor image segmentation of the cerebrum provides a single measure of cerebral small vessel disease severity related to cognitive change.

Cerebral small vessel disease (SVD) is the primary cause of vascular cognitive impairment and is associated with decline in executive function (EF) and information processing speed (IPS). Imaging biomarkers are needed that can monitor and identify individuals at risk of severe cognitive decline. Recently there has been interest in combining several magnetic resonance imaging (MRI) markers of SVD into a unitary score to describe disease severity. Here we apply a diffusion tensor image (DTI) segmentation technique (DSEG) to describe SVD related changes in a single unitary score across the whole cerebrum, to investigate its relationship with cognitive change over a three-year period. 98 patients (aged 43-89) with SVD underwent annual MRI scanning and cognitive testing for up to three years. DSEG provides a vector of 16 discrete segments describing brain microstructure of healthy and/or damaged tissue. By calculating the scalar product of each DSEG vector in reference to that of a healthy ageing control we generate an angular measure (DSEG θ) describing the patients' brain tissue microstructural similarity to a disease free model of a healthy ageing brain. Conventional MRI markers of SVD brain change were also assessed including white matter hyperintensities, cerebral atrophy, incident lacunes, cerebral-microbleeds, and white matter microstructural damage measured by DTI histogram parameters. The impact of brain change on cognition was explored using linear mixed-effects models. Post-hoc sample size analysis was used to assess the viability of DSEG θ as a tool for clinical trials. Changes in brain structure described by DSEG θ were related to change in EF and IPS (p < 0.001) and remained significant in multivariate models including other MRI markers of SVD as well as age, gender and premorbid IQ. Of the conventional markers, presence of new lacunes was the only marker to remain a significant predictor of change in EF and IPS in the multivariate models (p = 0.002). Change in DSEG θ was also related to change in all other MRI markers (p < 0.017), suggesting it may be used as a surrogate marker of SVD damage across the cerebrum. Sample size estimates indicated that fewer patients would be required to detect treatment effects using DSEG θ compared to conventional MRI and DTI markers of SVD severity. DSEG θ is a powerful tool for characterising subtle brain change in SVD that has a negative impact on cognition and remains a significant predictor of cognitive change when other MRI markers of brain change are accounted for. DSEG provides an automatic segmentation of the whole cerebrum that is sensitive to a range of SVD related structural changes and successfully predicts cognitive change. Power analysis shows DSEG θ has potential as a monitoring tool in clinical trials. As such it may provide a marker of SVD severity from a single imaging modality (i.e. DTIs)

Longitudinal decline in structural networks predicts dementia in cerebral small vessel disease.

OBJECTIVE: To determine whether longitudinal change in white matter structural network integrity predicts dementia and future cognitive decline in cerebral small vessel disease (SVD). To investigate whether network disruption has a causal role in cognitive decline and mediates the association between conventional MRI markers of SVD with both cognitive decline and dementia. METHODS: In the prospective longitudinal SCANS (St George's Cognition and Neuroimaging in Stroke) Study, 97 dementia-free individuals with symptomatic lacunar stroke were followed with annual MRI for 3 years and annual cognitive assessment for 5 years. Conversion to dementia was recorded. Structural networks were constructed from diffusion tractography using a longitudinal registration pipeline, and network global efficiency was calculated. Linear mixed-effects regression was used to assess change over time. RESULTS: Seventeen individuals (17.5%) converted to dementia, and significant decline in global cognition occurred (p = 0.0016). Structural network measures declined over the 3-year MRI follow-up, but the degree of change varied markedly between individuals. The degree of reductions in network global efficiency was associated with conversion to dementia (B = -2.35, odds ratio = 0.095, p = 0.00056). Change in network global efficiency mediated much of the association of conventional MRI markers of SVD with cognitive decline and progression to dementia. CONCLUSIONS: Network disruption has a central role in the pathogenesis of cognitive decline and dementia in SVD. It may be a useful disease marker to identify that subgroup of patients with SVD who progress to dementia