Probing LLMs for Joint Encoding of Linguistic Categories

Large Language Models (LLMs) exhibit impressive performance on a range of NLP tasks, due to the general-purpose linguistic knowledge acquired during pretraining. Existing model interpretability research (Tenney et al., 2019) suggests that a linguistic hierarchy emerges in the LLM layers, with lower layers better suited to solving syntactic tasks and higher layers employed for semantic processing. Yet, little is known about how encodings of different linguistic phenomena interact within the models and to what extent processing of linguistically-related categories relies on the same, shared model representations. In this paper, we propose a framework for testing the joint encoding of linguistic categories in LLMs. Focusing on syntax, we find evidence of joint encoding both at the same (related part-of-speech (POS) classes) and different (POS classes and related syntactic dependency relations) levels of linguistic hierarchy. Our cross-lingual experiments show that the same patterns hold across languages in multilingual LLMs.</p

Melody Generation using an Interactive Evolutionary Algorithm

Music generation with the aid of computers has been recently grabbed the attention of many scientists in the area of artificial intelligence. Deep learning techniques have evolved sequence production methods for this purpose. Yet, a challenging problem is how to evaluate generated music by a machine. In this paper, a methodology has been developed based upon an interactive evolutionary optimization method, with which the scoring of the generated melodies is primarily performed by human expertise, during the training. This music quality scoring is modeled using a Bi-LSTM recurrent neural network. Moreover, the innovative generated melody through a Genetic algorithm will then be evaluated using this Bi-LSTM network. The results of this mechanism clearly show that the proposed method is able to create pleasurable melodies with desired styles and pieces. This method is also quite fast, compared to the state-of-the-art data-oriented evolutionary systems.Comment: 5 pages, 4 images, submitted to MEDPRAI2019 conferenc

The modulation of adult neuroplasticity is involved in the mood-improving actions of atypical antipsychotics in an animal model of depression

Depression is a prevalent psychiatric disorder with an increasing impact in global public health. However, a large proportion of patients treated with currently available antidepressant drugs fail to achieve remission. Recently, antipsychotic drugs have received approval for the treatment of antidepressant-resistant forms of major depression. The modulation of adult neuroplasticity, namely hippocampal neurogenesis and neuronal remodeling, has been considered to have a key role in the therapeutic effects of antidepressants. However, the impact of antipsychotic drugs on these neuroplastic mechanisms remains largely unexplored. In this study, an unpredictable chronic mild stress protocol was used to induce a depressive-like phenotype in rats. In the last 3 weeks of stress exposure, animals were treated with two different antipsychotics: haloperidol (a classical antipsychotic) and clozapine (an atypical antipsychotic). We demonstrated that clozapine improved both measures of depressive-like behavior (behavior despair and anhedonia), whereas haloperidol aggravated learned helplessness in the forced-swimming test and behavior flexibility in a cognitive task. Importantly, an upregulation of adult neurogenesis and neuronal survival was observed in animals treated with clozapine, whereas haloperidol promoted a downregulation of these processes. Furthermore, clozapine was able to re-establish the stress-induced impairments in neuronal structure and gene expression in the hippocampus and prefrontal cortex. These results demonstrate the modulation of adult neuroplasticity by antipsychotics in an animal model of depression, revealing that the atypical antipsychotic drug clozapine reverts the behavioral effects of chronic stress by improving adult neurogenesis, cell survival and neuronal reorganization.This work was co-funded by the Life and Health Sciences Research Institute (ICVS), and Northern Portugal Regional Operational Programme (NORTE 2020), under the Portugal 2020 Partnership Agreement, through the European Regional Development Fund (FEDER) (Projects NORTE-01-0145-FEDER-000013 and NORTE-01-0145-FEDER-000023). This work has been also funded by FEDER funds, through the Competitiveness Factors Operational Programme (COMPETE) and by National funds, through the FCT, under the scope of the project POCI-01-0145-FEDER-007038. We thank Luís Martins and Ana Lima for the technical assistanceinfo:eu-repo/semantics/publishedVersio

The Factors Influencing Depression Endpoints Research (FINDER) study: final results of Italian patients with depression

<p>Abstract</p> <p>Background</p> <p>Factors Influencing Depression Endpoints Research (FINDER) is a 6-month, prospective, observational study carried out in 12 European countries aimed at investigating health-related quality of life (HRQoL) in outpatients receiving treatment for a first or new depressive episode. The Italian HRQoL data at 6 months is described in this report, and the factors associated with HRQoL changes were determined.</p> <p>Methods</p> <p>Data were collected at baseline, 3 and 6 months of treatment. HRQoL was measured using components of the 36-item Short Form Health Survey (SF-36; mental component summary (MCS), physical component summary (PCS)) and the European Quality of Life-5 Dimensions (EQ-5D; visual analogue scale (VAS) and health status index (HSI)). The Hospital Anxiety and Depression Scale (HADS) was adopted to evaluate depressive symptoms, while somatic and painful physical symptoms were assessed by using the 28-item Somatic Symptom Inventory (SSI-28) and a VAS.</p> <p>Results</p> <p>Of the initial 513 patients, 472 completed the 3-month observation and 466 the 6-month observation. The SF-36 and EQ-5D mean (± SD) scores showed HRQoL improvements at 3 months and a further smaller improvement at 6 months, with the most positive effects for SF-36 MCS (baseline 22.0 ± 9.2, 3 months 34.6 ± 10.0; 6 months 39.3 ± 9.5) and EQ-5D HSI (baseline 0.4 ± 0.3; 3 months 0.7 ± 0.3; 6 months 0.7 ± 0.2). Depression and anxiety symptoms (HADS-D mean at baseline 13.3 ± 4.2; HADS-A mean at baseline 12.2 ± 3.9) consistently decreased during the first 3 months (8.7 ± 4.3; 7.5 ± 3.6) and showed a further positive change at 6 months (6.9 ± 4.3; 5.8 ± 3.4). Somatic and painful symptoms (SSI and VAS) significantly decreased, with the most positive changes in the SSI-28 somatic item (mean at baseline 2.4 ± 0.7; mean change at 3 months: -0.5; 95% CI -0.6 to -0.5; mean change at 6 months: -0.7; 95% CI -0.8 to -0.7); in 'interference of overall pain with daily activities' (mean at baseline 45.2 ± 30.7; mean change at 3 months -17.4; 95% CI -20.0 to -14.8; mean change at 6 months -24.4; 95% CI -27.3 to -21.6) and in 'having pain while awake' (mean at baseline 41.1 ± 29.0; mean change at 3 months -13.7; 95% CI -15.9 to -11.5; mean change at 6 months -20.2; 95% CI -22.8 to -17.5) domains. The results from linear regression analyses showed that the antidepressant switch within classes was consistently associated with a worsening in SF-36 MCS, EQ-5D VAS and HSI compared to non-switching treatment. Furthermore, between-group antidepressants (AD) switch was associated with a worse SF-36 MCS and EQ-5D HSI. MCS (<it>P </it>= 0.028), PCS (<it>P </it>= 0.036) and HSI (<it>P </it>= 0.002) were inversely related to the number of each previous additional depressive episode. PCS (<it>P </it>= 0.009) and HSI (<it>P </it>= 0.005) were also less improved in patients suffering from a chronic medical condition. Moreover, PCS (<it>P </it>= 0.044) and EQ-5D VAS (<it>P </it>< 0.0001) worsening was consistently associated with the presence of a psychiatric illness in the 24 months before baseline. For every additional point on the SSI-somatic score and on the overall pain VAS score at baseline, HSI score were on average 0.062 (<it>P </it>< 0.001) and 0.001 (<it>P </it>= 0.005) smaller, respectively.</p> <p>Conclusions</p> <p>After starting AD treatment, HRQoL improvements at 3 and 6 months were observed. However, several factors can negatively influence HRQoL, such as the presence of somatic and painful symptoms, the presence of any chronic medical condition or previous psychiatric illness.</p

Intelligent decision support for maintenance: an overview and future trends

The changing nature of manufacturing, in recent years, is evident in industry’s willingness to adopt network-connected intelligent machines in their factory development plans. A number of joint corporate/government initiatives also describe and encourage the adoption of Artificial Intelligence (AI) in the operation and management of production lines. Machine learning will have a significant role to play in the delivery of automated and intelligently supported maintenance decision-making systems. While e-maintenance practice provides aframework for internet-connected operation of maintenance practice the advent of IoT has changed the scale of internetworking and new architectures and tools are needed. While advances in sensors and sensor fusion techniques have been significant in recent years, the possibilities brought by IoT create new challenges in the scale of data and its analysis. The development of audit trail style practice for the collection of data and the provision of acomprehensive framework for its processing, analysis and use should be avaluable contribution in addressing the new data analytics challenges for maintenance created by internet connected devices. This paper proposes that further research should be conducted into audit trail collection of maintenance data, allowing future systems to enable ‘Human in the loop’ interactions

Reduced frontal-subcortical white matter connectivity in association with suicidal ideation in major depressive disorder

Major depressive disorder (MDD) and suicidal behavior have been associated with structural and functional changes in the brain. However, little is known regarding alterations of brain networks in MDD patients with suicidal ideation. We investigated whether or not MDD patients with suicidal ideation have different topological organizations of white matter networks compared with MDD patients without suicidal ideation. Participants consisted of 24 patients with MDD and suicidal ideation, 25 age- and gender-matched MDD patients without suicidal ideation and 31 healthy subjects. A network-based statistics (NBS) and a graph theoretical analysis were performed to assess differences in the inter-regional connectivity. Diffusion tensor imaging (DTI) was performed to assess topological changes according to suicidal ideation in MDD patients. The Scale for Suicide Ideation (SSI) and the Korean version of the Barrett Impulsiveness Scale (BIS) were used to assess the severity of suicidal ideation and impulsivity, respectively. Reduced structural connectivity in a characterized subnetwork was found in patients with MDD and suicidal ideation by utilizing NBS analysis. The subnetwork included the regions of the frontosubcortical circuits and the regions involved in executive function in the left hemisphere (rostral middle frontal, pallidum, superior parietal, frontal pole, caudate, putamen and thalamus). The graph theoretical analysis demonstrated that network measures of the left rostral middle frontal had a significant positive correlation with severity of SSI (r=0.59, P=0.02) and BIS (r=0.59, P=0.01). The total edge strength that was significantly associated with suicidal ideation did not differ between MDD patients without suicidal ideation and healthy subjects. Our findings suggest that the reduced frontosubcortical circuit of structural connectivity, which includes regions associated with executive function and impulsivity, appears to have a role in the emergence of suicidal ideation in MDD patients

Analogue of the quantum hanle effect and polarization conversion in non-hermitian plasmonic metamaterials

This document is the Accepted Manuscript version of a Published Work that appeared in final form in Nano Letters, copyright © American Chemical Society after peer review and technical editing by the publisher. To access the final edited and published work see http://pubs.acs.org/page/policy/articlesonrequest/index.htmlThe Hanle effect, one of the first manifestations of quantum theory introducing the concept of coherent superposition between pure states, plays a key role in numerous aspects of science varying from applicative spectroscopy to fundamental astrophysical investigations. Optical analogues of quantum effects help to achieve deeper understanding of quantum phenomena and, in turn, to develop cross-disciplinary approaches to realizations of new applications in photonics. Here we show that metallic nanostructures can be designed to exhibit a plasmonic analogue of the quantum Hanle effect and the associated polarization rotation. In the original Hanle effect, time-reversal symmetry is broken by a static magnetic field. We achieve this by introducing dissipative level crossing of localized surface plasmons due to nonuniform losses, designed using a non-Hermitian formulation of quantum mechanics. Such artificial plasmonic "atoms" have been shown to exhibit strong circular birefringence and circular dichroism which depends on the value of loss or gain in the metal-dielectric nanostructure. © 2012 American Chemical Society.This work has been supported in part by EPSRC (UK). P.G. acknowledges Royal Society for a Newton International Fellowship. F.J.R.-F. acknowledges support from grant FPI of GV and the Spanish MICINN under contracts CONSOLIDER EMET CSD2008-00066 and TEC2011-28664-C02-02.Ginzburg, P.; Rodríguez Fortuño, FJ.; Martínez Abietar, AJ.; Zayats, AV. (2012). Analogue of the quantum hanle effect and polarization conversion in non-hermitian plasmonic metamaterials. Nano Letters. 12(12):6309-6314. https://doi.org/10.1021/nl3034174S63096314121