Assessing the ability of Transformer-based Neural Models to represent structurally unbounded dependencies

Filler-gap dependencies are among the most challenging syntactic constructions for com- putational models at large. Recently, Wilcox et al. (2018) and Wilcox et al. (2019b) provide some evidence suggesting that large-scale general-purpose LSTM RNNs have learned such long-distance filler-gap dependencies. In the present work we provide evidence that such models learn filler-gap dependencies only very imperfectly, despite being trained on massive amounts of data. Finally, we compare the LSTM RNN models with more modern state-of-the-art Transformer models, and find that these have poor-to-mixed degrees of success, despite their sheer size and low perplexity

31st Annual Meeting and Associated Programs of the Society for Immunotherapy of Cancer (SITC 2016) : part two

Background The immunological escape of tumors represents one of the main ob- stacles to the treatment of malignancies. The blockade of PD-1 or CTLA-4 receptors represented a milestone in the history of immunotherapy. However, immune checkpoint inhibitors seem to be effective in specific cohorts of patients. It has been proposed that their efficacy relies on the presence of an immunological response. Thus, we hypothesized that disruption of the PD-L1/PD-1 axis would synergize with our oncolytic vaccine platform PeptiCRAd. Methods We used murine B16OVA in vivo tumor models and flow cytometry analysis to investigate the immunological background. Results First, we found that high-burden B16OVA tumors were refractory to combination immunotherapy. However, with a more aggressive schedule, tumors with a lower burden were more susceptible to the combination of PeptiCRAd and PD-L1 blockade. The therapy signifi- cantly increased the median survival of mice (Fig. 7). Interestingly, the reduced growth of contralaterally injected B16F10 cells sug- gested the presence of a long lasting immunological memory also against non-targeted antigens. Concerning the functional state of tumor infiltrating lymphocytes (TILs), we found that all the immune therapies would enhance the percentage of activated (PD-1pos TIM- 3neg) T lymphocytes and reduce the amount of exhausted (PD-1pos TIM-3pos) cells compared to placebo. As expected, we found that PeptiCRAd monotherapy could increase the number of antigen spe- cific CD8+ T cells compared to other treatments. However, only the combination with PD-L1 blockade could significantly increase the ra- tio between activated and exhausted pentamer positive cells (p= 0.0058), suggesting that by disrupting the PD-1/PD-L1 axis we could decrease the amount of dysfunctional antigen specific T cells. We ob- served that the anatomical location deeply influenced the state of CD4+ and CD8+ T lymphocytes. In fact, TIM-3 expression was in- creased by 2 fold on TILs compared to splenic and lymphoid T cells. In the CD8+ compartment, the expression of PD-1 on the surface seemed to be restricted to the tumor micro-environment, while CD4 + T cells had a high expression of PD-1 also in lymphoid organs. Interestingly, we found that the levels of PD-1 were significantly higher on CD8+ T cells than on CD4+ T cells into the tumor micro- environment (p < 0.0001). Conclusions In conclusion, we demonstrated that the efficacy of immune check- point inhibitors might be strongly enhanced by their combination with cancer vaccines. PeptiCRAd was able to increase the number of antigen-specific T cells and PD-L1 blockade prevented their exhaus- tion, resulting in long-lasting immunological memory and increased median survival

International eDelphi Study to Reach Consensus on the Methotrexate Dosing Regimen in Patients With Psoriasis

Abstract Importance: A clear dosing regimen for methotrexate in psoriasis is lacking, and this might lead to a suboptimal treatment. Because methotrexate is affordable and globally available, a uniform dosing regimen could potentially optimize the treatment of patients with psoriasis worldwide. Objective: To reach international consensus among psoriasis experts on a uniform dosing regimen for treatment with methotrexate in adult and pediatric patients with psoriasis and identify potential future research topics. Design, setting, and participants: Between September 2020 and March 2021, a survey study with a modified eDelphi procedure that was developed and distributed by the Amsterdam University Medical Center and completed by 180 participants worldwide (55 [30.6%] resided in non-Western countries) was conducted in 3 rounds. The proposals on which no consensus was reached were discussed in a conference meeting (June 2021). Participants voted on 21 proposals with a 9-point scale (1-3 disagree, 4-6 neither agree nor disagree, 7-9 agree) and were recruited through the Skin Inflammation and Psoriasis International Network and European Academy of Dermatology and Venereology in June 2020. Apart from being a dermatologist/dermatology resident, there were no specific criteria for participation in the survey. The participants worked mainly at a university hospital (97 [53.9%]) and were experienced in treating patients with psoriasis with methotrexate (163 [91.6%] had more than 10 years of experience). Main outcomes and measures: In a survey with eDelphi procedure, we tried to reach consensus on 21 proposals. Consensus was defined as less than 15% voting disagree (1-3). For the consensus meeting, consensus was defined as less than 30% voting disagree. Results: Of 251 participants, 180 (71.7%) completed all 3 survey rounds, and 58 participants (23.1%) joined the conference meeting. Consensus was achieved on 11 proposals in round 1, 3 proposals in round 2, and 2 proposals in round 3. In the consensus meeting, consensus was achieved on 4 proposals. More research is needed, especially for the proposals on folic acid and the dosing of methotrexate for treating subpopulations such as children and vulnerable patients. Conclusions and relevance: In this eDelphi consensus study, consensus was reached on 20 of 21 proposals involving methotrexate dosing in patients with psoriasis. This consensus may potentially be used to harmonize the treatment with methotrexate in patients with psoriasis

International eDelphi Study to Reach Consensus on the Methotrexate Dosing Regimen in Patients with Psoriasis

Importance: A clear dosing regimen for methotrexate in psoriasis is lacking, and this might lead to a suboptimal treatment. Because methotrexate is affordable and globally available, a uniform dosing regimen could potentially optimize the treatment of patients with psoriasis worldwide. Objective: To reach international consensus among psoriasis experts on a uniform dosing regimen for treatment with methotrexate in adult and pediatric patients with psoriasis and identify potential future research topics. Design, Setting, and Participants: Between September 2020 and March 2021, a survey study with a modified eDelphi procedure that was developed and distributed by the Amsterdam University Medical Center and completed by 180 participants worldwide (55 [30.6%] resided in non-Western countries) was conducted in 3 rounds. The proposals on which no consensus was reached were discussed in a conference meeting (June 2021). Participants voted on 21 proposals with a 9-point scale (1-3 disagree, 4-6 neither agree nor disagree, 7-9 agree) and were recruited through the Skin Inflammation and Psoriasis International Network and European Academy of Dermatology and Venereology in June 2020. Apart from being a dermatologist/dermatology resident, there were no specific criteria for participation in the survey. The participants worked mainly at a university hospital (97 [53.9%]) and were experienced in treating patients with psoriasis with methotrexate (163 [91.6%] had more than 10 years of experience). Main Outcomes and Measures: In a survey with eDelphi procedure, we tried to reach consensus on 21 proposals. Consensus was defined as less than 15% voting disagree (1-3). For the consensus meeting, consensus was defined as less than 30% voting disagree. Results: Of 251 participants, 180 (71.7%) completed all 3 survey rounds, and 58 participants (23.1%) joined the conference meeting. Consensus was achieved on 11 proposals in round 1, 3 proposals in round 2, and 2 proposals in round 3. In the consensus meeting, consensus was achieved on 4 proposals. More research is needed, especially for the proposals on folic acid and the dosing of methotrexate for treating subpopulations such as children and vulnerable patients. Conclusions and Relevance: In this eDelphi consensus study, consensus was reached on 20 of 21 proposals involving methotrexate dosing in patients with psoriasis. This consensus may potentially be used to harmonize the treatment with methotrexate in patients with psoriasis.

Reduced Cancer Incidence in Huntington's Disease: Analysis in the Registry Study

Background: People with Huntington's disease (HD) have been observed to have lower rates of cancers. Objective: To investigate the relationship between age of onset of HD, CAG repeat length, and cancer diagnosis. Methods: Data were obtained from the European Huntington's disease network REGISTRY study for 6540 subjects. Population cancer incidence was ascertained from the GLOBOCAN database to obtain standardised incidence ratios of cancers in the REGISTRY subjects. Results: 173/6528 HD REGISTRY subjects had had a cancer diagnosis. The age-standardised incidence rate of all cancers in the REGISTRY HD population was 0.26 (CI 0.22-0.30). Individual cancers showed a lower age-standardised incidence rate compared with the control population with prostate and colorectal cancers showing the lowest rates. There was no effect of CAG length on the likelihood of cancer, but a cancer diagnosis within the last year was associated with a greatly increased rate of HD onset (Hazard Ratio 18.94, p < 0.001). Conclusions: Cancer is less common than expected in the HD population, confirming previous reports. However, this does not appear to be related to CAG length in HTT. A recent diagnosis of cancer increases the risk of HD onset at any age, likely due to increased investigation following a cancer diagnosis

Clinical and genetic characteristics of late-onset Huntington's disease

Background: The frequency of late-onset Huntington's disease (&gt;59 years) is assumed to be low and the clinical course milder. However, previous literature on late-onset disease is scarce and inconclusive. Objective: Our aim is to study clinical characteristics of late-onset compared to common-onset HD patients in a large cohort of HD patients from the Registry database. Methods: Participants with late- and common-onset (30–50 years)were compared for first clinical symptoms, disease progression, CAG repeat size and family history. Participants with a missing CAG repeat size, a repeat size of ≤35 or a UHDRS motor score of ≤5 were excluded. Results: Of 6007 eligible participants, 687 had late-onset (11.4%) and 3216 (53.5%) common-onset HD. Late-onset (n = 577) had significantly more gait and balance problems as first symptom compared to common-onset (n = 2408) (P &lt;.001). Overall motor and cognitive performance (P &lt;.001) were worse, however only disease motor progression was slower (coefficient, −0.58; SE 0.16; P &lt;.001) compared to the common-onset group. Repeat size was significantly lower in the late-onset (n = 40.8; SD 1.6) compared to common-onset (n = 44.4; SD 2.8) (P &lt;.001). Fewer late-onset patients (n = 451) had a positive family history compared to common-onset (n = 2940) (P &lt;.001). Conclusions: Late-onset patients present more frequently with gait and balance problems as first symptom, and disease progression is not milder compared to common-onset HD patients apart from motor progression. The family history is likely to be negative, which might make diagnosing HD more difficult in this population. However, the balance and gait problems might be helpful in diagnosing HD in elderly patients