Transformers over Directed Acyclic Graphs

Transformer models have recently gained popularity in graph representation learning as they have the potential to learn complex relationships beyond the ones captured by regular graph neural networks. The main research question is how to inject the structural bias of graphs into the transformer architecture, and several proposals have been made for undirected molecular graphs and, recently, also for larger network graphs. In this paper, we study transformers over directed acyclic graphs (DAGs) and propose architecture adaptations tailored to DAGs: (1) An attention mechanism that is considerably more efficient than the regular quadratic complexity of transformers and at the same time faithfully captures the DAG structure, and (2) a positional encoding of the DAG's partial order, complementing the former. We rigorously evaluate our approach over various types of tasks, ranging from classifying source code graphs to nodes in citation networks, and show that it is effective in two important aspects: in making graph transformers generally outperform graph neural networks tailored to DAGs and in improving SOTA graph transformer performance in terms of both quality and efficiency

Improving Self-supervised Molecular Representation Learning using Persistent Homology

Self-supervised learning (SSL) has great potential for molecular representation learning given the complexity of molecular graphs, the large amounts of unlabelled data available, the considerable cost of obtaining labels experimentally, and the hence often only small training datasets. The importance of the topic is reflected in the variety of paradigms and architectures that have been investigated recently. Yet the differences in performance seem often minor and are barely understood to date. In this paper, we study SSL based on persistent homology (PH), a mathematical tool for modeling topological features of data that persist across multiple scales. It has several unique features which particularly suit SSL, naturally offering: different views of the data, stability in terms of distance preservation, and the opportunity to flexibly incorporate domain knowledge. We (1) investigate an autoencoder, which shows the general representational power of PH, and (2) propose a contrastive loss that complements existing approaches. We rigorously evaluate our approach for molecular property prediction and demonstrate its particular features in improving the embedding space: after SSL, the representations are better and offer considerably more predictive power than the baselines over different probing tasks; our loss increases baseline performance, sometimes largely; and we often obtain substantial improvements over very small datasets, a common scenario in practice.Comment: NeurIPS 202

Impact-Oriented Contextual Scholar Profiling using Self-Citation Graphs

Quantitatively profiling a scholar's scientific impact is important to modern research society. Current practices with bibliometric indicators (e.g., h-index), lists, and networks perform well at scholar ranking, but do not provide structured context for scholar-centric, analytical tasks such as profile reasoning and understanding. This work presents GeneticFlow (GF), a suite of novel graph-based scholar profiles that fulfill three essential requirements: structured-context, scholar-centric, and evolution-rich. We propose a framework to compute GF over large-scale academic data sources with millions of scholars. The framework encompasses a new unsupervised advisor-advisee detection algorithm, a well-engineered citation type classifier using interpretable features, and a fine-tuned graph neural network (GNN) model. Evaluations are conducted on the real-world task of scientific award inference. Experiment outcomes show that the F1 score of best GF profile significantly outperforms alternative methods of impact indicators and bibliometric networks in all the 6 computer science fields considered. Moreover, the core GF profiles, with 63.6%-66.5% nodes and 12.5%-29.9% edges of the full profile, still significantly outrun existing methods in 5 out of 6 fields studied. Visualization of GF profiling result also reveals human explainable patterns for high-impact scholars

AliCHI: A Large-scale Multi-modal Dataset and Automated Evaluation Tool for Human-like Dialogue Systems

A well-designed interactive human-like dialogue system is expected to take actions (e.g. smiling) and respond in a pattern similar to humans. However, due to the limitation of single-modality (only speech) or small volume of currently public datasets, most dialogue systems can only respond in speech and cannot take human-like actions. In this work, we build a large-scale multi-modal dataset of human-to-human conversation in a face-to-face fashion, with fine-grained annotations. The raw data in video format contains 635 dialogue sessions, being collected from 200 participants on designed topics and lasting 52 hours in total. Moreover, we manually annotated the verbal and non-verbal behaviors in each dialogue session on their start/end timestamp. Furthermore, we developed a corresponding evaluation tool for human-like dialogue systems to automatically evaluates the accuracy of two basic tasks, turn-taking prediction, and backchannel prediction, on both time and content. We have opened the data, the tools will be released at the conference

Platinum-based chemotherapy plus cetuximab first-line for Asian patients with recurrent and/or metastatic squamous cell carcinoma of the head and neck: Results of an open-label, single-arm, multicenter trial

Background The purpose of this study was to assess the efficacy, safety, and pharmacokinetics of cisplatin-based chemotherapy plus cetuximab as first-line treatment in Chinese and Korean patients with recurrent and/or metastatic squamous cell carcinoma of the head and neck (SCCHN). Methods Patients (n = 68) received cetuximab weekly plus 3-week cycles of cisplatin/5-fluorouracil (5-FU) chemotherapy for up to 6 cycles. The primary endpoint was overall response rate. Results The overall response rate was 55.9%, including 2 complete responses (CRs). Median overall survival (OS) was 12.6 months and median progression-free survival (PFS) was 6.6 months. Grade 3/4 adverse events (AEs) were reported in 41 (60.3%) patients. The safety profile was in line with previous clinical experience. The pharmacokinetic profile was in line with that observed with cetuximab in white and Japanese patients. Conclusion The efficacy, safety, and pharmacokinetic findings from this study support the use of first-line platinum-based chemotherapy plus cetuximab in Chinese and Korean patients with recurrent and/or metastatic SCCHN (ClinicalTrials.gov NCT01177956). © 2014 The Authors Head & Neck Published by Wiley Periodicals, Inc. Head Neck 37: 1081–1087, 201

Efficacy and safety of immunotherapy combined with single-agent chemotherapy as second- or later-line therapy for metastatic non-small cell lung cancer

ObjectiveThis study sought to assess the efficacy and safety of immunotherapy combined with single-agent chemotherapy as a second- or later-line setting for metastatic non-small cell lung cancer (NSCLC) and to provide clinical evidence for this treatment regimen. The predictive value of extracellular vesicle (EV) membrane proteins was explored in patients who underwent this treatment.MethodsClinical data from patients diagnosed with metastatic NSCLC who received immunotherapy plus single-agent chemotherapy as a second- or later-line setting were retrospectively collected between March 2019 and January 2022. A total of 30 patients met the inclusion criteria, and all were pathologically confirmed to have NSCLC. Short-term efficacy, progression-free survival (PFS), EV markers for response prediction, and adverse events were assessed.ResultsEfficacy data were available for all 30 patients and included a partial response in 5 patients, stable disease in 18 patients, and disease progression in 7 patients. The objective response rate was 16.7%, the disease control rate was 76.7%, and the median PFS was 3.2 months. Univariate analysis showed that PFS was not associated with sex, age, smoking status, treatment lines, prior use of immunotherapy, or prior use of antiangiogenic drugs. The EV membrane proteins MET proto-oncogene, receptor tyrosine kinase (c-MET), epidermal growth factor receptor (EGFR), and vascular endothelial growth factor receptor 2 (VEGFR2) at baseline were associated with poor prognosis and correlated with the efficacy of immunotherapy plus chemotherapy. According to the receiver operating characteristics and Kaplan–Meier curve analyses, patients with high c-MET, EGFR, and VEGFR2 expression at baseline had significantly shorter PFS than those with low expression. In addition, VEGFR2 expression was increased after combined immunotherapy in responders, which was decreased in non-responders. The most common grade 2 or higher adverse events were neutropenia, gastrointestinal reactions, and thyroid dysfunction, all of which were tolerated.ConclusionsImmunotherapy plus single-agent chemotherapy as a second- or later-line treatment is safe, effective, and tolerable for metastatic NSCLC. EV markers can be used as predictive markers of efficacy in patients with metastatic NSCLC treated with immunotherapy plus chemotherapy to help monitor treatment efficacy and guide treatment decisions

Total metabolic tumor volume as a survival predictor for patients with diffuse large B-cell lymphoma in the GOYA study

This retrospective analysis of the phase III GOYA study investigated the prognostic value of baseline metabolic tumor volume parameters and maximum standardized uptake values for overall and progression-free survival (PFS) in treatment-naïve diffuse large B-cell lymphoma. Baseline total metabolic tumor volume (determined for tumors >1 mL using a threshold of 1.5 times the mean liver standardized uptake value +2 standard deviations), total lesion glycolysis, and maximum standardized uptake value positron emission tomography data were dichotomized based on receiver operating characteristic analysis and divided into quartiles by baseline population distribution. Of 1,418 enrolled patients, 1,305 had a baseline positron emission tomography scan with detectable lesions. Optimal cut-offs were 366 cm3 for total metabolic tumor volume and 3,004 g for total lesion glycolysis. High total metabolic tumor volume and total lesion glycolysis predicted poorer PFS, with associations retained after adjustment for baseline and disease characteristics (high total metabolic tumor volume hazard ratio: 1.71, 95% confidence interval [CI]: 1.352.18; total lesion glycolysis hazard ratio: 1.46; 95% CI: 1.15-1.86). Total metabolic tumor volume was prognostic for PFS in subgroups with International Prognostic Index scores 0-2 and 3-5, and those with different cell-of-origin subtypes. Maximum standardized uptake value had no prognostic value in this setting. High total metabolic tumor volume associated with high International Prognostic Index or non-germinal center B-cell classification identified the highest-risk cohort for unfavorable prognosis. In conclusion, baseline total metabolic tumor volume and total lesion glycolysis are independent predictors of PFS in patients with diffuse large B-cell lymphoma after first-line immunochemotherapy