Adversarial attacks on graph classification via Bayesian optimisation

Graph neural networks, a popular class of models effective in a wide range of graph-based learning tasks, have been shown to be vulnerable to adversarial attacks. While the majority of the literature focuses on such vulnerability in node-level classification tasks, little effort has been dedicated to analysing adversarial attacks on graph-level classification, an important problem with numerous real-life applications such as biochemistry and social network analysis. The few existing methods often require unrealistic setups, such as access to internal information of the victim models, or an impractically-large number of queries. We present a novel Bayesian optimisation-based attack method for graph classification models. Our method is black-box, query-efficient and parsimonious with respect to the perturbation applied. We empirically validate the effectiveness and flexibility of the proposed method on a wide range of graph classification tasks involving varying graph properties, constraints and modes of attack. Finally, we analyse common interpretable patterns behind the adversarial samples produced, which may shed further light on the adversarial robustness of graph classification models. An open-source implementation is available at https://github.com/xingchenwan/grabnel

DUET: 2D Structured and Approximately Equivariant Representations

Multiview Self-Supervised Learning (MSSL) is based on learning invariances with respect to a set of input transformations. However, invariance partially or totally removes transformation-related information from the representations, which might harm performance for specific downstream tasks that require such information. We propose 2D strUctured and EquivarianT representations (coined DUET), which are 2d representations organized in a matrix structure, and equivariant with respect to transformations acting on the input data. DUET representations maintain information about an input transformation, while remaining semantically expressive. Compared to SimCLR (Chen et al., 2020) (unstructured and invariant) and ESSL (Dangovski et al., 2022) (unstructured and equivariant), the structured and equivariant nature of DUET representations enables controlled generation with lower reconstruction error, while controllability is not possible with SimCLR or ESSL. DUET also achieves higher accuracy for several discriminative tasks, and improves transfer learning.Comment: Accepted at ICML 202

The Role of Entropy and Reconstruction in Multi-View Self-Supervised Learning

The mechanisms behind the success of multi-view self-supervised learning (MVSSL) are not yet fully understood. Contrastive MVSSL methods have been studied through the lens of InfoNCE, a lower bound of the Mutual Information (MI). However, the relation between other MVSSL methods and MI remains unclear. We consider a different lower bound on the MI consisting of an entropy and a reconstruction term (ER), and analyze the main MVSSL families through its lens. Through this ER bound, we show that clustering-based methods such as DeepCluster and SwAV maximize the MI. We also re-interpret the mechanisms of distillation-based approaches such as BYOL and DINO, showing that they explicitly maximize the reconstruction term and implicitly encourage a stable entropy, and we confirm this empirically. We show that replacing the objectives of common MVSSL methods with this ER bound achieves competitive performance, while making them stable when training with smaller batch sizes or smaller exponential moving average (EMA) coefficients. Github repo: https://github.com/apple/ml-entropy-reconstruction.Comment: 18 pages: 9 of main text, 2 of references, and 7 of supplementary material. Appears in the proceedings of ICML 202

Cardiopulmonary exercise pattern in patients with persistent dyspnoea after recovery from COVID-19

Cause and mechanisms of persistent dyspnoea after recovery from COVID-19 are not well described. The objective is to describe causal factors for persistent dyspnoea in patients after COVID-19. We examined patients reporting dyspnoea after recovery from COVID-19 by cardiopulmonary exercise testing. After exclusion of patients with pre-existing lung diseases, ten patients (mean age 50±13.1 years) were retrospectively analysed between May 14th and September 15th, 2020. On chest computed tomography, five patients showed residual ground glass opacities, and one patient showed streaky residua. A slight reduction of the mean diffusion capacity of the lung for carbon monoxide was noted in the cohort. Mean peak oxygen uptake was reduced with 1512±232 ml/min (72.7% predicted), while mean peakwork rate was preserved with 131±29 W (92.4% predicted). Mean alveolar-arterial oxygen gradient (AaDO2) at peak exercise was 25.6±11.8 mmHg. Mean value of lactate post exercise was 5.6±1.8 mmol/l. A gap between peak work rate in (92.4% predicted) to peak oxygen uptake (72.3% pred.) was detected in our study cohort. Mean value of lactate post exercise was high in our study population and even higher (n.s.) compared to the subgroup of patients with reduced peak oxygen uptake and other obvious reason for limitation. Both observations support the hypothesis of anaerobic metabolism. The main reason for dyspnoea may therefore be muscular