Towards Better Generalization with Flexible Representation of Multi-Module Graph Neural Networks

Graph neural networks (GNNs) have become compelling models designed to perform learning and inference on graph-structured data. However, little work has been done to understand the fundamental limitations of GNNs for scaling to larger graphs and generalizing to out-of-distribution (OOD) inputs. In this paper, we use a random graph generator to systematically investigate how the graph size and structural properties affect the predictive performance of GNNs. We present specific evidence that the average node degree is a key feature in determining whether GNNs can generalize to unseen graphs, and that the use of multiple node update functions can improve the generalization performance of GNNs when dealing with graphs of multimodal degree distributions. Accordingly, we propose a multi-module GNN framework that allows the network to adapt flexibly to new graphs by generalizing a single canonical nonlinear transformation over aggregated inputs. Our results show that the multi-module GNNs improve the OOD generalization on a variety of inference tasks in the direction of diverse structural features

How Much and When Do We Need Higher-order Information in Hypergraphs? A Case Study on Hyperedge Prediction

Hypergraphs provide a natural way of representing group relations, whose complexity motivates an extensive array of prior work to adopt some form of abstraction and simplification of higher-order interactions. However, the following question has yet to be addressed: How much abstraction of group interactions is sufficient in solving a hypergraph task, and how different such results become across datasets? This question, if properly answered, provides a useful engineering guideline on how to trade off between complexity and accuracy of solving a downstream task. To this end, we propose a method of incrementally representing group interactions using a notion of n-projected graph whose accumulation contains information on up to n-way interactions, and quantify the accuracy of solving a task as n grows for various datasets. As a downstream task, we consider hyperedge prediction, an extension of link prediction, which is a canonical task for evaluating graph models. Through experiments on 15 real-world datasets, we draw the following messages: (a) Diminishing returns: small n is enough to achieve accuracy comparable with near-perfect approximations, (b) Troubleshooter: as the task becomes more challenging, larger n brings more benefit, and (c) Irreducibility: datasets whose pairwise interactions do not tell much about higher-order interactions lose much accuracy when reduced to pairwise abstractions

Hypergraph Motifs and Their Extensions Beyond Binary

Hypergraphs naturally represent group interactions, which are omnipresent in many domains: collaborations of researchers, co-purchases of items, and joint interactions of proteins, to name a few. In this work, we propose tools for answering the following questions: (Q1) what are the structural design principles of real-world hypergraphs? (Q2) how can we compare local structures of hypergraphs of different sizes? (Q3) how can we identify domains from which hypergraphs are? We first define hypergraph motifs (h-motifs), which describe the overlapping patterns of three connected hyperedges. Then, we define the significance of each h-motif in a hypergraph as its occurrences relative to those in properly randomized hypergraphs. Lastly, we define the characteristic profile (CP) as the vector of the normalized significance of every h-motif. Regarding Q1, we find that h-motifs' occurrences in 11 real-world hypergraphs from 5 domains are clearly distinguished from those of randomized hypergraphs. Then, we demonstrate that CPs capture local structural patterns unique to each domain, and thus comparing CPs of hypergraphs addresses Q2 and Q3. The concept of CP is extended to represent the connectivity pattern of each node or hyperedge as a vector, which proves useful in node classification and hyperedge prediction. Our algorithmic contribution is to propose MoCHy, a family of parallel algorithms for counting h-motifs' occurrences in a hypergraph. We theoretically analyze their speed and accuracy and show empirically that the advanced approximate version MoCHy-A+ is more accurate and faster than the basic approximate and exact versions, respectively. Furthermore, we explore ternary hypergraph motifs that extends h-motifs by taking into account not only the presence but also the cardinality of intersections among hyperedges. This extension proves beneficial for all previously mentioned applications.Comment: Extended version of VLDB 2020 paper arXiv:2003.0185

MIDAS: Microcluster-Based Detector of Anomalies in Edge Streams

Given a stream of graph edges from a dynamic graph, how can we assign anomaly scores to edges in an online manner, for the purpose of detecting unusual behavior, using constant time and memory? Existing approaches aim to detect individually surprising edges. In this work, we propose MIDAS, which focuses on detecting microcluster anomalies, or suddenly arriving groups of suspiciously similar edges, such as lockstep behavior, including denial of service attacks in network traffic data. MIDAS has the following properties: (a) it detects microcluster anomalies while providing theoretical guarantees about its false positive probability; (b) it is online, thus processing each edge in constant time and constant memory, and also processes the data 162-644 times faster than state-of-the-art approaches; (c) it provides 42%-48% higher accuracy (in terms of AUC) than state-of-the-art approaches.Comment: 8 pages, Accepted at AAAI Conference on Artificial Intelligence (AAAI), 2020 [oral paper]; minor fixes, updated experiment

Osteoporosis Prediction from Hand and Wrist X-rays using Image Segmentation and Self-Supervised Learning

Osteoporosis is a widespread and chronic metabolic bone disease that often remains undiagnosed and untreated due to limited access to bone mineral density (BMD) tests like Dual-energy X-ray absorptiometry (DXA). In response to this challenge, current advancements are pivoting towards detecting osteoporosis by examining alternative indicators from peripheral bone areas, with the goal of increasing screening rates without added expenses or time. In this paper, we present a method to predict osteoporosis using hand and wrist X-ray images, which are both widely accessible and affordable, though their link to DXA-based data is not thoroughly explored. Initially, our method segments the ulnar, radius, and metacarpal bones using a foundational model for image segmentation. Then, we use a self-supervised learning approach to extract meaningful representations without the need for explicit labels, and move on to classify osteoporosis in a supervised manner. Our method is evaluated on a dataset with 192 individuals, cross-referencing their verified osteoporosis conditions against the standard DXA test. With a notable classification score (AUC=0.83), our model represents a pioneering effort in leveraging vision-based techniques for osteoporosis identification from the peripheral skeleton sites.Comment: Extended Abstract presented at Machine Learning for Health (ML4H) symposium 2023, December 10th, 2023, New Orleans, United States, 10 page

Hierarchical Joint Graph Learning and Multivariate Time Series Forecasting

Multivariate time series is prevalent in many scientific and industrial domains. Modeling multivariate signals is challenging due to their long-range temporal dependencies and intricate interactions--both direct and indirect. To confront these complexities, we introduce a method of representing multivariate signals as nodes in a graph with edges indicating interdependency between them. Specifically, we leverage graph neural networks (GNN) and attention mechanisms to efficiently learn the underlying relationships within the time series data. Moreover, we suggest employing hierarchical signal decompositions running over the graphs to capture multiple spatial dependencies. The effectiveness of our proposed model is evaluated across various real-world benchmark datasets designed for long-term forecasting tasks. The results consistently showcase the superiority of our model, achieving an average 23\% reduction in mean squared error (MSE) compared to existing models.Comment: Temporal Graph Learning Workshop @ NeurIPS 2023, New Orleans, United State

Heart Rate Variability and Urinary Catecholamines from Job Stress in Korean Male Manufacturing Workers According to Work Seniority

The aim of this study was to evaluate the relationships between job stress and indicators of autonomic nervous system activity in employees of the manufacturing industry. A total of 140 employees from a company that manufactures consumer goods (i.e., diapers and paper towels) were recruited for participation in this study. Job stress was assessed using Karasek`s Job Content Questionnaire. Heart rate variability (HRV) was measured using a heart rate monitor, and urinary catecholamines were measured by an HPLC-ECD. Information on demographic characteristics, previous job history, smoking status and alcohol consumption was also collected. Job stress did not have a significant effect on HRV or catecholamines. However, low-frequency HRV was significantly higher in the high-strain group of subjects with a short duration of employment. Low- and high-frequency HRV were higher in the high-strain group than in the low-strain group, but these differences were not statistically significant. The results of the present study indicate that low-frequency HRV was significantly higher in the high-strain group of subjects with a short duration of employment. In addition, the results of this study show that HRV can be used as a potential physiologic indicator of job stress in employees with a short duration of employment.Schubert C, 2009, BIOL PSYCHOL, V80, P325, DOI 10.1016/j.biopsycho.2008.11.005Mitoma M, 2008, PROG NEURO-PSYCHOPH, V32, P679, DOI 10.1016/j.pnpbp.2007.11.011Burr RL, 2007, SLEEP, V30, P913Collins SM, 2005, AM J IND MED, V48, P182, DOI 10.1002/ajim.20204Kang MG, 2004, YONSEI MED J, V45, P838Sztajzel J, 2004, SWISS MED WKLY, V134, P514Towa S, 2004, EXP ANIM TOKYO, V53, P137, DOI 10.1538/expanim.53.137Bunker SJ, 2003, MED J AUSTRALIA, V178, P272Peter R, 2002, J EPIDEMIOL COMMUN H, V56, P294VANGELOVA K, 2002, CENT EUR J PUBL HEAL, V10, P149HA M, 2001, STANDARDIZATION DEVvan Amelsvoort LGPM, 2000, INT ARCH OCC ENV HEA, V73, P255Goldstein IB, 1999, PSYCHOSOM MED, V61, P387PIERCECCHIMARTI MD, 1999, MED LAW, V18, P125Sluiter JK, 1998, OCCUP ENVIRON MED, V55, P407KARASEK R, 1998, J OCCUP HEALTH PSYCH, V3, P322ROVERE MTL, 1998, LANCET, V351, P478Sloan RP, 1996, PSYCHOSOM MED, V58, P25VANDERBEEK AJ, 1995, OCCUP ENVIRON MED, V52, P464HUIKURI HV, 1993, CIRCULATION, V87, P1220PAGANI M, 1991, J AUTONOM NERV SYST, V35, P33PAGANI M, 1991, CIRCULATION, V83, P1143PEASTON RT, 1988, J CHROMATOGR-BIOMED, V424, P263