Improved Algorithms for Online Rent Minimization Problem Under Unit-Size Jobs

We consider the Online Rent Minimization problem, where online jobs with release times, deadlines, and processing times must be scheduled on machines that can be rented for a fixed length period of T. The objective is to minimize the number of machine rents. This problem generalizes the Online Machine Minimization problem where machines can be rented for an infinite period, and both problems have an asymptotically optimal competitive ratio of O(log(p_max/p_min)) for general processing times, where p_max and p_min are the maximum and minimum processing times respectively. However, for small values of p_max/p_min, a better competitive ratio can be achieved by assuming unit-size jobs. Under this assumption, Devanur et al. (2014) gave an optimal e-competitive algorithm for Online Machine Minimization, and Chen and Zhang (2022) gave a (3e+7) ? 15.16-competitive algorithm for Online Rent Minimization. In this paper, we significantly improve the competitive ratio of the Online Rent Minimization problem under unit size to 6, by using a clean oracle-based online algorithm framework

A Closer Look at the Adversarial Robustness of Deep Equilibrium Models

Deep equilibrium models (DEQs) refrain from the traditional layer-stacking paradigm and turn to find the fixed point of a single layer. DEQs have achieved promising performance on different applications with featured memory efficiency. At the same time, the adversarial vulnerability of DEQs raises concerns. Several works propose to certify robustness for monotone DEQs. However, limited efforts are devoted to studying empirical robustness for general DEQs. To this end, we observe that an adversarially trained DEQ requires more forward steps to arrive at the equilibrium state, or even violates its fixed-point structure. Besides, the forward and backward tracks of DEQs are misaligned due to the black-box solvers. These facts cause gradient obfuscation when applying the ready-made attacks to evaluate or adversarially train DEQs. Given this, we develop approaches to estimate the intermediate gradients of DEQs and integrate them into the attacking pipelines. Our approaches facilitate fully white-box evaluations and lead to effective adversarial defense for DEQs. Extensive experiments on CIFAR-10 validate the adversarial robustness of DEQs competitive with deep networks of similar sizes.Comment: Accepted at NeurIPS 2022. Our code is available at https://github.com/minicheshire/DEQ-White-Box-Robustnes

Spatial-Temporal Data Modeling with Graph Neural Networks

University of Technology Sydney. Faculty of Engineering and Information Technology.Spatial-temporal graph modeling is an important task to analyze the spatial relations and temporal trends of components in a system. It aims to model the dynamic node-level inputs by assuming inter-dependency between connected nodes. A basic assumption behind spatial-temporal graph modeling is that a node's future information is conditioned on its historical information as well as its neighbors' historical information. Therefore how to capture spatial and temporal dependencies simultaneously becomes a primary challenge. Current studies on spatial-temporal graph modeling face four major shortcomings: 1) Most graph neural networks only focus on the low frequency band of graph signals; 2) Current studies assume the graph structure of data reflects the genuine dependency relationships among nodes; 3) Existing studies on spatial-temporal graph neural networks are not applicable to pure multivariate time series data due to the absence of a predefined graph and lack of a general framework; 4) Existing approaches either model spatial-temporal dependencies locally or model spatial correlations and temporal correlations separately. The aim of this thesis is to study spatial-temporal data from the perspective of deep learning on graphs. I have studied the research objective in deep depth with four research questions: (1) How to coordinate the low, middle, and high frequency band of graph signals in graph convolution networks. (2) How to model spatial-temporal graph data effectively and efficiently; (3) How to handle spatial dependencies when a graph is totally missing, incomplete or inaccurate in spatial-temporal graph modeling; (4) In contrast to traditional spatial-temporal graph neural networks that handle spatial dependencies and temporal dependencies in separate, how to unify space and time as a whole in message passing. To address the aforementioned four research problems, I proposed four algorithms or models that can achieve satisfactory results. Specifically, I proposed an Automatic Graph Convolutional Network to learn graph frequency bands for graph convolution filters automatically; I introduced an efficient and effective framework that integrates diffusion graph convolution and dilated temporal convolution to capture spatial-temporal dependencies simultaneously. I developed a novel joint-learning algorithm that can capture spatial-temporal dependencies and learn latent graph structures at the same time; I designed a unified graph neural network that captures the inner spatial-temporal dependencies without compromising space-time integrity. To validate the proposed methods, I have conducted experiments on real-world datasets with a range of tasks including node classification, graph classification, and spatial-temporal graph forecasting. Experimental results demonstrate the effectiveness of the proposed methods

XFlow: Benchmarking Flow Behaviors over Graphs

The occurrence of diffusion on a graph is a prevalent and significant phenomenon, as evidenced by the spread of rumors, influenza-like viruses, smart grid failures, and similar events. Comprehending the behaviors of flow is a formidable task, due to the intricate interplay between the distribution of seeds that initiate flow propagation, the propagation model, and the topology of the graph. The study of networks encompasses a diverse range of academic disciplines, including mathematics, physics, social science, and computer science. This interdisciplinary nature of network research is characterized by a high degree of specialization and compartmentalization, and the cooperation facilitated by them is inadequate. From a machine learning standpoint, there is a deficiency in a cohesive platform for assessing algorithms across various domains. One of the primary obstacles to current research in this field is the absence of a comprehensive curated benchmark suite to study the flow behaviors under network scenarios. To address this disparity, we propose the implementation of a novel benchmark suite that encompasses a variety of tasks, baseline models, graph datasets, and evaluation tools. In addition, we present a comprehensive analytical framework that offers a generalized approach to numerous flow-related tasks across diverse domains, serving as a blueprint and roadmap. Drawing upon the outcomes of our empirical investigation, we analyze the advantages and disadvantages of current foundational models, and we underscore potential avenues for further study. The datasets, code, and baseline models have been made available for the public at: https://github.com/XGraphing/XFlo

Improving Adversarial Robustness of DEQs with Explicit Regulations Along the Neural Dynamics

Deep equilibrium (DEQ) models replace the multiple-layer stacking of conventional deep networks with a fixed-point iteration of a single-layer transformation. Having been demonstrated to be competitive in a variety of real-world scenarios, the adversarial robustness of general DEQs becomes increasingly crucial for their reliable deployment. Existing works improve the robustness of general DEQ models with the widely-used adversarial training (AT) framework, but they fail to exploit the structural uniquenesses of DEQ models. To this end, we interpret DEQs through the lens of neural dynamics and find that AT under-regulates intermediate states. Besides, the intermediate states typically provide predictions with a high prediction entropy. Informed by the correlation between the entropy of dynamical systems and their stability properties, we propose reducing prediction entropy by progressively updating inputs along the neural dynamics. During AT, we also utilize random intermediate states to compute the loss function. Our methods regulate the neural dynamics of DEQ models in this manner. Extensive experiments demonstrate that our methods substantially increase the robustness of DEQ models and even outperform the strong deep network baselines.Comment: Accepted at ICML 2023. Our code is available at https://github.com/minicheshire/DEQ-Regulating-Neural-Dynamic

Synergistic toughening of hard, nacre-mimetic MoSi2 coatings by self-assembled hierarchical structure

Like many other intermetallic materials, MoSi2 coatings are typically hard, but prone to catastrophic failure due to their low toughness at ambient temperature. In this paper, a self-assembled hierarchical structure that closely resembles that of nacre (i.e., mother of pearl) was developed in a MoSi2 -based coating through a simple, yet cost-effective, depostion technique. The newly formed coating is tough and can withstand multiple indentations at high loads. Key design features responsible for this remarkable outcome were identified. They include a functionally graded multilayer featuring elastic modulus oscillation, varying sublayer thickness and a columnar structure that are able to attenuate stress concentrations; interlocking boundaries between adjacent sublayers that improve the bonding and arrest the cracks; a transitional layer that bridges the coating and substrate and facilitates load transfer. Moreover, the contributions of six important structural characteristics to damage resistance are quantified using finite elemnet analysis and in an additive manner (i.e., from low- to high-level complexity). The in-situ toughened coating is envisaged to enhance the mechanical performance and extend the lifespan of metal components used in safety-critical applications