508 research outputs found
A one-pass clustering based sketch method for network monitoring
Network monitoring solutions need to cope with increasing network traffic volumes, as a result, sketch-based monitoring methods have been extensively studied to trade accuracy for memory scalability and storage reduction. However, sketches are sensitive to skewness in network flow distributions due to hash collisions, and need complicated performance optimization to adapt to line-rate packet streams. We provide Jellyfish, an efficient sketch method that performs one-pass clustering over the network stream. One-pass clustering is realized by adapting the monitoring granularity from the whole network flow to fragments called subflows, which not only reduces the ingestion rate but also provides an efficient intermediate representation for the input to the sketch. Jellyfish provides the network-flow level query interface by reconstructing the network-flow level counters by merging subflow records from the same network flow. We provide probabilistic analysis of the expected accuracy of both existing sketch methods and Jellyfish. Real-world trace-driven experiments show that Jellyfish reduces the average estimation errors by up to six orders of magnitude for per-flow queries, by six orders of magnitude for entropy queries, and up to ten times for heavy-hitter queries.This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant 61972409; in part by Hong Kong Research Grants Council (RGC) under Grant TRS T41-603/20-R, Grant GRF-16213621, and Grant ITF ACCESS; in part by the Spanish I+D+i project TRAINER-A, funded by MCIN/AEI/10.13039/501100011033, under Grant PID2020-118011GB-C21; and in part by the Catalan Institution
for Research and Advanced Studies (ICREA Academia).Peer ReviewedPostprint (author's final draft
Causal-Based Supervision of Attention in Graph Neural Network: A Better and Simpler Choice towards Powerful Attention
Recent years have witnessed the great potential of attention mechanism in
graph representation learning. However, while variants of attention-based GNNs
are setting new benchmarks for numerous real-world datasets, recent works have
pointed out that their induced attentions are less robust and generalizable
against noisy graphs due to lack of direct supervision. In this paper, we
present a new framework which utilizes the tool of causality to provide a
powerful supervision signal for the learning process of attention functions.
Specifically, we estimate the direct causal effect of attention to the final
prediction, and then maximize such effect to guide attention attending to more
meaningful neighbors. Our method can serve as a plug-and-play module for any
canonical attention-based GNNs in an end-to-end fashion. Extensive experiments
on a wide range of benchmark datasets illustrated that, by directly supervising
attention functions, the model is able to converge faster with a clearer
decision boundary, and thus yields better performances
Robust Mid-Pass Filtering Graph Convolutional Networks
Graph convolutional networks (GCNs) are currently the most promising paradigm
for dealing with graph-structure data, while recent studies have also shown
that GCNs are vulnerable to adversarial attacks. Thus developing GCN models
that are robust to such attacks become a hot research topic. However, the
structural purification learning-based or robustness constraints-based defense
GCN methods are usually designed for specific data or attacks, and introduce
additional objective that is not for classification. Extra training overhead is
also required in their design. To address these challenges, we conduct in-depth
explorations on mid-frequency signals on graphs and propose a simple yet
effective Mid-pass filter GCN (Mid-GCN). Theoretical analyses guarantee the
robustness of signals through the mid-pass filter, and we also shed light on
the properties of different frequency signals under adversarial attacks.
Extensive experiments on six benchmark graph data further verify the
effectiveness of our designed Mid-GCN in node classification accuracy compared
to state-of-the-art GCNs under various adversarial attack strategies.Comment: Accepted by WWW'2
AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks
Transformer-based pre-trained models with millions of parameters require
large storage. Recent approaches tackle this shortcoming by training adapters,
but these approaches still require a relatively large number of parameters. In
this study, AdapterBias, a surprisingly simple yet effective adapter
architecture, is proposed. AdapterBias adds a token-dependent shift to the
hidden output of transformer layers to adapt to downstream tasks with only a
vector and a linear layer. Extensive experiments are conducted to demonstrate
the effectiveness of AdapterBias. The experiments show that our proposed method
can dramatically reduce the trainable parameters compared to the previous works
with a minimal decrease in task performances compared with fine-tuned
pre-trained models. We further find that AdapterBias automatically learns to
assign more significant representation shifts to the tokens related to the task
in consideration.Comment: The first two authors contributed equally. This paper was published
in Findings of NAACL 202
Learning with Free Object Segments for Long-Tailed Instance Segmentation
One fundamental challenge in building an instance segmentation model for a
large number of classes in complex scenes is the lack of training examples,
especially for rare objects. In this paper, we explore the possibility to
increase the training examples without laborious data collection and
annotation. We find that an abundance of instance segments can potentially be
obtained freely from object-centric images, according to two insights: (i) an
object-centric image usually contains one salient object in a simple
background; (ii) objects from the same class often share similar appearances or
similar contrasts to the background. Motivated by these insights, we propose a
simple and scalable framework FreeSeg for extracting and leveraging these
"free" object foreground segments to facilitate model training in long-tailed
instance segmentation. Concretely, we investigate the similarity among
object-centric images of the same class to propose candidate segments of
foreground instances, followed by a novel ranking of segment quality. The
resulting high-quality object segments can then be used to augment the existing
long-tailed datasets, e.g., by copying and pasting the segments onto the
original training images. Extensive experiments show that FreeSeg yields
substantial improvements on top of strong baselines and achieves
state-of-the-art accuracy for segmenting rare object categories
Neuron with Steady Response Leads to Better Generalization
Regularization can mitigate the generalization gap between training and
inference by introducing inductive bias. Existing works have already proposed
various inductive biases from diverse perspectives. However, none of them
explores inductive bias from the perspective of class-dependent response
distribution of individual neurons. In this paper, we conduct a substantial
analysis of the characteristics of such distribution. Based on the analysis
results, we articulate the Neuron Steadiness Hypothesis: the neuron with
similar responses to instances of the same class leads to better
generalization. Accordingly, we propose a new regularization method called
Neuron Steadiness Regularization (NSR) to reduce neuron intra-class response
variance. Based on the Complexity Measure, we theoretically guarantee the
effectiveness of NSR for improving generalization. We conduct extensive
experiments on Multilayer Perceptron, Convolutional Neural Networks, and Graph
Neural Networks with popular benchmark datasets of diverse domains, which show
that our Neuron Steadiness Regularization consistently outperforms the vanilla
version of models with significant gain and low additional computational
overhead.Comment: Accepted by NeurIPS'2
Starch dust explosion and flame burn injury in a patient complicated with severe cellulites caused by non-O1 Vibrio cholerae
- …