As Natural Language Processing (NLP) algorithms continually achieve new
milestones, out-of-distribution generalization remains a significant challenge.
This paper addresses the issue of multi-source adaptation for unfamiliar
domains: We leverage labeled data from multiple source domains to generalize to
unknown target domains at training. Our innovative framework employs
example-based Hypernetwork adaptation: a T5 encoder-decoder initially generates
a unique signature from an input example, embedding it within the source
domains' semantic space. This signature is subsequently utilized by a
Hypernetwork to generate the task classifier's weights. We evaluated our method
across two tasks - sentiment classification and natural language inference - in
29 adaptation scenarios, where it outpaced established algorithms. In an
advanced version, the signature also enriches the input example's
representation. We also compare our finetuned architecture to few-shot GPT-3,
demonstrating its effectiveness in essential use cases. To our knowledge, this
marks the first application of Hypernetworks to the adaptation for unknown
domains.Comment: First two authors contributed equally to this work. Our code and data
are available at: https://github.com/TomerVolk/Hyper-PAD