Natural language prompts have been shown to facilitate cross-task
generalization for large language models. However, with no or limited labeled
examples, the cross-task performance is highly sensitive to the choice of
prompts, while selecting a high-performing prompt is challenging given the
scarcity of labels. To address the issue, we propose a Zero-Label Prompt
Selection (ZPS) method that selects prompts without any labeled data or
gradient update. Specifically, given the candidate human-written prompts for a
task, ZPS labels a set of unlabeled data with a prompt ensemble and uses the
pseudo-labels for prompt selection. Experiments show that ZPS improves over
prior methods by a sizeable margin in zero-label performance. We also extend
ZPS to a few-shot setting and show its advantages over strong baselines such as
prompt tuning and model tuning