Most existing chain-of-thought (CoT) prompting methods suffer from the issues
of generalizability and consistency, as they often rely on instance-specific
solutions that may not be applicable to other cases and lack task-level
consistency in their reasoning steps. To address these limitations, we propose
a comprehensive framework, StrategyLLM, harnessing the capabilities of LLMs to
tackle various tasks. The framework improves generalizability by formulating
general problem-solving strategies and enhances consistency by producing
consistent solutions using these strategies. StrategyLLM employs four LLM-based
agents: strategy generator, executor, optimizer, and evaluator, working
together to generate, evaluate, and select promising strategies for a given
task automatically. The experimental results demonstrate that StrategyLLM
outperforms the competitive baseline CoT-SC that requires human-annotated
solutions on 13 datasets across 4 challenging tasks without human involvement,
including math reasoning (39.2% β 43.3%), commonsense reasoning
(70.3% β 72.5%), algorithmic reasoning (51.7% β 62.0%),
and symbolic reasoning (30.0% β 79.2%)