In this work, we conduct an assessment of the optimization capabilities of
LLMs across various tasks and data sizes. Each of these tasks corresponds to
unique optimization domains, and LLMs are required to execute these tasks with
interactive prompting. That is, in each optimization step, the LLM generates
new solutions from the past generated solutions with their values, and then the
new solutions are evaluated and considered in the next optimization step.
Additionally, we introduce three distinct metrics for a comprehensive
assessment of task performance from various perspectives. These metrics offer
the advantage of being applicable for evaluating LLM performance across a broad
spectrum of optimization tasks and are less sensitive to variations in test
samples. By applying these metrics, we observe that LLMs exhibit strong
optimization capabilities when dealing with small-sized samples. However, their
performance is significantly influenced by factors like data size and values,
underscoring the importance of further research in the domain of optimization
tasks for LLMs