Standardized and quantified evaluation of machine behaviors is a crux of
understanding LLMs. In this study, we draw inspiration from psychometric
studies by leveraging human personality theory as a tool for studying machine
behaviors. Originating as a philosophical quest for human behaviors, the study
of personality delves into how individuals differ in thinking, feeling, and
behaving. Toward building and understanding human-like social machines, we are
motivated to ask: Can we assess machine behaviors by leveraging human
psychometric tests in a principled and quantitative manner? If so, can we
induce a specific personality in LLMs? To answer these questions, we introduce
the Machine Personality Inventory (MPI) tool for studying machine behaviors;
MPI follows standardized personality tests, built upon the Big Five Personality
Factors (Big Five) theory and personality assessment inventories. By
systematically evaluating LLMs with MPI, we provide the first piece of evidence
demonstrating the efficacy of MPI in studying LLMs behaviors. We further devise
a Personality Prompting (P^2) method to induce LLMs with specific personalities
in a controllable way, capable of producing diverse and verifiable behaviors.
We hope this work sheds light on future studies by adopting personality as the
essential indicator for various downstream tasks, and could further motivate
research into equally intriguing human-like machine behaviors.Comment: Accepted at NeurIPS 2023 (Spotlight