1 research outputs found
Code Llama: Open Foundation Models for Code
We release Code Llama, a family of large language models for code based on
Llama 2 providing state-of-the-art performance among open models, infilling
capabilities, support for large input contexts, and zero-shot instruction
following ability for programming tasks. We provide multiple flavors to cover a
wide range of applications: foundation models (Code Llama), Python
specializations (Code Llama - Python), and instruction-following models (Code
Llama - Instruct) with 7B, 13B and 34B parameters each. All models are trained
on sequences of 16k tokens and show improvements on inputs with up to 100k
tokens. 7B and 13B Code Llama and Code Llama - Instruct variants support
infilling based on surrounding content. Code Llama reaches state-of-the-art
performance among open models on several code benchmarks, with scores of up to
53% and 55% on HumanEval and MBPP, respectively. Notably, Code Llama - Python
7B outperforms Llama 2 70B on HumanEval and MBPP, and all our models outperform
every other publicly available model on MultiPL-E. We release Code Llama under
a permissive license that allows for both research and commercial use