CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models

Code generation models based on the pre-training and fine-tuning paradigm have been increasingly attempted by both academia and industry, resulting in well-known industrial models such as Codex, CodeGen, and PanGu-Coder. To evaluate the effectiveness of these models, multiple existing benchmarks are proposed, including only cases of generating a standalone function, i.e., a function that may invoke or access only built-in functions and standard libraries. However, non-standalone functions, which typically are not included in the existing benchmarks, constitute more than 70% of the functions in popular open-source projects, and evaluating models' effectiveness on standalone functions cannot reflect these models' effectiveness on pragmatic code generation scenarios. To help bridge the preceding gap, in this paper, we propose a benchmark named CoderEval, consisting of 230 Python and 230 Java code generation tasks carefully curated from popular real-world open-source projects and a self-contained execution platform to automatically assess the functional correctness of generated code. CoderEval supports code generation tasks from six levels of context dependency, where context refers to code elements such as types, APIs, variables, and consts defined outside the function under generation but within the dependent third-party libraries, current class, file, or project. CoderEval can be used to evaluate the effectiveness of models in generating code beyond only standalone functions. By evaluating three code generation models on CoderEval, we find that the effectiveness of these models in generating standalone functions is substantially higher than that in generating non-standalone functions. Our analysis highlights the current progress and pinpoints future directions to further improve a model's effectiveness by leveraging contextual information for pragmatic code generation

Expert consensus on early childhood caries management

Abstract Early childhood caries (ECC) is a significant chronic disease of childhood and a rising public health burden worldwide. ECC may cause a higher risk of new caries lesions in both primary and permanent dentition, affecting lifelong oral health. The occurrence of ECC has been closely related to the core microbiome change in the oral cavity, which may be influenced by diet habits, oral health management, fluoride use, and dental manipulations. So, it is essential to improve parental oral health and awareness of health care, to establish a dental home at the early stage of childhood, and make an individualized caries management plan. Dental interventions according to the minimally invasive concept should be carried out to treat dental caries. This expert consensus mainly discusses the etiology of ECC, caries-risk assessment of children, prevention and treatment plan of ECC, aiming to achieve lifelong oral health