The Behavior of Large Language Models When Prompted to Generate Code Explanations

This paper systematically investigates the generation of code explanations by Large Language Models (LLMs) for code examples commonly encountered in introductory programming courses. Our findings reveal significant variations in the nature of code explanations produced by LLMs, influenced by factors such as the wording of the prompt, the specific code examples under consideration, the programming language involved, the temperature parameter, and the version of the LLM. However, a consistent pattern emerges for Java and Python, where explanations exhibit a Flesch-Kincaid readability level of approximately 7-8 grade and a consistent lexical density, indicating the proportion of meaningful words relative to the total explanation size. Additionally, the generated explanations consistently achieve high scores for correctness, but lower scores on three other metrics: completeness, conciseness, and specificity

A comparison of decontextualized and contextualized reading skills in persons with severe aphasia

A repeated measures comparative design compared the reading comprehension accuracy scores of three participants across two conditions: Condition A -- Decontextualized Stimulus Reading Task (DSRT), resembling traditional reading therapy, and Condition B -- Contextual Choice Reading Conversation (CCRC) with a communication partner. In the DSRT condition, participants read a sentence-length question prior to selecting one of 3-to-5 printed responses with no supports. In CCRC, partners presented graphic sentence stimuli representing conversational questions and response choices with the following supports: simultaneous auditory input, supplemental drawings or gestures, natural repetitions, consistent topic, and conversational order. Experimental conditions were administered in counterbalanced order across 5 sessions. Responses were scored for accuracy based on factual world knowledge (DSRT) or verification by spouses or family members, for a maximum of 10 points for each session per condition. Raw scores, mean scores, and standard deviations from each condition were compared with descriptive and nonparametric statistics. Results showed significant improvement in reading comprehension accuracy when the CCRC method was applied. Clinically, this suggests that persons with severe aphasia can read well enough to use partner supported conversations that utilize reading comprehension

Guidance on the principles of language accessibility in National Curriculum Assessments : research background

This review accompanies the document, which describes the principles which should guide the development of clear assessment questions. The purpose of the review is to present and discuss in detail the research underpinning these principles. It begins from the standpoint that National Curriculum assessments, indeed any assessments, should be: - appropriate to the age of the pupils - an effective measure of their abilities, skills and concept development - fair to all irrespective of gender, language, religion, ethnic or social origin or disability. (Ofqual, 2011) The Regulatory Framework for National Assessments: National Curriculum and Early Years Foundation Stage (Ofqual, 2011) sets out a number of common criteria which apply to all aspects of the development and implementation of National Assessments. One of these criteria refers to the need for assessment procedures to minimise bias: “The assessment should minimise bias, differentiating only on the basis of each learner’s ability to meet National Curriculum requirements” (Section 5.39, page 16). The Framework goes on to argue that: “Minimising bias is about ensuring that an assessment does not produce unreasonably adverse outcomes for particular groups of learners” (Annex 1, page 29). This criterion reinforces the guiding principle that any form of assessment should provide information about the knowledge and understanding of relevant content material. That is to say that the means through which this knowledge and understanding is examined, the design of the assessment and the language used should as far as possible be transparent, and should not influence adversely the performance of those being assessed. There is clearly a large number of ways in which any given assessment task can be presented and in which questions can be asked. Some of these ways will make the task more accessible – that is, easier to complete successfully – and some will get in the way of successful completion. Section 26 of the Fair Access by Design (Ofqual, 2010) document lists a number of guiding principles for improving the accessibility of assessment questions, although the research basis for these principles is not made completely clear in that document. The aim of the current review is to examine the research background more closely in order to provide a more substantial basis for a renewed set of principles to underpin the concept of language accessibility. In the review, each section will be prefaced by a statement of the principles outlined in Guidance on the Principles of Language Accessibility in National Curriculum Assessments and then the research evidence underpinning these principles will be reviewed

The Quality of Newly Local Content Based Instructional Reading Materials for Senior High School Students

This study aimed to analyze the quality of advanced local content-based instructional reading material. It was ten descriptive texts and a reading comprehension test with the topic focused on Bangka Island tourism places. The developed product was specially constructed by considering the reading level of senior high school students in Pangkalpinang, Indonesia, which is identified at level four. The quality of the product reflects on its effectiveness and criterion-related validity. The point is determined from students' scores on the reading comprehension test administered after they have been taught using the developed product in 6 meeting sessions. The criterion-related validity of developed reading comprehension was recognized by measuring the correlation between students' scores on progressive reading comprehension tests and their scores on standardized reading level tests. Based on the result, the developed product can be considered adequate because 35 students (85.36%) out of 41 students in reading comprehension test accomplished the minimum mastery criterion, which was 70. The product was also considered good in terms of its criterion-related validity because the correlation coefficient was 0.618. Since findings show that the developed local content-based instructional reading materials are effective and valid, it can conclude that the developed product has good quality when used as instructional reading materials for senior high school students. Keywords: Instructional Reading Materials, Local Content, Senior High School Student