Guidance on the principles of language accessibility in National Curriculum Assessments : research background

This review accompanies the document, which describes the principles which should guide the development of clear assessment questions. The purpose of the review is to present and discuss in detail the research underpinning these principles. It begins from the standpoint that National Curriculum assessments, indeed any assessments, should be: - appropriate to the age of the pupils - an effective measure of their abilities, skills and concept development - fair to all irrespective of gender, language, religion, ethnic or social origin or disability. (Ofqual, 2011) The Regulatory Framework for National Assessments: National Curriculum and Early Years Foundation Stage (Ofqual, 2011) sets out a number of common criteria which apply to all aspects of the development and implementation of National Assessments. One of these criteria refers to the need for assessment procedures to minimise bias: “The assessment should minimise bias, differentiating only on the basis of each learner’s ability to meet National Curriculum requirements” (Section 5.39, page 16). The Framework goes on to argue that: “Minimising bias is about ensuring that an assessment does not produce unreasonably adverse outcomes for particular groups of learners” (Annex 1, page 29). This criterion reinforces the guiding principle that any form of assessment should provide information about the knowledge and understanding of relevant content material. That is to say that the means through which this knowledge and understanding is examined, the design of the assessment and the language used should as far as possible be transparent, and should not influence adversely the performance of those being assessed. There is clearly a large number of ways in which any given assessment task can be presented and in which questions can be asked. Some of these ways will make the task more accessible – that is, easier to complete successfully – and some will get in the way of successful completion. Section 26 of the Fair Access by Design (Ofqual, 2010) document lists a number of guiding principles for improving the accessibility of assessment questions, although the research basis for these principles is not made completely clear in that document. The aim of the current review is to examine the research background more closely in order to provide a more substantial basis for a renewed set of principles to underpin the concept of language accessibility. In the review, each section will be prefaced by a statement of the principles outlined in Guidance on the Principles of Language Accessibility in National Curriculum Assessments and then the research evidence underpinning these principles will be reviewed

Practice, principles, and theory in the design of instructional text

This study is concerned with an analysis of the research arising from three quite different perspectives on instructional text - the `physical characteristics' research (legibility, layout, and readability), the `improvement of text' research (visual illustrations, adjunct aids, and typographical cueing), and the `learning theories' research (representation of knowledge, human memory, and quality of learning). From this analysis there is synthesised principles for the design of instructional text against which heuristic practice in text design is evaluated and from which a nascent theory of instructional text design is evolved. The principles derived from the various research perspectives provide a basis for the manipulation of text design elements in order to ensure that (a) existing knowledge in the reader can be activated, and (b) new knowledge can be assimilated in a manner facilitative of comprehension by (i) presentation in a structured and organised way, and (ii) appropriately highlighted through verbal and typographic cueing supported, as required, by verbal illustration and organisation. The emerging theory of instructional text design suggests: a topical analysis to determine the heirarchic relationship of ideas within the topic and the desired learning outcomes or objectives; a consideration of the linguistic aspects of the text; a consideration of the role of visual illustrations; and a consideration of the physical parameters of the text. These activities are concerned, respectively, with the design areas of structure and organisation, readability, visual illustration, and legibility, and are summed up in the acronym SORVIL

Usability issues and design principles for visual programming languages

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Despite two decades of empirical studies focusing on programmers and the problems with programming, usability of textual programming languages is still hard to achieve. Its younger relation, visual programming languages (VPLs) also share the same problem of poor usability. This research explores and investigates the usability issues relating to VPLs in order to suggest a set of design principles that emphasise usability. The approach adopted focuses on issues arising from the interaction and communication between the human (programmers), the computer (user interface), and the program. Being exploratory in nature, this PhD reviews the literature as a starting point for stimulating and developing research questions and hypotheses that experimental studies were conducted to investigate. However, the literature alone cannot provide a fully comprehensive list of possible usability problems in VPLs so that design principles can be confidently recommended. A commercial VPL was, therefore, holistically evaluated and a comprehensive list of usability problems was obtained from the research. Six empirical studies employing both quantitative and qualitative methodology were undertaken as dictated by the nature of the research. Five of these were controlled experiments and one was qualitative-naturalistic. The experiments studied the effect of a programming paradigm and of representation of program flow on novices' performances. The results indicated superiority of control-flow programs in relation to data-flow programs; a control-flow preference among novices; and in addition that directional representation does not affect performance while traversal direction does - due to cognitive demands imposed upon programmers. Results of the qualitative study included a list of 145 usability problems and these were further categorised into ten problem areas. These findings were integrated with other analytical work based upon the review of the literature in a structured fashion to form a checklist and a set of design principles for VPLs that are empirically grounded and evaluated against existing research in the literature. Furthermore, an extended framework for Cognitive Dimensions of Notations is also discussed and proposed as an evaluation method for diagrammatic VPLs on the basis of the qualitative study. The above consists of the major findings and deliverables of this research. Nevertheless, there are several other findings identified on the basis of the substantial amount of data obtained in the series of experiments carried out, which have made a novel contribution to knowledge in the fields of Human-Computer Interaction, Psychology of Programming, and Visual Programming Languages

Towards a new model of readability

This thesis attempts to develop a new model for a renewed concept of readability. The thesis begins by discussing the rationale for carrying out this research. Next, the extensive literature around the topic of readability is reviewed. The literature suggests that most research into readability has stemmed from a positivist paradigm, and has used quantitative methods to assess text comprehensibility. This approach has been widely criticised and, recently, more qualitative methods stemming from an interpretive paradigm have been employed. It seems that both quantitative and qualitative methods have strengths and limitations. Therefore, the research I have carried out has explored the concept of readability by combining these two research approaches. The data collection methods include readability formulae; text feature analyses; miscue analyses; retellings and interviews. This research has been conducted in the United Kingdom and involved 16 male and 16 female pupils with an age range from 6 to 11 years old. All the participants were fluent readers. Data were analysed using; (1) six online readability formulae - ATOS (1997); Dale-Chall (1948); Flesch-Kincaid (1948); FOG (1952); SMOG (1969); and Spache (1953); (2) Reading Miscue Inventory (Goodman, Watson & Burke, 2005); (3) Judging Richness of Retellings (Irwin & Mitchell, 1983); (4) text feature analysis forms; and (5) a cross-interview analysis approach. Two computer software programmes i.e Statistical Package for the Social Sciences (SPSS 17) and Qualitative Data Analysis (Nvivo 7) were used to organise and analyse the quantitative and qualitative data. The findings suggest that the concept of readability is influenced by both reader and text factors. The reader factors involve a complex relationship of nine embedded elements within the reader, namely interest, prior knowledge, attitude, reading ability, motivation, purpose of reading, engagement, age and gender. The text factors include eight elements, these being the physical features of the text, genre, content, author, linguistic difficulties, legibility, illustrations and organization of the text. This research comes to the conclusion that the concept of readability is a complex matching process involving the dynamic interaction between both reader and text factors and bound by certain contexts

Knowledge restructing and the development of expertise in computer programming

This thesis reports a number of empirical studies exploring the development of expertise in computer programming. Experiments 1 and 2 are concerned with the way in which the possession of design experience can influence the perception and use of cues to various program structures. Experiment 3 examines how violations to standard conventions for constructing programs can affect the comprehension of expert, intermediate and novice subjects. Experiment 4 looks at the differences in strategy that are exhibited by subjects of varying skill level when constructing programs in different languages. Experiment 5 takes these ideas further to examine the temporal distribution of different forms of strategy during a program generation task. Experiment 6 provides evidence for salient cognitive structures derived from reaction time and error data in the context of a recognition task. Experiments 7 and 8 are concerned with the role of working memory in program generation and suggest that one aspect of expertise in the programming domain involves the acquisition of strategies for utilising display-based information. The final chapter attempts to bring these experimental findings together in terms of a model of knowledge organisation that stresses the importance of knowledge restructuring processes in the development of expertise. This is contrasted with existing models which have tended to place emphasis upon schemata acquisition and generalisation as the fundamental modes of learning associated with skill development. The work reported here suggests that a fine-grained restructuring of individual schemata takes places during the later stages of skill development. It is argued that those mechanisms currently thought to be associated with the development of expertise may not fully account for the strategic changes and the types of error typically found in the transition between novice, intermediate and expert problem solvers. This work has a number of implications for existing theories of skill acquisition. In particular, it questions the ability of such theories to account for subtle changes in the various manifestations of skilled performance that are associated with increasing expertise. Secondly, the work reported in this thesis attempts to show how specific forms of training might give rise to the knowledge restructuring process that is proposed. Finally, the thesis stresses the important role of display-based problem solving in complex tasks such as programming and highlights the role of programming language notation as a mediating factor in the development and acquisition of problem solving strategies

Reading in the Content Area: Its Impact on Teaching in the Social Studies Classroom

This study focused on evaluating the sufficiency of research in reading in the content area used to instruct classroom teachers. The research used was conducted between 1970 and 2000 and incorporated into textbooks written between 1975 and 2005. Studies examined were those reported in the following journals: Review of Educational Research, Review of Research in Education, Social Education, Theory and Research in Social Education, Reading Research Quarterly, and Research in the Teaching of English. Some attention was also given to two major educational curriculum and issue journals- Educational Leadership and Phi Delta Kappan as these sources might identify relevant research studies for further investigation. References cited in more than one text helped identify and establish a baseline of those studies considered most significant by textbook authors. The findings of this study showed that the majority of citations looked at the following themes: -Learners acquire meaning from the printed page through thought. -Reading can and should be done for different purposes using a variety of materials. -A number of techniques can be used to teach reading skills. -Reading materials need to be selected according to changes in a child‘s interests. -Reading ability is the level of reading difficulty that students can cope with. It depends on ability rather than age or grade level. -Readability contributes to both the reader‘s degree of comprehension and the need for teacher assistance when reading difficulty exceeds the reader‘s capability. -Reading instruction, in some form, needs to be carried on into the secondary grades. Research findings from the 1970s were concerned with reading strategies, reading skills, reading comprehension, readability, attitudes towards reading, vocabulary, study skills, and content area reading programs. In the 1980s research cited in content area reading books looked at reading comprehension, reading skills, vocabulary, learning strategies, curriculum issues, purposes for reading and writing, content area reading programs, readability, schema theory, thinking skills, summarizing, comprehension strategies, and cooperative learning. By the 1990s more research cited in content area reading books focused on reading strategies, curriculum issues, how to read documents and graphs, reading skills, vocabulary, attitudes towards reading, reading comprehension, and activating background knowledge

Using signaling to aid computer program comprehension

Guidelines for using style to improve computer program comprehension have often been proposed without empirical testing. This thesis reports on the results of three controlled experiments that investigated ways program style may be used to aid comprehension of source code listings. Experiments 1 and 2 were conducted using advanced computer science students as subjects and short Pascal programs. Results showed that student programmers used meaningful identifier names as important sources of information during comprehension of short programs. A review of the literature showed the need for the thesis' proposed methodology for designing controlled experiments on program comprehension that produce results which generalize well to situations involving professional programmers working on real world tasks. This methodology was used to design Experiment 3. Text comprehension researchers have investigated the use of signaling, or the placement of non-content information, in a text in order to emphasize certain ideas and/or clarify the organization. Experiment 3 investigated the role of signaling in another domain, that of computer program source code listings. The experiment had professional programmers study a 913-line C program. Three types of signals were investigated: preview statements, headings, and typographic cueing. The major results were (a) meaningful module names served as headings in the source code listing and helped professional programmers understand and locate information in the program; (b) header comments, when written as preview statements, helped professional programmers understand and locate information in the program; (c) typographic cueing, designed to provide emphasis and segmentation cues, helped programmers understand the program; and (d) the effects of meaningful names, header comments, and typographic cueing were additive. No significant interactions of effects were observed. Based on these results, guidelines are proposed for ways programmers may use comments and module names in source code to act as signals that aid future readers. In addition, guidelines are suggested for adding typographic signaling to provide emphasis to the comments and names

Using visual representations to improve instructional materials for distance education computing students

Understanding how to develop instructional materials for distance education students is a challenging problem, but it is exacerbated when a domain is complex to teach, such as computer science. Visual representations have a history of use in computing as a means to alleviate the difficulties of learning abstract concepts. However, it is not clear whether improvements observed are as a result of improvements in the visual representations used in instructional materials or due to individual differences in students. This research examines the two themes of individual differences and visual representation in order to investigate how they collectively impact on improving instructional materials for distance education students studying computer science. It investigates the impact of different representations on learning while additionally investigating the relationship between individual differences and student learning.The research in this thesis shows that visual representations are important in designing instructional materials. In particular, texts with visual representations have the power to cue students to perceive instructional materials as easier to process and more engaging.Investigation into the impact of concrete high-imagery versus abstract low-imagery visual representations illustrated that concrete visual representations incurred fewer cognitive overheads for computer science students and were able to ameliorate the challenges of learning computing.The research in this thesis into individual differences demonstrated that Imagers did benefit more from studying instructional materials containing text with visual components. However the research indicates that appropriate selection of individual difference tests is dependent upon the application, i.e., whether the results are to be used to assess generalised tendencies or episodes in learning and whether the tests examine underlying approaches to cognition or practices in education.An underlying question was whether students studying instructional materials containing low-imagery visual representations would cope as well as those studying high-imagery ones. Accomplished learners demonstrated that they could perform as well as with those receiving high-imagery visual representations. However, studying and recalling these materials did incur more cognitive processing.This thesis argues that improving instructional materials by including appropriate visual representations is a useful basis for improving learning for distance education computer science students

A framework for the design of usable electronic text

This thesis examines the human issues underlying the design and usability of electronic text systems. In so doing it develops a framework for the conceptualisation of these issues that aims to guide designers of electronic texts in their attempts to produce usable systems. The thesis commences with a review of the traditional human factors literature on electronic text according to three basic themes: its concern with perceptual, manipulatory and structural issues. From this examination it is concluded that shortcomings in translating this work into design result from the adoption of overly narrow uni-disciplinary views of reading taken from cognitive psychology and information science which are inappropriate to serve the needs of electronic text designers. In an attempt to provide a more relevant description of the reading process a series of studies examining readers and their views as well as uses of texts is reported. In the first, a repertory grid based investigation revealed that all texts can be described in reader-relvant terms according to three criteria: why a text is read, what a text contains and how it is read. These criteria then form the basis of two investigations of reader-text interaction using academic journals and user manuals. The results of these studies highlighted the need to consider readers' models of a document's structure in discussing text usability. Subsequent experimental work on readers' models of academic articles demonstrated not only that such models are important aspects of reader-text interaction but that data of this form could usefully be employed in the design of an electronic text system. The proposed framework provides a broad, qualitative model of the important issues for designers to consider when developing a product It consists of four interactive elements that focus attention on aspects of reading that have been identified as central to usability. Simple tests of the utility and validity of the framework are reported and it is shown that the framework both supports reasoned analysis and subsequent prediction of reader behaviour as well as providing a parsimonious account of their verbal utterances while reading. The thesis concludes with an analysis of the likely uses of such a framework and the potential for electronic text systems in an increasingly information-hungry world