Rubric Design for Separating the Roles of Open-Ended Assessments

End-of-course assessments play important roles in the ongoing attempt to improve instruction in physics courses. Comparison of students' performance on assessments before and after instruction gives a measure of student learning. In addition, analysis of students' answers to assessment items provides insight into students' difficulties with specific concepts and practices. While open-ended assessments scored with detailed rubrics provide useful information about student reasoning to researchers, end users need to score students' responses so that they may obtain meaningful feedback on their instruction. One solution that satisfies end users and researchers is a grading rubric that separates scoring student work and uncovering student difficulties. We have constructed a separable rubric for the Colorado Classical Mechanics/Math Methods Instrument that has been used by untrained graders to score the assessment reliably, and by researchers to unpack common student difficulties. Here we present rubric development, measures of inter-rater reliability, and some uncovered student difficulties.Comment: 4 pages, PERC 2014 Proceeding

Universal Back-End Design

Accessibility in back-end development is often overlooked, with the majority of discussions and efforts centered on front-end design. To make applications usable for a wider audience, developers must also prioritize incorporating accessibility from the back-end. Back-end web accessibility encompasses the design and development of web-based systems and applications that are accessible to all users, including those with disabilities. This involves optimizing the underlying code and infrastructure for accessibility and implementing features that enable users with disabilities to navigate and interact with the site or application. Ensuring back-end web accessibility is crucial for creating an inclusive online environment accessible to everyone, regardless of their abilities. Presently, there is a significant gap in research regarding back-end accessibility specifics. This study investigates the challenges in implementing back-end accessibility, explores best practices, and aims to facilitate future research on its impact on user experience and system performance

Talking With Scholars: Developing a Research Environment for Oral History Collections

Scholars are yet to make optimal use of Oral History collections. For the uptake of digital research tools in the daily working practice of researchers, practices and conventions commonly adhered to in the subfields in the humanities should be taken into account during development. To this end, in the Oral History Today project a research tool for exploring Oral History collections is developed in close collaboration with scholarly researchers. This paper describes four stages of scholarly research and the first steps undertaken to incorporate requirements of these stages in a digital research environment.Comment: Demo paper for The 2nd International Workshop on Supporting Users Exploration of Digital Libraries (Malta, 26th September, 2013

Developing Feasible and Effective School-Based Interventions for Children With ASD: A Case Study of the Iterative Development Process

Despite an emphasis on identifying evidence-based practices among researchers and using evidence-based practices among professionals in the field of education, there are still problems with uptake and implementation in real-world settings. This lack of diffusion of practices is evident in educational programming for children with autism spectrum disorder (ASD). One solution is to use an iterative process to develop interventions in which researchers work in collaboration with the end users to test and refine interventions. However, there are very few guidelines for developing feasible and effective interventions through these iterative processes. This article provides a description of the iterative process used to develop the Advancing Social-Communication and Play (ASAP) intervention, a supplemental program designed for public preschool classrooms serving students with ASD, and examples of how data from the sequence of iterative design studies shaped the intervention development. The research team offers guidelines for other researchers looking to engage in intervention development using an iterative process in the context of partnerships with end users, including suggestions for planning and executing an intervention development grant

The Configuration of Older Users as Drivers of Innovation in the Design of Digital Technologies

This paper develops hypotheses on the discovery of older people as "users" in the publicly funded development of digital technologies and underlying innovation policy motives. We then describe the effects of this innovation policy on the development of products and older people as their users in the context of Ambient Assisted Living (AAL). To this end, we reconstruct the involvement of users in two AAL funding programs, one at the European level and one at the national level (Germany). Based on this, we discuss the resulting consequences by describing how older people are configured as users in technology development with a focus on the concept of user-centered design (UCD) and what this configuration means for the developed technologies as well as for the older users. We describe how the participation of older people in technology development projects is a complex task that is not without controversy in social science research on user participation. We conclude by arguing for alternative technology development strategies and funding practices

Assessing housing quality and its impact on health, safety and sustainability

Background The adverse health and environmental effects of poor housing quality are well established. A central requirement for evidence-based policies and programmes to improve housing standards is a valid, reliable and practical way of measuring housing quality that is supported by policy agencies, the housing sector, researchers and the public. Methods This paper provides guidance on the development of housing quality-assessment tools that link practical measures of housing conditions to their effects on health, safety and sustainability, with particular reference to tools developed in New Zealand and England. Results The authors describe how information on housing quality can support individuals, agencies and the private sector to make worthwhile improvements to the health, safety and sustainability of housing. The information gathered and the resultant tools developed should be guided by the multiple purposes and end users of this information. Other important issues outlined include deciding on the scope, detailed content, practical administration issues and how the information will be analysed and summarised for its intended end users. There are likely to be considerable benefits from increased international collaboration and standardisation of approaches to measuring housing hazards. At the same time, these assessment approaches need to consider local factors such as climate, geography, culture, predominating building practices, important housing-related health issues and existing building codes. Conclusions An effective housing quality-assessment tool has a central role in supporting improvements to housing. The issues discussed in this paper are designed to motivate and assist the development of such tools

Exploiting the potential of large databases of electronic health records for research using rapid search algorithms and an intuitive query interface.

Objective: UK primary care databases, which contain diagnostic, demographic and prescribing information for millions of patients geographically representative of the UK, represent a significant resource for health services and clinical research. They can be used to identify patients with a specified disease or condition (phenotyping) and to investigate patterns of diagnosis and symptoms. Currently, extracting such information manually is time-consuming and requires considerable expertise. In order to exploit more fully the potential of these large and complex databases, our interdisciplinary team developed generic methods allowing access to different types of user. Materials and methods: Using the Clinical Practice Research Datalink database, we have developed an online user-focused system (TrialViz), which enables users interactively to select suitable medical general practices based on two criteria: suitability of the patient base for the intended study (phenotyping) and measures of data quality. Results: An end-to-end system, underpinned by an innovative search algorithm, allows the user to extract information in near real-time via an intuitive query interface and to explore this information using interactive visualization tools. A usability evaluation of this system produced positive results. Discussion: We present the challenges and results in the development of TrialViz and our plans for its extension for wider applications of clinical research. Conclusions: Our fast search algorithms and simple query algorithms represent a significant advance for users of clinical research databases

The Cognitive Atlas: Employing Interaction Design Processes to Facilitate Collaborative Ontology Creation

The Cognitive Atlas is a collaborative knowledge-building project that aims to develop an ontology that characterizes the current conceptual framework among researchers in cognitive science and neuroscience. The project objectives from the beginning focused on usability, simplicity, and utility for end users. Support for Semantic Web technologies was also a priority in order to support interoperability with other neuroscience projects and knowledge bases. Current off-the-shelf semantic web or semantic wiki technologies, however, do not often lend themselves to simple user interaction designs for non-technical researchers and practitioners; the abstract nature and complexity of these systems acts as point of friction for user interaction, inhibiting usability and utility. Instead, we take an alternate interaction design approach driven by user centered design processes rather than a base set of semantic technologies. This paper reviews the initial two rounds of design and development of the Cognitive Atlas system, including interactive design decisions and their implementation as guided by current industry practices for the development of complex interactive systems