Challenges for IT-Enabled Formative Assessment of Complex 21st Century Skills

In this article, we identify and examine opportunities for formative assessment provided by information technologies (IT) and the challenges which these opportunities present. We address some of these challenges by examining key aspects of assessment processes that can be facilitated by IT: datafication of learning; feedback and scaffolding; peer assessment and peer feedback. We then consider how these processes may be applied in relation to the assessment of horizontal, general complex 21st century skills (21st CS), which are still proving challenging to incorporate into curricula as well as to assess. 21st CS such as creativity, complex problem solving, communication, collaboration and self-regulated learning contain complex constructs incorporating motivational and affective components. Our analysis has enabled us to make recommendations for policy, practice and further research. While there is currently much interest in and some progress towards the development of learning/assessment analytics for assessing 21st CS, the complexity of assessing such skills, together with the need to include affective aspects means that using IT-enabled techniques will need to be combined with more traditional methods of teacher assessment as well as peer assessment for some time to come. Therefore learners, teachers and school leaders must learn how to manage the greater variety of sorts and sources of feedback including resolving tensions of inconsistent feedback from different sources

The Staphylococcus aureus RNome and Its Commitment to Virulence

Staphylococcus aureus is a major human pathogen causing a wide spectrum of nosocomial and community-associated infections with high morbidity and mortality. S. aureus generates a large number of virulence factors whose timing and expression levels are precisely tuned by regulatory proteins and RNAs. The aptitude of bacteria to use RNAs to rapidly modify gene expression, including virulence factors in response to stress or environmental changes, and to survive in a host is an evolving concept. Here, we focus on the recently inventoried S. aureus regulatory RNAs, with emphasis on those with identified functions, two of which are directly involved in pathogenicity

Exploring subjective constructions of health in China: a Q-methodological investigation

Background: With an increasing awareness of people’s satisfaction and feeling, health-related quality of life (HRQoL) has become an essential aspect of measuring health. HRQoL is fundamentally a foreign concept introduced to China from the West. While a growing number of studies applied western HRQoL measures, few content validity tests examined the legitimacy of applying Western developed HRQoL measures in a Chinese cultural setting. If there are distinct differences in health conceptualisation between China and the West, it can be argued that those western measures may fail to ask the most appropriate and important questions among a Chinese population in assessing health. As a limited number of studies have investigated Chinese people’s understandings of health, this study aimed to explore how health is defined and described in China. Methods: A Q-methodological study was conducted to explore subjective constructions of health among Chinese participants. A scoping review of Chinese generic HRQoL measures, supplemented by a series of qualitative interviews conducted in China, produced a list of 42 statements representing aspects of health considered as being important in a Chinese cultural setting. Chinese participants in face-to-face interviews ranked and sorted these statements. Data were analysed to identify clusters of participants who shared a similar perspective, using a by-person factor analysis procedure. Results: 110 Chinese participants with various demographics characteristics completed sorting interviews. Five independent factors emerged: (I) “Physical independence and social interaction skills”; (II) “Physical health”; (III) “Sensations and feelings”; (IV) “Lifestyles”; (V) “Learning and working abilities”. Conclusions: The Q-study showed that many health statements were rated highly as most important by a diverse range of Chinese participants but were not covered in the commonly used Western HRQoL measure EQ-5D. It then suggests that the EQ-5D descriptive system might need modification to improve its capacity to measure health status in China. The study thus raises a general question as to how appropriate the Western-developed HRQoL measures are when used to assess health in a significantly different cultural setting

Using subject-matter experts to assess content representation: An MDS analysis

Demonstration of content domain representation is of central importance in test validation. An expanded version of the method of content evaluation proposed by Sireci & Geisinger (1992) was evaluated with respect to a national licensure examination and a nationally standardized social studies achievement test. Two groups of 15 subject-matter experts (SMEs) rated the similarity of all item pairs comprising a test, and then rated the relevance of the items to the content domains listed in the test blueprints. The similarity ratings were analyzed using multidimensional scaling (MDS); the item relevance ratings were analyzed using procedures proposed by Hambleton (1984) and Aiken (1980). The SMES’ perceptions of the underlying content structures of the tests emerged in the MDS solutions. All dimensions were germane to the content domains measured by the tests. Some of these dimensions were consistent with the content structure specified in the test blueprint, others were not. Correlation and regression analyses of the MDS item coordinates and item relevance ratings indicated that using both item similarity and item relevance data provided greater information of content representation than did using either approach alone. The implications of the procedure for test validity are discussed and suggestions for future research are provided. Index terms: construct validity, content validity, cluster analysis, multidimensional scaling, subject-matter experts, test construction

Internal consistency and reliability of Primary Care Assessment Tool (PCATool-Brasil) for child health services

Health strategies based on primary health care have been expanding in Brazil. An instrument applied to users, the Primary Care Assessment Tool (the PCATool), which measures the extent of primary care, has been validated in the United States. We sought to adapt the PCATool to Brazil and analyze its validation and reliability through a cross-sectional validation study of the Child PCATool.Validation included: translation, back-translation, adaptation, debriefing, content and construct validate, internal consistency, and reliability analysis. The questionnaire was applied to 468 parents or guardians of children registered with 18 primary health services in Porto Alegre, representing the services’ normal pediatric population. Using factor analysis, 8 domains were identified, with Cronbach’s α ranging from 0.74 to 0.88.Validation resulted in a 45-item scale, divided into 8 attributes (Access to First Contact, Continuity, Coordination, 3 attributes of Comprehensiveness, Community Orientation, and Family Orientation). These results showed that the PCATool-Brasil has adequate validity and reliability and could be used as a national instrument to evaluate primary health care after its application to other population settings in the country.Fundação de Amparo à Pesquisa, Estado do Rio Grande do Sul e Escola de Saúde Pública, Rio Grande do Su