Psychometrics in Practice at RCEC

A broad range of topics is dealt with in this volume: from combining the psychometric generalizability and item response theories to the ideas for an integrated formative use of data-driven decision making, assessment for learning and diagnostic testing. A number of chapters pay attention to computerized (adaptive) and classification testing. Other chapters treat the quality of testing in a general sense, but for topics like maintaining standards or the testing of writing ability, the quality of testing is dealt with more specifically.\ud All authors are connected to RCEC as researchers. They present one of their current research topics and provide some insight into the focus of RCEC. The selection of the topics and the editing intends that the book should be of special interest to educational researchers, psychometricians and practitioners in educational assessment

Performance Measures Using Electronic Health Records: Five Case Studies

Presents the experiences of five provider organizations in developing, testing, and implementing four types of electronic quality-of-care indicators based on EHR data. Discusses challenges, and compares results with those from traditional indicators

A flexible architecture for privacy-aware trust management

In service-oriented systems a constellation of services cooperate, sharing potentially sensitive information and responsibilities. Cooperation is only possible if the different participants trust each other. As trust may depend on many different factors, in a flexible framework for Trust Management (TM) trust must be computed by combining different types of information. In this paper we describe the TAS3 TM framework which integrates independent TM systems into a single trust decision point. The TM framework supports intricate combinations whilst still remaining easily extensible. It also provides a unified trust evaluation interface to the (authorization framework of the) services. We demonstrate the flexibility of the approach by integrating three distinct TM paradigms: reputation-based TM, credential-based TM, and Key Performance Indicator TM. Finally, we discuss privacy concerns in TM systems and the directions to be taken for the definition of a privacy-friendly TM architecture.\u

Analyzing collaborative learning processes automatically

In this article we describe the emerging area of text classification research focused on the problem of collaborative learning process analysis both from a broad perspective and more specifically in terms of a publicly available tool set called TagHelper tools. Analyzing the variety of pedagogically valuable facets of learners’ interactions is a time consuming and effortful process. Improving automated analyses of such highly valued processes of collaborative learning by adapting and applying recent text classification technologies would make it a less arduous task to obtain insights from corpus data. This endeavor also holds the potential for enabling substantially improved on-line instruction both by providing teachers and facilitators with reports about the groups they are moderating and by triggering context sensitive collaborative learning support on an as-needed basis. In this article, we report on an interdisciplinary research project, which has been investigating the effectiveness of applying text classification technology to a large CSCL corpus that has been analyzed by human coders using a theory-based multidimensional coding scheme. We report promising results and include an in-depth discussion of important issues such as reliability, validity, and efficiency that should be considered when deciding on the appropriateness of adopting a new technology such as TagHelper tools. One major technical contribution of this work is a demonstration that an important piece of the work towards making text classification technology effective for this purpose is designing and building linguistic pattern detectors, otherwise known as features, that can be extracted reliably from texts and that have high predictive power for the categories of discourse actions that the CSCL community is interested in

Sustainable urban development in practice:the SAVE concept

The need for sustainable development of the urban environment presents the research community with a number of challenges and opportunities. A considerable volume of research has been undertaken into the constituent parts of this complex problem and a number of tool kits and methodologies have been developed to enable and encourage the application of specific aspects of research in practice. However, there is limited evidence of the holistic integration of the body of knowledge arising from the research within real-life decision-making practices. In this paper we present an overview of the existing body of knowledge relating to sustainable development of the urban environment and propose a generic framework for its integration within current practices. This framework recognises the need to: understand social, economic, and environmental issues; understand the decision-making processes; provide a means of measurement, assessment, or valuation of the issues; provide analytical methods for the comparative assessment of complex data to enable an evaluation of strategies and design options and to communicate effectively throughout the process with a wide range of stakeholders. The components of a novel sustainability assessment, visualisation and enhancement (SAVE) framework, developed by the authors to ‘operationalise’ the body of knowledge are presented and justified. These include: decision-mapping methods to identify points of intervention; indicator identification and measurement approaches; appropriate mathematical and analytical tools and an interactive simulation and visualisation platform which integrates and communicates complex multivariate information to diverse stakeholder groups. We report on the application of the SAVE framework to a major urban development project and reflect on its current and potential impact on the development. Conclusions are also drawn about its general applicability