Understanding Cognition Across Modalities for the Assessment of Digital Resources

Drawing from the theories of the cognitive process, this paper explores the transmission, retention and transformation of information across oral, written, and digital modes of communication and how these concepts can be used to examine the assessment of digital resource tools. The exploration of interactions across modes of communication is used to gain an understanding of the interaction between the student, digital resource and teacher. Cognitive theory is considered as a basis for the assessment of digital resource tools. Lastly, principles for the assessment of digital resource tools are presented along with how assessment can be incorporated in the educational practice to enhance learning in higher education

Evaluating the Validity of Technology-Enhanced Educational Assessment Items and Tasks: An Empirical Approach to Studying Item Features and Scoring Rubrics.

With the advent of the newly developed Common Core State Standards and the Next Generation Science Standards, innovative assessments, including technology-enhanced items and tasks, will be needed to meet the challenges of developing valid and reliable assessments in a world of computer-based testing. In a recent critique of the next generation assessments in math (i.e., Smarter Balanced), Rasmussen (2015) observed that many aspects of the technology “enhancements” can be expected to do more harm than good as the computer interfaces may introduce construct irrelevant variance. This paper focused on issues surrounding the design of TEIs and how cognitive load theory (Miller, 1956) is a promising framework that can be applied to computer-based item design to mitigate the effects of computer interface usability. Two studies were conducted. In the first study I used multi-level modeling to assess the effect of item characteristics on examinees’ relative performance. I hypothesized that item level characteristics, namely response format, would significantly contribute to the amount of variance explained by item characteristics over and above student characteristics. In study two, I used two exemplar items to show how data concerning examinees’ actions—produced through latent class analyses—can be used as evidence in validity investigations. Results from study 1 suggested that item type does not explain the variation in student scores over and above examinee characteristics. Results from study two suggested that LCA is a useful tool for diagnosing potential issues in the design of items and the design of their scoring rubrics. Evidence provided from both studies illuminates the immediate need to further research computer-based items that are beginning to be used widely in high stakes, large-scale assessments. In an effort to move away from traditional multiple choice items and toward more authentic measurement by incorporating technology based item features, we may be affecting how examinees respond to the item due to inadvertent increases in cognitive load. Future research involving experimental manipulation is necessary for understanding how item characteristics impact how examinees responses

Revisiting the design intent concept in the context of mechanical CAD education

[EN] Design intent is generally understood simply as a CAD model¿s anticipated behavior when altered. However, this representation provides a simplified view of the model¿s construction and purpose, which may hinder its general understanding and future reusability. Our vision is that design intent communication may be improved by recognizing the multifaceted nature of design intent, and by instructing users to convey each facet of design intent through the better-fitted CAD resource. This paper reviews the current understanding of design intent and its relationship to design rationale and builds on the idea that communication of design intent conveyed via CAD models can be satisfied at three levels provided that specialized instruction is used to instruct users in selection of the most suitable level for each intent.Otey, J.; Company, P.; Contero, M.; Camba, J. (2018). Revisiting the design intent concept in the context of mechanical CAD education. Computer-Aided Design and Applications. 15(1):47-60. https://doi.org/10.1080/16864360.2017.1353733S476015

Assessment for complex learning resources: Development and validation of an integrated model

Today's e-learning systems meet the challenge to provide interactive, personalized environments that support self-regulated learning as well as social collaboration and simulation. At the same time assessment procedures have to be adapted to the new learning environments by moving from isolated summative assessments to integrated assessment forms. Therefore, learning experiences enriched with complex didactic resources - such as virtualized collaborations and serious games - have emerged. In this extension of [1] an integrated model for e-assessment (IMA) is outlined, which incorporates complex learning resources and assessment forms as main components for the development of an enriched learning experience. For a validation the IMA was presented to a group of experts from the fields of cognitive science, pedagogy, and e-learning. The findings from the validation lead to several refinements of the model, which mainly concern the component forms of assessment and the integration of social aspects. Both aspects are accounted for in the revised model, the former by providing a detailed sub-model for assessment forms

Purposive Teaching Styles for Transdisciplinary AEC Education: A Diagnostic Learning Styles Questionnaire

With the progressive globalisation trend within the Architecture, Engineering, and Construction (AEC) industry, transdisciplinary education and training is widely acknowledged as being one of the key factors for leveraging AEC organisational success. Conventional education and training delivery approaches within AEC therefore need a paradigm shift in order to be able to address the emerging challenges of global practices. This study focuses on the use of Personalised Learning Environments (PLEs) to specifically address learners’ needs and preferences (learning styles) within managed Virtual Learning Environments (VLEs). This research posits that learners can learn better (and be more readily engaged in managed learning environments) with a bespoke PLE, in which the deployment of teaching and learning material is augmented towards their individual needs. In this respect, there is an exigent need for the Higher Educational Institutions (HEIs) to envelop these new approaches into their organisational learning strategy. However, part of this process requires decision-makers to fully understand the core nuances and interdependencies of functions and processes within the organisation, along with Critical Success Factors (CSFs) and barriers. This paper presents findings from the development of a holistic conceptual Diagnostic Learning Styles Questionnaire (DLSQ) Framework, comprised of six interrelated dependencies (i.e. Business Strategy, Pedagogy, Process, Resources, Systems Development, and Evaluation). These dependencies influence pedagogical effectiveness. These finding contribute additional understanding to the intrinsic nature of pedagogy in leveraging transdisciplinary AEC training within organisations (to improve learner effectiveness). This framework can help organisations augment and align their strategic priorities to learner-specific traits

A Study into the assessment and reporting of employability skills of senior secondary students

In order to investigate the most effective ways of assessing and reporting on the employability skills of senior secondary students, the Commonwealth Department of Education, Science and Training (DEST) commissioned this report. The main activity was to evaluate options for assessing and reporting on eight employability skills against five criteria—validity, reliability, objectivity, feasibility, and usability—and to recommend a preferred approach. This work was undertaken by the Australian Council for Educational Research (ACER) in the period July 2007 to January 2008, during which time the (new) Department of Education, Employment and Workplace Relations (DEEWR) became responsible for the initiative. The starting point for this work was to come to terms with the eight employability skills and their respective facets from the Employability Skills Framework (ACCI & BCA, 2002). The employability skills are: Communication; Initiative & Enterprise; Learning; Planning & Organising; Problem Solving; Self-management; Teamwork; and Technology. Facets are elements of the skill that employers have identified as important, with the specific mix and priority of facets being job-dependent. One of the facets of Communication, for example, is ‘Reading independently’. As an adjunct activity to this study, the University of Western Sydney conducted a survey of the current level of employer satisfaction with the eight identified employability skills and how employers assess them (Costley, Power, Watson, Steele, & Sproats, 2007). Consultations were undertaken with employers, and representatives of parent organisations, school systems including teachers and leaders, and Australia’s three peak business organisations, ACCI, BCA, and the Australian Industry Group (AIG). An advisory group set up by DEST had the same representation. In addition, expert input was sought on detailed matters of assessment and reporting

Towards an Integrated Assessment Framework: Using Activity Theory to Understand, Evaluate, and Enhance Programmatic Assessment in Integrated Content and Language Learning

This article uses activity theory to analyse two different portfolio approaches as tools for programmatic assessment of Integrated Content and Language (ICL) programs. The two approaches include a) a model in which students construct portfolios by selecting artifacts from a range of different contexts and provide reflective commentary, and b) a model in which the portfolio consists of major textual artifacts produced across a design project, with no reflective component. Activity theory provides a tool to explore what these models can offer in terms of an assessment of the integration of content and language in disciplinary contexts, where texts serve to mediate the ongoing work of a discipline. By highlighting the work that texts do in context as well as the access to student meta-knowledge afforded by each portfolio, activity theory provides a means to understand the strengths and limitations of both models. Perhaps most importantly, it points to the need for portfolios to include well-designed reflections that can support both student learning and effective programmatic assessment

Assessing and supporting argumentation with online rubrics

published_or_final_versio