Towards Computerized Adaptive Assessment Based on Structured Tasks

Tvarožek, J., Kravčík, M., & Bieliková, M. (2008). Towards Computerized Adaptive Assessment Based on Structured Tasks. In W. Nejdl et al. (Eds.), Adaptive Hypermedia and Adaptive Web-Based Systems (pp. 224-234). Springer Berlin / Heidelberg.In an attempt to support traditional classroom assessment processes with fully computerized methods, we have developed a method for adaptive assessment suitable for well structured domains with high emphasis on problem solving and capable of robust continuous assessment, potentially encouraging student’s achievements, reflective thinking, and creativity. The method selects problems according to the student’s demonstrated ability, structured task description schemes allow for a detailed analysis of student’s errors, and on-demand generation of task instances facilitates independent student work. We evaluated the proposed method using a software system we had developed in the domain of middle school mathematics.This work was partially supported by the Cultural and Educational Grant Agency of the Slovak Republic, grant No. KEGA 3/5187/07 and the TENCompetence Integrated Project that is funded by the European Commission’s 6th Framework Programme, priority IST/Technology Enhanced Learning. [http://www.tencompetence.org

The use of feedback in web-based instruction : achievement, feedback study time, and efficiency

The purpose of this research study is to compare the effects of an instructional treatment that presented adaptive feedback based on students’ perceptions about their answer correctness with a nonadaptive treatment on student performance, feedback study time, and lesson efficiency in a computer-based environment. Because of advances in technology, the instruction was delivered in a Web-based environment. Two versions of the Web-based lesson were designed. The lesson consisted of a pretest, a tutorial, and a posttest. The pre- and posttest were similar in that they tested students’ ability to classify defined concepts, a higher cognitive task. The tutorial presented instructional text with inserted verbal information questions. Undergraduate volunteers enrolled in entry level education courses in the Watson School of Education were randomly assigned to one of the two treatment groups. One group received varied feedback information based upon the combined assessment of response correctness and the student’s response certitude. The other group received feedback information that did not vary. Results indicate that the effects of adaptive feedback were not significantly different from the effects of nonadaptive feedback on concept learning. In the adaptive group, high certitude wrongs, low certitude corrects, and low certitude wrong responses resulted in higher feedback study time than high certitude correct responses. High certitude significantly correlated with fine discrimination errors in concept learning. In terms of feedback efficiency, adaptive feedback was significantly more efficient than nonadaptive feedback; however, for overall lesson efficiency, there were no significant differences between the two treatment groups. These results are discussed in terms of cost-benefit implications for the design of effective Web-based instruction. Implications for future research are discussed in reference to results of this study and past research

Feedback type as a moderator of the relationship between achievement goals and feedback reactions

The aim of the current study is to shed new light on the inconsistent relationship between performance-approach (PAp) goals and feedback reactions by examining feedback type as a moderator. Results of a field experiment (N = 939) using a web-based work simulation task showed that the effect of achievement-approach goals was moderated by feedback type. Relative to individuals pursuing mastery-approach goals, individuals pursuing PAp goals responded more negatively to comparative feedback but not to task-referenced feedback. In line with the hypothesized mediated moderation model, the interaction between achievement goals and feedback type also indirectly affected task performance through feedback reactions. Providing employees with feedback is a key psychological principle used in a wide range of human resource and performance management instruments (e.g., developmental assessment centres, multi-source/360 degrees feedback, training, selection, performance appraisal, management education, computer-adaptive testing, and coaching). The current study suggests that organizations need to strike a balance between encouraging learning and encouraging performance, as too much emphasis on comparative performance (both in goal inducement and in feedback style) may be detrimental to employees' reactions and rate of performance improvement

Progressive assessment of student engagement with web-based guided learning

Purpose – The purpose of this research is to investigate student engagement in guided web-based learning systems.It looks into students' engagement and their behavioral patterns in two types of guided learning systems (i.e. a fully- and a partially-guided).The research also aims to demonstrate how the engagement evolves from the beginning towards the end of the interactions; which enables analysis to be performed on the quality of engagement. Design/methodology/approach – An experimental study was conducted on 41 students from a public university in Malaysia using two web-based systems as the main learning tools.The students' engagement data were captured three times during the interactions and once at the end of the experimental study using student self-report.Findings – The main outcome of this study suggests that student engagement was changing over time either in positive or negative patterns.The directions of change in both types of guided learning were mainly influenced by the students' background of knowledge.Practical implications – This study demonstrates that student engagement is dynamic. Therefore, progressive assessment is a practical approach to obtain the engagement data which can be used to regulate and improve student engagement in web-based systems.As a result, an adaptive and intelligent web-based learning environment can be created. Originality/value – This research proposes a new approach to improve students' engagement in web-based instruction, that is, through a progressive assessment of their current experience

Remote interpreter API model for supporting computer programming adaptive learning

Despite the adoption of Learning Management Systems (LMS) has been continuously growing in the last decade few, if any, scholars addressed the lack of the interactivity in online learning systems. The requirement for an interactive learning model has been increasingly prominent as content providers realize the importance of personalization of content suits to learners’ learning progress. This paper demonstrates the adoption of adaptive learning into existing LMS engine to overcome the limitation of either systems plugins or server specification pertinent to students’ psychomotor abilities in computer programming learning. In this regard,  the psychomotor is understood as the ability of students to write the program code as correctly as expected. In this study, a web API model that works to run an interpreter based-program code remotely was developed. The web API model can be utilized by LMS so it becomes the solution to the problem. The structure of the web API model has been adapted to the needs of the learning assessment. The implementation of the developed API web model is done in Python and PHP programming languages. The performance test was done by submitting 10 to 100 program codes simultaneously indicated no significant difference to the required resources (CPU usage and memory usage) to run the program code. Furthermore, for response time, the average time needed to run Python and PHP program code is also no significant difference. The average of CPU usage required by the web API to run a Python program code is 0.2058% with 0.5973 seconds as a response time. Meanwhile, to run the PHP program code, the average CPU usage required is 0.8074% with 0.3110 seconds response time. It can be concluded that the web API performance does not overburden the server

Open PhD Workshop on Technology-Enhanced Learning and Semantics, Software and Services

The 7FP project SISTER focuses, especially, on strengthening the PhD and PostDoc level of education and training of researchers, and thus attracting more young scientists to the research profession and retaining them. The project SISTER is structured around two ICT strategic research areas - Software and services, and Intelligent Content and Semantics. The research workshops and seminars will support the research in the particular area through brainstorming sessions, discussions and strategic planning. Some of them will be of benefit to the PhD students and Post Docs and the advancement in their careers, while others will be devoted to further research collaboration in selected EU research programmes. The main research areas addressed are: Creation of digital libraries with intelligent content. Semantic annotation of digital content - Creation of ontologies for the digital content in the libraries. Semantic annotation of the learning materials in the repositories. The created ontologies and their semantic annotation will allow searching materials using semantic web techniques. Development of adaptive intelligent learning systems based on intelligent ontologies and digital learning materials. New innovative pedagogical approaches, assessment models and organisational models for lifelong competence development. Software for the effective support of users who create, store, use and exchange knowledge resources, learning activities, units of learning and competence development programmes within a learning network. Models and tools for competence development into a common, easy to use infrastructure. Training programs to learn users how to work with the infrastructure, and to train instructors and companies (specifically SMEs) to deliver services using the infrastructure. Responsive environments for technology-enhanced learning higher education and business organisations "that motivate, engage and inspire learners, and which can be embedded in the business processes and human resources management systems of organisations". Special attention will be given to using the research outcomes related to Intelligent Content and Semantics and Digital Libraries for building intelligent Adaptive and intuitive learning systems and Web 2.0 oriented applications. Development of a semantics-based reference frameworks for the conceptualisation of learning content, learning objectives, and teaching strategies, and the implementation of pedagogically-driven and semantically-enhanced adaptive learning systems. This will lead to consolidating existing theoretical and technological frameworks for explicitly modelling educational content, teaching strategies, and learner characteristics, and integrating them under a common semantic model.Open PhD Workshop on Technology-Enhanced Learning and Semantics, Software and Services in conjunction with the 13th International Conference on Artificial Intelligence: Methodology, Systems, Applications - AI@Work (AIMSA 2008) 04-06 September, Varna, Bulgari

Addictive links: The motivational value of adaptive link annotation

Adaptive link annotation is a popular adaptive navigation support technology. Empirical studies of adaptive annotation in the educational context have demonstrated that it can help students to acquire knowledge faster, improve learning outcomes, reduce navigational overhead, and encourage non-sequential navigation. In this paper, we present our exploration of a lesser known effect of adaptive annotation, its ability to significantly increase students' motivation to work with non-mandatory educational content. We explored this effect and confirmed its significance in the context of two different adaptive hypermedia systems. The paper presents and discusses the results of our work

Utilising ontology-based modelling for learning content management

Learning content management needs to support a variety of open, multi-format Web-based software applications. We propose multidimensional, model-based semantic annotation as a way to support the management of access to and change of learning content. We introduce an information architecture model as the central contribution that supports multi-layered learning content structures. We discuss interactive query access, but also change management for multi-layered learning content management. An ontology-enhanced traceability approach is the solution