Customizable and scalable automated assessment of C/C++ programming assignments

The correction of exercises in programming courses is a laborious task that has traditionally been performed in a manual way. This situation, in turn, delays the access by students to feedback that can contribute significantly to their training as future professionals. Over the years, several approaches have been proposed to automate the assessment of students' programs. Static analysis is a known technique that can partially simulate the process of manual code review performed by lecturers. As such, it is a plausible option to assess whether students' solutions meet the requirements imposed on the assignments. However, implementing a personalized analysis beyond the rules included in existing tools may be a complex task for the lecturer without a mechanism that guides the work. In this paper, we present a method to provide automated and specific feedback to immediately inform students about their mistakes in programming courses. To that end, we developed the CAC++ library, which enables constructing tailored static analysis programs for C/C++ practices. The library allows for great flexibility and personalization of verifications to adjust them to each particular task, overcoming the limitations of most of the existing assessment tools. Our approach to providing specific feedback has been evaluated for a period of three academic years in a course related to object-oriented programming. The library allowed lecturers to reduce the size of the static analysis programs developed for this course. During this period, the academic results improved and undergraduates positively valued the aid offered when undertaking the implementation of assignments.Universidad de Cádiz, Grant/Award Numbers: sol-201500054192-tra, sol-201600064680-tra; Ministerio de Ciencia, Innovación y Universidades, Grant/Award Number: RTI2018-093608-B-C33; European Regional Development Fun

Methodology for automating software systems

Applying ITS technology to the shuttle diagnostics would not require the rigor of the Petri Net representation, however it is important in providing the animated simulated portion of the interface and the demands placed on the system to support the training aspects to have a homogeneous and consistent underlying knowledge representation. By keeping the diagnostic rule base, the hardware description, the software description, user profiles, desired behavioral knowledge, and the user interface in the same notation, it is possible to reason about the all of the properties of petri nets, on any selected portion of the simulation. This reasoning provides foundation for utilization of intelligent tutoring systems technology

Formative computer based assessment in diagram based domains

This research argues that the formative assessment of student coursework in free-form, diagram-based domains can be automated using CBA techniques in a way which is both feasible and useful. Formative assessment is that form of assessment in which the objective is to assist the process of learning undertaken by the student. The primary deliverable associated with formative assessment is feedback. CBA courseware provides facilities to implement the full lifecycle of an exercise through an integrated, online system. This research demonstrates that CBA offers unique opportunities for student learning through formative assessment, including allowing students to correct their solutions over a larger number of submissions than it would be feasible to allow within the context of traditional assessment forms. The approach to research involves two main phases. The first phase involves designing and implementing an assessment course using the CourseMarker / DATsys CBA system. This system, in common with may other examples of CBA courseware, was intended primarily to conduct summative assessment. The benefits and limitations of the system are identified. The second phase identifies three extensions to the architecture which encapsulate the difference in requirements between summative assessment and formative assessment, presents a design for the extensions, documents their implementation as extensions to the CourseMarker / DATsys architecture and evaluates their contribution. The three extensions are novel extensions for free-form CBA which allow the assessment of the aesthetic layout of student diagrams, the marking of student solutions where multiple model solutions are acceptable and the prioritisation and truncation of feedback prior to its presentation to the student. Evaluation results indicate that the student learning process can be assisted through formative assessment which is automated using CBA courseware. The students learn through an iterative process in which feedback upon a submitted student coursework solution is used by the student to improve their solution, after which they may re-submit and receive further feedback

Towards a Research Agenda on Computer-Based Assessment - Challenges and Needs for European Educational Measurement

In 2006 the European Parliament and the Council of Europe have passed recommendations on key competences for lifelong learning and the use of a common reference tool to observe and promote progress in terms of the achievement of goals formulated in ¿Lisbon strategy¿ in March 2000 (revised in 2006, see http://ec.europa.eu/growthandjobs/) and its follow-up declarations. For those areas which are not already covered by existing measurements (foreign languages and learning-to-learn skills), indicators for the identification of such skills are now needed, as well as effective instruments for carrying out large-scale assessments in Europe. In this context it is hoped that electronic testing could improve the effectiveness of the needed assessments, i.e. to improve identification of skills, by reducing costs of the whole operation (financial efforts, human resources etc.). The European Commission is asked to assist Member States to define the organisational and resource implications for them of the construction and administration of tests, including looking into the possibility of adopting e-testing as the means to administer the tests. In addition to traditional testing approaches carried out in a paper-pencil mode, there are a variety of aspects needed to be taken into account when computer-based testing is deployed, such as software quality, secure delivery, if Internet-based: reliable network capacities, support, maintenance, software costs for development and test delivery, including licences. Future European surveys are going to introduce new ways of assessing student achievements. Tests can be calibrated to the specific competence level of each student and become more stimulating, going much further than it can be achieved with traditional multiple choice questions. Simulations provide better means of contextualising skills to real life situations and providing a more complete picture of the actual competence to be assessed. However, a variety of challenges require more research into the barriers posed by the use of technologies, e.g. in terms of computer, performance and security. The ¿Quality of Scientific Information¿ Action (QSI) and the Centre for Research on Lifelong Learning (CRELL) are carrying out a research project on quality criteria of Open Source skills assessment tools. 2 workshops were carried out in previous years bringing together European key experts from assessment research and practice in order to identify and discuss quality criteria relevant for carrying out large-scale assessments at a European level. This report reflects the contributions made on experiences and key challenges for European skills assessment.JRC.G.9-Econometrics and statistical support to antifrau

Formative computer based assessment in diagram based domains

This research argues that the formative assessment of student coursework in free-form, diagram-based domains can be automated using CBA techniques in a way which is both feasible and useful. Formative assessment is that form of assessment in which the objective is to assist the process of learning undertaken by the student. The primary deliverable associated with formative assessment is feedback. CBA courseware provides facilities to implement the full lifecycle of an exercise through an integrated, online system. This research demonstrates that CBA offers unique opportunities for student learning through formative assessment, including allowing students to correct their solutions over a larger number of submissions than it would be feasible to allow within the context of traditional assessment forms. The approach to research involves two main phases. The first phase involves designing and implementing an assessment course using the CourseMarker / DATsys CBA system. This system, in common with may other examples of CBA courseware, was intended primarily to conduct summative assessment. The benefits and limitations of the system are identified. The second phase identifies three extensions to the architecture which encapsulate the difference in requirements between summative assessment and formative assessment, presents a design for the extensions, documents their implementation as extensions to the CourseMarker / DATsys architecture and evaluates their contribution. The three extensions are novel extensions for free-form CBA which allow the assessment of the aesthetic layout of student diagrams, the marking of student solutions where multiple model solutions are acceptable and the prioritisation and truncation of feedback prior to its presentation to the student. Evaluation results indicate that the student learning process can be assisted through formative assessment which is automated using CBA courseware. The students learn through an iterative process in which feedback upon a submitted student coursework solution is used by the student to improve their solution, after which they may re-submit and receive further feedback

Aeronautical engineering: A continuing bibliography with indexes (supplement 202)

This bibliography lists 447 reports, articles and other documents introduced into the NASA scientific and technical information system in June 1986

The evaluation of a novel haptic machining VR-based process planning system using an original process planning usability method

This thesis provides an original piece of work and contribution to knowledge by creating a new process planning system; Haptic Aided Process Planning (HAPP). This system is based on the combination of haptics and virtual reality (VR). HAPP creates a simulative machining environment where Process plans are automatically generated from the real time logging of a user’s interaction. Further, through the application of a novel usability test methodology, a deeper study of how this approach compares to conventional process planning was undertaken. An abductive research approach was selected and an iterative and incremental development methodology chosen. Three development cycles were undertaken with evaluation studies carried out at the end of each. Each study, the pre-pilot, pilot and industrial, identified progressive refinements to both the usability of HAPP and the usability evaluation method itself. HAPP provided process planners with an environment similar to which they are already familiar. Visual images were used to represent tools and material whilst a haptic interface enabled their movement and positioning by an operator in a manner comparable to their native setting. In this way an intuitive interface was developed that allowed users to plan the machining of parts consisting of features that can be machined on a pillar drill, 21/2D axis milling machine or centre lathe. The planning activities included single or multiple set ups, fixturing and sequencing of cutting operations. The logged information was parsed and output to a process plan including route sheets, operation sheets, tool lists and costing information, in a human readable format. The system evaluation revealed that HAPP, from an expert planners perspective is perceived to be 70% more satisfying to use, 66% more efficient in completing process plans, primarily due to the reduced cognitive load, is more effective producing a higher quality output of information and is 20% more learnable than a traditional process planning approach

The Federal Conference on Intelligent Processing Equipment

Research and development projects involving intelligent processing equipment within the following U.S. agencies are addressed: Department of Agriculture, Department of Commerce, Department of Energy, Department of Defense, Environmental Protection Agency, Federal Emergency Management Agency, NASA, National Institutes of Health, and the National Science Foundation