The Automatic Assessment of Multiple Artefacts: An Investigation into Design Diagrams and Their Implementations

As the Higher Education sector has moved towards student-centred learning so too has the growth in electronic support for learning. E-assessment has been a part of this growth as increasingly assessment and its feedback is seen as an integral part of the students’ learning process. Mature e-assessment systems exist, particularly where answers to questions are restricted to a prescribed list of alternatives. However, for free response artefacts, where there is a limited restriction placed on answers to questions, automated assessment systems are embryonic. This dissertation presents an investigation into the automated assessment of free response artefacts. Design diagrams and their accompanying source code implementations are examples of free response artefacts. A case study is developed that investigates how to automatically generate formative feedback for a design diagram by utilizing its accompanying implementation. The dissertation presents a two-staged solution, initially analysing the design diagram in isolation before comparing it with the implementation. A framework for this approach has been developed and tested using a tool applied to coursework submitted by undergraduate computer science students. The tool was evaluated by comparing the formative feedback comments generated by the tool with those produced by a team of computer science educators. Evaluation was undertaken via two Likert questionnaires, one completed by students and one completed by a team of computer scientists. The results presented are favourable, with the majority of comments produced by the tool being seen to be as least as good as those generated by the computer science educators

On the evaluation of reference models for software engineering practice.

This paper argues that conceptual models and more specifically reference models play a key role in the specification and design of information systems. However, an effective evaluation strategy of such models is a relatively immature field. The paper presents the key challenges for this evaluation activity and articulates an approach for understanding how to evaluate models based on the information and cognitive theories of structuralism and conversation theory. An example of a reference model developed for the Higher Education domain is used as a case study to illustrate how the approach may be applied

Representing Interactional and External Environmental Semantics using Unified Modelling Language (UML)

Object-oriented methodology is widely used in the information system development field. Nonetheless, recent research studies have discovered that the modelling grammars that use object-oriented methodology lack necessary constructs to represent certain real-world semantics. Therefore, the use of such grammars with their shortcomings can produce defective conceptual models, thereby producing defective information systems. Evermann and Wand (2005, 2009) studied this issue and proposed a set of rules for object-oriented grammatical constructs to represent static and behaviour semantics of a real-world phenomenon. This paper extends their work by proposing object-oriented grammatical rules for the interactional and external environmental semantics of a real-world phenomenon. This representation is exemplified using an object-oriented modelling grammar namely Unified Modelling Language (UML). Subsequently, the set of new rules has been validated using a case study. This extended UML facilitates seamless integration between the conceptual model and its system model

Highlighting model elements to improve OCL comprehension

Models, metamodels, and model transformations play a central role in Model-Driven Development (MDD). Object Constraint Language (OCL) was initially proposed as part of the Unified Modeling Language (UML) standard to add the precision and validation capabilities lacking in its diagrams, and to express well-formedness rules in its metamodel. OCL has several other applications, such as defining design metrics, code-generation templates, or validation rules for model transformations, required in MDD. Learning OCL as part of a UML course at the university would seem natural but is still the exception rather than the rule. We believe that this is mainly due to a widespread perception that OCL is hard to learn, as gleaned from claims made in the literature. Based on data gathered over the past school years from numerous undergraduate students of di↵erent Software Engineering courses, we analyzed how learning design by contract clauses with UML+OCL compares with several other Software Engineering Body Of Knowledge (SWEBOK) topics. The outcome of the learning process was collected in a rigorous setup, supported by an e-learning platform. We performed inferential statistics on that data to support our conclusions and identify the relevant explanatory variables for students’ success/failure. The obtained findings lead us to extend an existing OCL tool with two novel features: one is aimed at OCL apprentices and goes straight to the heart of the matter by allowing to visualize how OCL expressions traverse UML class diagrams; the other is intended for researchers and allows to compute OCL complexity metrics, making it possible to replicate a research study like the one we are presenting.Modelos, metamodelos e transformações de modelo desempenham um papel central em MDD. OCL foi inicialmente proposta como parte da UML para adicionar os recursos de precisão e validação que faltavam nestes diagramas, e também para expressar regras de boa formação no metamodelo. OCL possui outras aplicações, tais como definir métricas de desenho, modelos de geração de código ou regras de validação para transformações de modelo, exigidas em MDD. Aprender OCL como parte de um curso de UML na universidade parecia portanto natural, não sendo no entanto o que se verifica. Acreditamos que isso se deva a uma percepção generalizada de que OCL é difícil de aprender, tendo em conta afirmações feitas na literatura. Com base em dados recolhidos em anos letivos anteriores de vários alunos de licenciatura de diferentes cursos de Engenharia de Software, analisámos como a aprendizagem por cláusulas contratuais de UML + OCL se compara a outros tópicos do SWEBOK. O resultado do processo de aprendizagem foi recolhido de forma rigorosa, apoiado por uma plataforma de e-learning. Realizámos estatísticas inferenciais sobre os dados para apoiar as nossas conclusões, de forma a identificar as variáveis explicativas relevantes para o sucesso / fracasso dos alunos. As conclusões obtidas levaram-nos a estender uma ferramenta OCL com duas novas funcionalidades: a primeira é voltada para os estudantes de OCL e permite visualizar como as expressões percorrem um diagrama de classes UML; a segunda é voltada para investigadores e permite calcular métricas de complexidade OCL, habilitando a réplica de um estudo semelhante ao apresentado

Models for Learning (Mod4L) Final Report: Representing Learning Designs

The Mod4L Models of Practice project is part of the JISC-funded Design for Learning Programme. It ran from 1 May – 31 December 2006. The philosophy underlying the project was that a general split is evident in the e-learning community between development of e-learning tools, services and standards, and research into how teachers can use these most effectively, and is impeding uptake of new tools and methods by teachers. To help overcome this barrier and bridge the gap, a need is felt for practitioner-focused resources which describe a range of learning designs and offer guidance on how these may be chosen and applied, how they can support effective practice in design for learning, and how they can support the development of effective tools, standards and systems with a learning design capability (see, for example, Griffiths and Blat 2005, JISC 2006). Practice models, it was suggested, were such a resource. The aim of the project was to: develop a range of practice models that could be used by practitioners in real life contexts and have a high impact on improving teaching and learning practice. We worked with two definitions of practice models. Practice models are: 1. generic approaches to the structuring and orchestration of learning activities. They express elements of pedagogic principle and allow practitioners to make informed choices (JISC 2006) However, however effective a learning design may be, it can only be shared with others through a representation. The issue of representation of learning designs is, then, central to the concept of sharing and reuse at the heart of JISC’s Design for Learning programme. Thus practice models should be both representations of effective practice, and effective representations of practice. Hence we arrived at the project working definition of practice models as: 2. Common, but decontextualised, learning designs that are represented in a way that is usable by practitioners (teachers, managers, etc).(Mod4L working definition, Falconer & Littlejohn 2006). A learning design is defined as the outcome of the process of designing, planning and orchestrating learning activities as part of a learning session or programme (JISC 2006). Practice models have many potential uses: they describe a range of learning designs that are found to be effective, and offer guidance on their use; they support sharing, reuse and adaptation of learning designs by teachers, and also the development of tools, standards and systems for planning, editing and running the designs. The project took a practitioner-centred approach, working in close collaboration with a focus group of 12 teachers recruited across a range of disciplines and from both FE and HE. Focus group members are listed in Appendix 1. Information was gathered from the focus group through two face to face workshops, and through their contributions to discussions on the project wiki. This was supplemented by an activity at a JISC pedagogy experts meeting in October 2006, and a part workshop at ALT-C in September 2006. The project interim report of August 2006 contained the outcomes of the first workshop (Falconer and Littlejohn, 2006). The current report refines the discussion of issues of representing learning designs for sharing and reuse evidenced in the interim report and highlights problems with the concept of practice models (section 2), characterises the requirements teachers have of effective representations (section 3), evaluates a number of types of representation against these requirements (section 4), explores the more technically focused role of sequencing representations and controlled vocabularies (sections 5 & 6), documents some generic learning designs (section 8.2) and suggests ways forward for bridging the gap between teachers and developers (section 2.6). All quotations are taken from the Mod4L wiki unless otherwise stated

An online corpus of UML Design Models : construction and empirical studies

We address two problems in Software Engineering. The first problem is how to assess the severity of software defects? The second problem we address is that of studying software designs. Automated support for assessing the severity of software defects helps human developers to perform this task more efficiently and more accurately. We present (MAPDESO) for assessing the severity of software defects based on IEEE Standard Classification for Software Anomalies. The novelty of the approach lies in its use of uses ontologies and ontology-based reasoning which links defects to system level quality properties. One of the main reasons that makes studying of software designs challenging is the lack of their availability. We decided to collect software designs represented by UML models stored in image formats and use image processing techniques to convert them to models. We present the 'UML Repository' which contains UML diagrams (in image and XMI format) and design metrics. We conducted a series of empirical studies using the UML Repository. These empirical studies are a drop in the ocean empirical studies that can be conducted using the repository. Yet these studies show the versatility of useful studies that can be based on this novel repository of UML designs.Erasmus Mundus program (JOSYLEEN)Algorithms and the Foundations of Software technolog