Evidence Based Design of Heuristics: Usability and Computer Assisted Assessment

The research reported here examines the usability of Computer Assisted Assessment(CAA) and the development of domain specific heuristics. CAA is being adopted within educational institutions and the pedagogical implications are widely investigated, but little research has been conducted into the usability of CAA applications. The thesis is: severe usability problems exist in GAA applications causing unacceptable consequences, and that using an evidence based design approach GAA heuristics can be devised The thesis reports a series of evaluations that show severe usability problems do occur in three CAA applications. The process of creating domain specific heuristics is analysed, critiqued and a novel evidence based design approach for the design of domain specific heuristics is proposed. Gathering evidence from evaluations and the literature, a set of heuristics for CAA are presented. There are four main contributions to knowledge in the thesis: the heuristics; the corpus of usability problems; the Damage Index for prioritising usability problems from multiple evaluations and the evidence based design approach to synthesise heuristics. The focus of the research evolves with the first objective being to determine If severe usability problems exist that can cause users d?ffIculties and dissatisfaction with unacceptable consequences whitct using existing commercial CAA software applications? Using a survey methodology, students' report a level of satisfaction but due to low inter-group consistency surveys are judged to be ineffective at eliciting usability problems. Alternative methods are analysed and the heuristic evaluation method is judged to be suitable. A study is designed to evaluate Nielsen's heuristic set within the CAA domain and they are deemed to be ineffective based on the formula proposed by Hanson et al. (2003). Domain specific heuristics are therefore necessary and further studies are designed to build a corpus of usability problems to facilitate the evidence based design approach to synthesise a set of heuristics, in order to aggregate the corpus and prioritise the severity of the problems a Damage Index formula is devised. The work concludes with a discussion of the heuristic design methodology and potential for future work; this includes the application of the CAA heuristics and applying the heuristic design methodology to other specific domains

Assisted assignment of automotive safety requirements

ISO 26262, a functional-safety standard, uses Automotive Safety Integrity Levels (ASILs) to assign safety requirements to automotive-system elements. System designers initially assign ASILs to system-level hazards and then allocate them to elements of the refined system architecture. Through ASIL decomposition, designers can divide a function & rsquo;s safety requirements among multiple components. However, in practice, manual ASIL decomposition is difficult and produces varying results. To overcome this problem, a new tool automates ASIL allocation and decomposition. It supports the system and software engineering life cycle by enabling users to efficiently allocate safety requirements regarding systematic failures in the design of critical embedded computer systems. The tool is applicable to industries with a similar concept of safety integrity levels. © 1984-2012 IEEE

A synthesis of logic and biology in the design of dependable systems

The technologies of model-based design and dependability analysis in the design of dependable systems, including software intensive systems, have advanced in recent years. Much of this development can be attributed to the application of advances in formal logic and its application to fault forecasting and verification of systems. In parallel, work on bio-inspired technologies has shown potential for the evolutionary design of engineering systems via automated exploration of potentially large design spaces. We have not yet seen the emergence of a design paradigm that combines effectively and throughout the design lifecycle these two techniques which are schematically founded on the two pillars of formal logic and biology. Such a design paradigm would apply these techniques synergistically and systematically from the early stages of design to enable optimal refinement of new designs which can be driven effectively by dependability requirements. The paper sketches such a model-centric paradigm for the design of dependable systems that brings these technologies together to realise their combined potential benefits

A synthesis of logic and bio-inspired techniques in the design of dependable systems

Much of the development of model-based design and dependability analysis in the design of dependable systems, including software intensive systems, can be attributed to the application of advances in formal logic and its application to fault forecasting and verification of systems. In parallel, work on bio-inspired technologies has shown potential for the evolutionary design of engineering systems via automated exploration of potentially large design spaces. We have not yet seen the emergence of a design paradigm that effectively combines these two techniques, schematically founded on the two pillars of formal logic and biology, from the early stages of, and throughout, the design lifecycle. Such a design paradigm would apply these techniques synergistically and systematically to enable optimal refinement of new designs which can be driven effectively by dependability requirements. The paper sketches such a model-centric paradigm for the design of dependable systems, presented in the scope of the HiP-HOPS tool and technique, that brings these technologies together to realise their combined potential benefits. The paper begins by identifying current challenges in model-based safety assessment and then overviews the use of meta-heuristics at various stages of the design lifecycle covering topics that span from allocation of dependability requirements, through dependability analysis, to multi-objective optimisation of system architectures and maintenance schedules

Theoretical models of the role of visualisation in learning formal reasoning

Although there is empirical evidence that visualisation tools can help students to learn formal subjects such as logic, and although particular strategies and conceptual difficulties have been identified, it has so far proved difficult to provide a general model of learning in this context that accounts for these findings in a systematic way. In this paper, four attempts at explaining the relative difficulty of formal concepts and the role of visualisation in this learning process are presented. These explanations draw on several existing theories, including Vygotsky's Zone of Proximal Development, Green's Cognitive Dimensions, the Popper-Campbell model of conjectural learning, and cognitive complexity. The paper concludes with a comparison of the utility and applicability of the different models. It is also accompanied by a reflexive commentary[0] (linked to this paper as a hypertext) that examines the ways in which theory has been used within these arguments, and which attempts to relate these uses to the wider context of learning technology research

Advances in Teaching & Learning Day Abstracts 2004

Proceedings of the Advances in Teaching & Learning Day Regional Conference held at The University of Texas Health Science Center at Houston in 2004

Computers in writing instruction

For computers to be useful in writing instruction, innovations should be valuable for students and feasible for teachers to implement. Research findings yield contradictory results in measuring the effects of different uses of computers in writing, in part because of the methodological complexity of such measurements. Yet the computer seems to be a promising tool in several new, theoretically based approaches to writing instruction. Research of these kinds of computer applications should continue, paying attention to context variables that influence the implementation process importantly

Fuzzy Logic in Clinical Practice Decision Support Systems

Computerized clinical guidelines can provide significant benefits to health outcomes and costs, however, their effective implementation presents significant problems. Vagueness and ambiguity inherent in natural (textual) clinical guidelines is not readily amenable to formulating automated alerts or advice. Fuzzy logic allows us to formalize the treatment of vagueness in a decision support architecture. This paper discusses sources of fuzziness in clinical practice guidelines. We consider how fuzzy logic can be applied and give a set of heuristics for the clinical guideline knowledge engineer for addressing uncertainty in practice guidelines. We describe the specific applicability of fuzzy logic to the decision support behavior of Care Plan On-Line, an intranet-based chronic care planning system for General Practitioners

Web assisted teaching: an undergraduate experience

The emergence of the Internet has created a number of claims as to the future of education and the possibility of dramatically changing the way in which education is delivered. Much of the attention has focussed on the adoption of teaching methods that are solely web-based. We set out to incorporate web-based teaching as support for more traditional teaching methods to improve the learning outcomes for students. This first step into web-based teaching was developed to harness the benefits of web-based teaching tools without supplanting traditional teaching methods. The aim of this paper is to report our experience with web-assisted teaching in two undergraduate courses, Accounting Information Systems and Management Accounting Services, during 2000. The paper evaluates the approach taken and proposes a tentative framework for developing future web-assisted teaching applications. We believe that web-assisted and web-based teaching are inevitable outcomes of the telecommunications and computer revolution and that academics cannot afford to become isolated from the on-line world. A considered approach is needed to ensure the integration of web-based features into the overall structure of a course. The components of the course material and the learning experiences students are exposed to need to be structured and delivered in a way that ensures they support student learning rather than replacing one form of learning with another. Therefore a careful consideration of the structure, content, level of detail and time of delivery needs to be integrated to create a course structure that provides a range of student learning experiences that are complimentary rather than competing. The feedback was positive from both extramural (distance) and internal students, demonstrating to us that web sites can be used as an effective teaching tool in support of more traditional teaching methods as well as a tool for distance education. The ability to harness the positives of the web in conjunction with more traditional teaching modes is one that should not be overlooked in the move to adopt web based instruction methods. Web-based teaching need not be seen as an all or nothing divide but can be used as a useful way of improving the range and type of learning experiences open to students. The Web challenges traditional methods and thinking but it also provides tools to develop innovative solutions to both distance and on campus learning. Further research is needed to determine how we can best meet the needs of our students while maintaining high quality learning outcomes