Measuring and comparing the reliability of the structured walkthrough evaluation method with novices and experts

Effective evaluation of websites for accessibility remains problematic. Automated evaluation tools still require a significant manual element. There is also a significant expertise and evaluator effect. The Structured Walkthrough method is the translation of a manual, expert accessibility evaluation process adapted for use by novices. The method is embedded in the Accessibility Evaluation Assistant (AEA), a web accessibility knowledge management tool. Previous trials examined the pedagogical potential of the tool when incorporated into an undergraduate computing curriculum. The results of the evaluations carried out by novices yielded promising, consistent levels of validity and reliability. This paper presents the results of an empirical study that compares the reliability of accessibility evaluations produced by two groups (novices and experts). The main results of this study indicate that overall reliability of expert evaluations was 76% compared to 65% for evaluations produced by novices. The potential of the Structured Walkthrough method as a useful and viable tool for expert evaluators is also examined. Copyright 2014 ACM

An examination of automatic video retrieval technology on access to the contents of an historical video archive

Purpose – This paper aims to provide an initial understanding of the constraints that historical video collections pose to video retrieval technology and the potential that online access offers to both archive and users. Design/methodology/approach – A small and unique collection of videos on customs and folklore was used as a case study. Multiple methods were employed to investigate the effectiveness of technology and the modality of user access. Automatic keyframe extraction was tested on the visual content while the audio stream was used for automatic classification of speech and music clips. The user access (search vs browse) was assessed in a controlled user evaluation. A focus group and a survey provided insight on the actual use of the analogue archive. The results of these multiple studies were then compared and integrated (triangulation). Findings – The amateur material challenged automatic techniques for video and audio indexing, thus suggesting that the technology must be tested against the material before deciding on a digitisation strategy. Two user interaction modalities, browsing vs searching, were tested in a user evaluation. Results show users preferred searching, but browsing becomes essential when the search engine fails in matching query and indexed words. Browsing was also valued for serendipitous discovery; however the organisation of the archive was judged cryptic and therefore of limited use. This indicates that the categorisation of an online archive should be thought of in terms of users who might not understand the current classification. The focus group and the survey showed clearly the advantage of online access even when the quality of the video surrogate is poor. The evidence gathered suggests that the creation of a digital version of a video archive requires a rethinking of the collection in terms of the new medium: a new archive should be specially designed to exploit the potential that the digital medium offers. Similarly, users' needs have to be considered before designing the digital library interface, as needs are likely to be different from those imagined. Originality/value – This paper is the first attempt to understand the advantages offered and limitations held by video retrieval technology for small video archives like those often found in special collections

Methods and Tools for Objective Assessment of Psychomotor Skills in Laparoscopic Surgery

Training and assessment paradigms for laparoscopic surgical skills are evolving from traditional mentor–trainee tutorship towards structured, more objective and safer programs. Accreditation of surgeons requires reaching a consensus on metrics and tasks used to assess surgeons’ psychomotor skills. Ongoing development of tracking systems and software solutions has allowed for the expansion of novel training and assessment means in laparoscopy. The current challenge is to adapt and include these systems within training programs, and to exploit their possibilities for evaluation purposes. This paper describes the state of the art in research on measuring and assessing psychomotor laparoscopic skills. It gives an overview on tracking systems as well as on metrics and advanced statistical and machine learning techniques employed for evaluation purposes. The later ones have a potential to be used as an aid in deciding on the surgical competence level, which is an important aspect when accreditation of the surgeons in particular, and patient safety in general, are considered. The prospective of these methods and tools make them complementary means for surgical assessment of motor skills, especially in the early stages of training. Successful examples such as the Fundamentals of Laparoscopic Surgery should help drive a paradigm change to structured curricula based on objective parameters. These may improve the accreditation of new surgeons, as well as optimize their already overloaded training schedules

Technology adoption in the BIM implementation for lean architectural practice

Justification for Research: the construction companies are facing barriers and challenges in BIM adoption as there is no clear guidance or best practice studies from which they can learn and build up their capacity for BIM use in order to increase productivity, efficiency, quality, and to attain competitive advantages in the global market and to achieve the targets in environmental sustainability. Purpose: this paper aims to explain a comprehensive and systemic evaluation and assessment of the relevant BIM technologies as part of the BIM adoption and implementation to demonstrate how efficiency gains have been achieved towards a lean architectural practice. Design/Methodology/Approach: The research is undertaken through a KTP (Knowledge transfer Partnership) project between the University of Salford and the John McCall Architects based in Liverpool, which is an SME (Small Medium Enterprise). The overall aim of KTP is to develop Lean Design Practice through the BIM adoption and implementation. The overall BIM implementation approach uses a socio-technical view in which it does not only consider the implementation of technology but also considers the socio-cultural environment that provides the context for its implementation. The technology adoption methodology within the BIM implementation approach is the action research oriented qualitative and quantitative research for discovery, comparison, and experimentation as the KTP project with JMA provides an environment for “learning by doing” Findings: research has proved that BIM technology adoption should be undertaken with a bottom-up approach rather than top-down approach for successful change management and dealing with the resistance to change. As a result of the BIM technology adoption, efficiency gains are achieved through the piloting projects and the design process is improved through the elimination of wastes and value generation. Originality/Value: successful BIM adoption needs an implementation strategy. However, at operational level, it is imperative that professional guidelines are required as part of the implementation strategy. This paper introduces a systematic approach for BIM technology adoption based on a case study implementation and it demonstrates a guideline at operational level for other SME companies of architectural practices

Free Technology Academy : a Joint Venture of Free Software and OER

The decision to publish educational materials openly and under free licenses brings up the challenge of doing it in a sustainable way. Some lessons can be learned from the business models for production, maintenance and distribution of Free and Open Source Software. The Free Technology Academy (FTA) has taken on these challenges and has implemented some of these models. We briefly review the FTA educational programme, methodologies and organisation, and see to which extent these models are proving successful in the case of the FTA

Automatic collision avoidance of ships

One of the key elements in automatic simulation of ship manoeuvring in confined waterways is route finding and collision avoidance. This paper presents a new practical method of automatic trajectory planning and collision avoidance based on an artificial potential field and speed vector. Collision prevention regulations and international navigational rules have been incorporated into the algorithm. The algorithm is fairly straightforward and simple to implement, but has been shown to be effective in finding safe paths for all ships concerned in complex situations. The method has been applied to some typical test cases and the results are very encouraging