The ABCD of usability testing

We introduce a methodology for tracking and auditing feedback, errors and suggestions for software packages. This short paper describes how we innovate on the evaluation mechanism, introducing an (Antecedent, Barrier, Consequence and Development) ABCD form, embedded within an eParticipation platform to enable end users to easily report on any usability issues. This methodology will be utilised to improve the STEP cloud eParticipation platform (part of the current STEP Horizon2020 project http://step4youth.eu. The platform is currently being piloted in real life contexts, with the participation of public authorities that are integrating the eParticipation platform into their regular decision-making practices. The project is involving young people, through engagement and motivation strategies and giving them a voice in Environmental decision making at the local level. The pilot evaluation aims to demonstrate how open engagement needs to be embedded within public sector processes and the usability methodology reported here will help to identify the key barriers for wide scale deployment of the platform

Project Testbed: Argument Mapping and Deliberation Analytics

One key goal of the Catalyst project was to design metrics that could capture and represent aspects of the conversation’s structural quality, to assist harvesters and moderators. Many such metrics, alerts and visualizations were developed in the course of the project, but initial user testing has shown that users find it difficult to interpret abstract signals. Following that, we have both introduced new analytics that we felt could be more directly useful, and improved the representation of existing ones. We evaluated their usefulness in a smaller conversation and in experimental settings

Toward a document evaluation methodology: What does research tell us about the validity and reliability of evaluation methods?

Although the usefulness of evaluating documents has become generally accepted among communication professionals, the supporting research that puts evaluation practices empirically to the test is only beginning to emerge. This article presents an overview of the available research on troubleshooting evaluation methods. Four lines of research are distinguished concerning the validity of evaluation methods, sample composition, sample size, and the implementation of evaluation results during revisio

Collective Intelligence Analytics Dashboard Usability Evaluation

Online deliberations can reach a size where it is not possible anymore to quickly infer what is going on in a debate. This report presents results from the usefulness and usability evaluation of visualisations that aid the sense-making of large debates. Based on the results of the evaluations we prepared a set of recommendations to inform CI tool providers about the usefulness and usability of each visualisation

Investigating Automatic Static Analysis Results to Identify Quality Problems: an Inductive Study

Background: Automatic static analysis (ASA) tools examine source code to discover "issues", i.e. code patterns that are symptoms of bad programming practices and that can lead to defective behavior. Studies in the literature have shown that these tools find defects earlier than other verification activities, but they produce a substantial number of false positive warnings. For this reason, an alternative approach is to use the set of ASA issues to identify defect prone files and components rather than focusing on the individual issues. Aim: We conducted an exploratory study to investigate whether ASA issues can be used as early indicators of faulty files and components and, for the first time, whether they point to a decay of specific software quality attributes, such as maintainability or functionality. Our aim is to understand the critical parameters and feasibility of such an approach to feed into future research on more specific quality and defect prediction models. Method: We analyzed an industrial C# web application using the Resharper ASA tool and explored if significant correlations exist in such a data set. Results: We found promising results when predicting defect-prone files. A set of specific Resharper categories are better indicators of faulty files than common software metrics or the collection of issues of all issue categories, and these categories correlate to different software quality attributes. Conclusions: Our advice for future research is to perform analysis on file rather component level and to evaluate the generalizability of categories. We also recommend using larger datasets as we learned that data sparseness can lead to challenges in the proposed analysis proces

An improved negative selection algorithm based on the hybridization of cuckoo search and differential evolution for anomaly detection

The biological immune system (BIS) is characterized by networks of cells, tissues, and organs communicating and working in synchronization. It also has the ability to learn, recognize, and remember, thus providing the solid foundation for the development of Artificial Immune System (AIS). Since the emergence of AIS, it has proved itself as an area of computational intelligence. Real-Valued Negative Selection Algorithm with Variable-Sized Detectors (V-Detectors) is an offspring of AIS and demonstrated its potentials in the field of anomaly detection. The V-Detectors algorithm depends greatly on the random detectors generated in monitoring the status of a system. These randomly generated detectors suffer from not been able to adequately cover the non-self space, which diminishes the detection performance of the V-Detectors algorithm. This research therefore proposed CSDE-V-Detectors which entail the use of the hybridization of Cuckoo Search (CS) and Differential Evolution (DE) in optimizing the random detectors of the V-Detectors. The DE is integrated with CS at the population initialization by distributing the population linearly. This linear distribution gives the population a unique, stable, and progressive distribution process. Thus, each individual detector is characteristically different from the other detectors. CSDE capabilities of global search, and use of L´evy flight facilitates the effectiveness of the detector set in the search space. In comparison with V-Detectors, cuckoo search, differential evolution, support vector machine, artificial neural network, na¨ıve bayes, and k-NN, experimental results demonstrates that CSDE-V-Detectors outperforms other algorithms with an average detection rate of 95:30% on all the datasets. This signifies that CSDE-V-Detectors can efficiently attain highest detection rates and lowest false alarm rates for anomaly detection. Thus, the optimization of the randomly detectors of V-Detectors algorithm with CSDE is proficient and suitable for anomaly detection tasks