The LASER Workshop: Learning from Authoritative Security Experiment Results
Abstract
Background: Human beings are an integral part of computer
security, whether we actively participate or simply
build the systems. Despite this importance, understanding
users and their interaction with security is a blind spot
for most security practitioners and designers. / Aim: Define principles for conducting experiments into
usable security and privacy, to improve study robustness
and usefulness. / Data: The authors’ experiences conducting several research
projects complemented with a literature survey.
Method: We extract principles based on relevance to the
advancement of the state of the art. We then justify our
choices by providing published experiments as cases of
where the principles are and are not followed in practice
to demonstrate the impact. Each principle is a discipline specific
instantiation of desirable experiment-design elements
as previously established in the domain of philosophy
of science. / Results: Five high-priority principles – (i) give participants
a primary task; (ii) incorporate realistic risk;
(iii) avoid priming the participants; (iv) perform doubleblind
experiments whenever possible and (v) think carefully
about how meaning is assigned to the terms threat
model, security, privacy, and usability. / Conclusion: The principles do not replace researcher
acumen or experience, however they can provide a valuable
service for facilitating evaluation, guiding younger
researchers and students, and marking a baseline common
language for discussing further improvements