Assessing usability of device types with novel function sets that are adopted by diverse user groups requires one to explore a variety of approaches. In this paper, we develop such an approach to assess usability of smartphone devices. Using a three-stage Delphi-method study, we identify sets of benchmark tasks that can be used to assess usability for various user types. These task sets enable one to evaluate smartphone platforms from two perspectives: ease of learning (for those unfamiliar with smartphone use) and ease of use (for experienced users). We then demonstrate an approach for using this task set by performing an exploratory study of both inexperienced smartphone users (using a convenience sample) and experienced users (using the keystroke model). Our exploration illustrates the methodology for using such a task set and, in so doing, reveals significant differences among the leading smartphone platforms between novice and expert users. As such, we provide some preliminary evidence that ease of use is indeed significantly different from ease of learning
