Sensitive and Scalable Online Evaluation with Theoretical Guarantees

Multileaved comparison methods generalize interleaved comparison methods to provide a scalable approach for comparing ranking systems based on regular user interactions. Such methods enable the increasingly rapid research and development of search engines. However, existing multileaved comparison methods that provide reliable outcomes do so by degrading the user experience during evaluation. Conversely, current multileaved comparison methods that maintain the user experience cannot guarantee correctness. Our contribution is two-fold. First, we propose a theoretical framework for systematically comparing multileaved comparison methods using the notions of considerateness, which concerns maintaining the user experience, and fidelity, which concerns reliable correct outcomes. Second, we introduce a novel multileaved comparison method, Pairwise Preference Multileaving (PPM), that performs comparisons based on document-pair preferences, and prove that it is considerate and has fidelity. We show empirically that, compared to previous multileaved comparison methods, PPM is more sensitive to user preferences and scalable with the number of rankers being compared.Comment: CIKM 2017, Proceedings of the 2017 ACM on Conference on Information and Knowledge Managemen

Anticipating user eXperience with a desired product: The AUX framework

Positive user experience (UX) has become a key factor in designing interactive products. It acts as a differentiator which can determine a product’s success on the mature market. However, current UX frameworks and methods do not fully support the early stages of product design and development. During these phases, assessment of UX is challenging as no actual user-product interaction can be tested. This qualitative study investigated anticipated user experience (AUX) to address this problem. Using the co-discovery method, participants were asked to imagine a desired product, anticipate experiences with it, and discuss their views with another participant. Fourteen sub-categories emerged from the data, and relationships among them were defined through co-occurrence analysis. These data formed the basis of the AUX framework which consists of two networks which elucidate 1) how users imagine a desired product and 2) how they anticipate positive experiences with that product. Through this AUX framework, important factors in the process of imagining future products and experiences were learnt, including the way in which these factors interrelate. Focusing on and exploring each component of the two networks in the framework will allow designers to obtain a deeper understanding of the required pragmatic and hedonic qualities of product, intended uses of product, user characteristics, potential contexts of experience, and anticipated emotions embedded within the experience. This understanding, in turn, will help designers to better foresee users’ underlying needs and to focus on the most important aspects of their positive experience. Therefore, the use of the AUX framework in the early stages of product development will contribute to the design for pleasurable UX

Library Search UX report summer 2016

During Summer 2016, Imperial College London's Library Information Systems team ran a round of user experience research into the information-seeking behaviour of undergraduate and postgraduate students with a specific focus on the use of the library catalogue and discovery interface. The purpose of the work was to understand user behaviours and preferences to target development of practical improvements to the Library Search interface

Transforming Museum Experience Using User-Experience Based New Product Development

Abstract. Museum is a public space in which the society can find both educational and recreational values. According to Hooper-Greenhill (1999), museum should function as one of educational media for society, therefore the relevance of its product and its function is considered important. However, along with the vast development modern civilization in big cities in Indonesia, museum becomes less popular compared to other public spaces such as mall. The problem lies on the unninovative product presented in museum, which is described as stagnant, not interactive and unattractive by its target market: children and family. Therefore, researcher attempt to solve the problem through proposing new product development (NPD) method. In today’s world, industrial-age product development methods which only focused on feature is proven to be inefective, leaving company to build something people don’t need. Thus, new product development method is invented, putting user research as first step of the entire product development process. Taking also into account, a great product is not merely build based on feature, but experience. That’s where researcher need to consider User Experience as the core of the product. Therefore, this research is conducted to examine the first step of product development process based on Crawford & Benedetto Model with focus on User Experience. Bandung Geological Museum is choosen as the site of research based on its significancy toward West Java people, specifically Bandung people. Researcher attempt to asses current need of museum visitors compared to the current experience presented using Laddering Method interview proposed by Zaaman & Abeele in 2010 based on following aspect: motivation, feel, thought and learn. Later, researcher presented them with three different level of product interactivity based on Fragidis et al model proposed in 2014. The result shows that the most desireable experience desired by children as Bandung Geological Museum visitors are related to these keyword: co-creation interactivity, familiarity and dinosaurs-themed.As a response to the finding, researcher propose an idea of creating buildable paper dinosaurs mask with voice changer feature. This idea is a result of brainstorming session, produce and build with PT Sembilan Matahari, a creative design studio based in Bandung.Keyword: museum, user experience, new product development(NPD

Agile Requirements Engineering: A systematic literature review

Nowadays, Agile Software Development (ASD) is used to cope with increasing complexity in system development. Hybrid development models, with the integration of User-Centered Design (UCD), are applied with the aim to deliver competitive products with a suitable User Experience (UX). Therefore, stakeholder and user involvement during Requirements Engineering (RE) are essential in order to establish a collaborative environment with constant feedback loops. The aim of this study is to capture the current state of the art of the literature related to Agile RE with focus on stakeholder and user involvement. In particular, we investigate what approaches exist to involve stakeholder in the process, which methodologies are commonly used to present the user perspective and how requirements management is been carried out. We conduct a Systematic Literature Review (SLR) with an extensive quality assessment of the included studies. We identified 27 relevant papers. After analyzing them in detail, we derive deep insights to the following aspects of Agile RE: stakeholder and user involvement, data gathering, user perspective, integrated methodologies, shared understanding, artifacts, documentation and Non-Functional Requirements (NFR). Agile RE is a complex research field with cross-functional influences. This study will contribute to the software development body of knowledge by assessing the involvement of stakeholder and user in Agile RE, providing methodologies that make ASD more human-centric and giving an overview of requirements management in ASD.Ministerio de Economía y Competitividad TIN2013-46928-C3-3-RMinisterio de Economía y Competitividad TIN2015-71938-RED

Critical Success Factors for Positive User Experience in Hotel Websites: Applying Herzberg's Two Factor Theory for User Experience Modeling

This research presents the development of a critical success factor matrix for increasing positive user experience of hotel websites based upon user ratings. Firstly, a number of critical success factors for web usability have been identified through the initial literature review. Secondly, hotel websites were surveyed in terms of critical success factors identified through the literature review. Thirdly, Herzberg's motivation theory has been applied to the user rating and the critical success factors were categorized into two areas. Finally, the critical success factor matrix has been developed using the two main sets of data.Comment: Journal articl

Bespoke controllers and their impact on game feel

This project explores the role and effect of bespoke game controllers on game feel and accessibility. The project aims to develop a range of custom controllers and associated games, designed to tightly couple the user interaction with the game execution. Player analytics will be collected to quantify, explore and improve the user experience.Game feel encompasses a wide range of definitions including intuitive controls, physical interaction with virtual objects, and the appeal of playing. Game feel is subjective and therefore challenging to quantify, but something every player instinctively recognises. There is a growing niche of games utilising bespoke hardware to deliver a unique player experience, such as games played with Unicorn horns or by shouting, rather than the traditional controller. The project seeks to push the boundaries of games and explore how game design, user experience and accessibility are affected when the limitation of traditional input systems is removed. The project has developed four bespoke controllers, utilising creative technologies and physical sensors. The controllers are based on inputs using force sensitive sensors, flex sensors, distance sensor and orientation sensors. Prototype games have been developed for each controller designed to exploit the controllers unique form of user interaction. The next steps are to evaluate the player experience and explore how the various controllers are utilised using a data-driven approach.The project proposes that the bespoke controller could widen engagement and improve the user experience by delivering a more intuitive interface and improving game feel. The project will provide insights and design guidelines for both of the development of the custom game controllers and how they can be linked to certain game mechanics in the associated games

What Are We Doing with the Website: Transition, Templates, and User Experience in One Special Collections Library

[Excerpt] At the Eberly Family Special Collections Library (SCL), we have found that our website is often the first place a researcher will look to learn about our repository. Our online web presence is a business card, our chance to make a positive first impression. While our library, among others, has devoted time and resources to the development of new access tools and discovery layers, we have learned that our online presence also needs updates, revisions, and improvements. New tools and access points are valuable, but we can also improve existing tools even as we look forward to new developments in access and discovery. Through conscious efforts to include end users’ feedback in our website design decisions, we create more effective online tools. Our website is a crucial component of our efforts to direct users to our collections, and to publicize our services and programs. In this same vein, our end users can contribute to this design partnership through dedicated user experience testing. The SCL experimented with collaborative decision-making with its website committee, as well as with user experience testing in order to support our requests for additional web development work from the Libraries’ Information Technology department (I-Tech). Through this process, our library gained a more holistic understanding of the needs of online special collections and archives users; we also learned how to communicate more effectively between the department who worked with end users (SCL) and the department performing the actual web development work (I-Tech). While development work was limited to working within the mandatory web template, our user experience testing and the efforts of our internal website committee resulted in a better online experience for our stakeholders, based on the feedback we received from usability testing. Although our website is always a work in progress, we feel that we were able to develop practical ways to adjust to a website migration within in a dispersed and hierarchical information technology environment

Serious Game Evaluation as a Meta-game

Purpose – This paper aims to briefly outline the seamless evaluation approach and its application during an evaluation of ORIENT, a serious game aimed at young adults. Design/methodology/approach – In this paper, the authors detail a unobtrusive, embedded evaluation approach that occurs within the game context, adding value and entertainment to the player experience whilst accumulating useful data for the development team. Findings – The key result from this study was that during the “seamless evaluation” approach, users were unaware that they had been participating in an evaluation, with instruments enhancing rather than detracting from the in-role game experience. Practical implications – This approach, seamless evaluation, was devised in response to player expectations, perspectives and requirements, recognising that in the evaluation of games the whole process of interaction including its evaluation must be enjoyable and fun for the user. Originality/value – Through using seamless evaluation, the authors created an evaluation completely embedded within the “magic circle” of an in-game experience that added value to the user experience whilst also yielding relevant results for the development team

Development of a user experience enhanced teleoperation approach

In this paper, we have investigated various techniques that can be used to enhance user experience for robot teleoperation. In our teleoperation system design, the human operator are provided with both immersive visual feedback and intuitive skill transfer interface such that when controlling a telerobot arm, a user is able to feeļ in a first person perspective in terms of both visual and haptic sense. A number of high-tech devices including Omni haptic joystick, MYO armband, Oculus Rift DK2 headset, and Kinect v2 camera are integrated. The surface electromyography (sEMG) signal allows operator to naturally and efficiently transfer his/her motion skill to the robot, based on the properly designed elastic force feedback. For visual feedback, operators can control the pose of a camera on the head of the robot via the wearable visual headset, such that the operator is able to perceive from the roboţs perspective. Extensive tests have been performed with human subjects to evaluate the design, and the experimental results have shown that superior performance and better user experience have been achieved by the proposed method in comparison with the traditional methods