A comparison of field-based and lab-based experiments to evaluate user experience of personalised mobile devices

There is a growing debate in the literature regarding the tradeoffs between lab and field evaluation of mobile devices. This paper presents a comparison of field-based and lab-based experiments to evaluate user experience of personalised mobile devices at large sports events. A lab experiment is recommended when the testing focus is on the user interface and application-oriented usability related issues. However, the results suggest that a field experiment is more suitable for investigating a wider range of factors affecting the overall acceptability of the designed mobile service. Such factors include the system function and effects of actual usage contexts aspects. Where open and relaxed communication is important (e.g., where participant groups are naturally reticent to communicate), this is more readily promoted by the use of a field study

Designing intelligent support for learning from and in everyday contexts

Motivation and engagement in learning benefit from a good match of learning settings and materials to individual learner contexts. This includes intrinsic context factors such as prior knowledge and personal interests but also extrinsic factors such as the current environment. Recent developments in adaptive and intelligent technology enable the personalisation of context-aware learning. For example, computer vision algorithms, machine translation, and Augmented Reality make it possible to support the creation of meaningful connections between learners and their context. However, for successful adoption in everyday life, these technologies also need to consider the learner experience. This thesis investigates the design of personalised context-aware learning experiences through the lens of ubiquitous and self-directed language learning as a multi-faceted learning domain. Specifically, it presents and discusses the design, implementation, and evaluation of technology support for learning in and from learners’ everyday contexts with a strong focus on the learner perspective and user experience. The work is guided by four different roles that technology can take on in context-aware ubiquitous learning: For enhancing learning situations, it can (1) sense and (2) trigger in learners’ everyday contexts. For enhancing learning contents, it can (3) augment activities and (4) generate learning material from learner everyday contexts. With regards to the sensing role, the thesis investigates how learners typically use mobile learning apps in everyday contexts. Activity and context logging, combined with experience sampling, confirm that mobile learning sessions spread across the day and occur in different settings. However, they are typically short and frequently interrupted. This indicates that learners may benefit from better integrating learning into everyday contexts, e.g. by supporting task resumption. Subsequently, we explore how this integration could be supported with intelligent triggers linked to opportune moments for learning. We conceptualise and evaluate different trigger types based on interaction patterns and context detection. Our findings show that simple interactions (e.g. plugging in headphones) are promising for capturing both availability and willingness to engage in a learning activity. We discuss how similar interaction triggers could be adapted to match individual habits. In the area of enhancing learning contents, we first investigate how enjoyable everyday activities could be augmented for learning without disrupting these activities. Specifically, we assess the learner experience with interactive grammar support in e-readers and adapted captions for audio-visual media. Participants in our studies felt that the learning augmentations successfully supported their learning process. The information load of the learning support should match the learners’ current needs to maintain the activity flow. Learners may need encouragement to opt for novel concepts optimised for learning (e.g. time-synchronised captions) rather than sticking to habits (e.g. standard captions). Next, the thesis explores learner needs and preferences in generating their own personalised learning material from their context. We design and evaluate automated content generation methods that generate learning opportunities from objects in the learner’s environment. The connection to the learner’s context is established with state-of-the-art technology, such as object detection and Augmented Reality. Through several user studies, we show that learning performance and engagement with auto-generated personalised learning material is comparable to predefined and manually generated content. Findings further indicate that the success of personalisation depends on the effort required to generate content and whether the generation results match the learner’s expectations. Through the different perspectives examined in this thesis, we provide new insights into challenges and opportunities that we synthesise in a framework for context-aware ubiquitous learning technology. The findings also have more general implications for the interaction design of personalised and context-aware intelligent systems. Notably, for the auto-generation of personalised content, it is essential to consider not only correctness from a technological perspective but also how users may perceive the results.Lernmotivation und Engagement profitieren davon, wenn Lernumgebungen und Lernmaterialien auf den individuellen Kontext der Lernenden abgestimmt sind. Dieser umfasst sowohl intrinsische Faktoren wie Vorkenntnisse und persönliche Interessen, aber auch extrinsische Faktoren wie die aktuelle Umgebung. Aktuelle Weiterentwicklungen im Bereich adaptiver und intelligenter Technologien ermöglichen es, Lernen kontextbewusst zu personalisieren. So können mithilfe von Computer-Vision-Algorithmen, maschineller Übersetzung und Augmented Reality sinnvolle Verknüpfungen zwischen Lernenden und ihrem Kontext geschaffen werden. Allerdings müssen diese Technologien für einen erfolgreichen Einsatz im Alltag auch die Lernerfahrung mit einbeziehen. Diese Arbeit untersucht die Gestaltung personalisierter kontextbewusster Lernerfahrungen aus der Perspektive des ubiquitären und self-directed Learning im Sprachenlernen, einem vielseitigen Lernbereich. Insbesondere wird die Konzeption, Implementierung und Evaluierung von Technologieunterstützung für das Sprachenlernen in und aus dem Alltagskontext der Lernenden vorgestellt und diskutiert, wobei der Schwerpunkt auf der Perspektive der Lernenden und der Nutzererfahrung liegt. Die Arbeit orientiert sich an vier verschiedenen Rollen, die Technologie im kontextbewussten Lernen einnehmen kann. Um Lernsituationen anzureichern, kann Technologie im Alltagskontext von Lernenden (1) erfassen und (2) auslösen. Um Lerninhalte anzureichern, kann Technologie aus dem Alltagskontext (3) Aktivitäten augmentieren und (4) Inhalte generieren. Im Hinblick auf die erfassende Rolle von Technologie wird in dieser Arbeit untersucht, wie die Lernenden mobile Lern-Apps in alltäglichen Kontexten nutzen. Die Aufzeichnung von Aktivitäten und Kontexten in Kombination mit Experience Sampling bestätigt, dass Lerneinheiten im mobilen Lernen über den Tag verteilt sind und in verschiedenen Umgebungen stattfinden. Allerdings sind sie in der Regel kurz und werden häufig unterbrochen. Dies deutet darauf hin, dass die Lernenden von einer besseren Integration des Lernens in ihren Alltagskontext profitieren könnten, z. B. durch Unterstützung des Wiedereinstiegs nach einer Unterbrechung. Anschließend untersuchen wir, wie diese Integration durch intelligente Trigger unterstützt werden könnte, die mit passenden Lernzeitpunkten verknüpft sind. Wir konzipieren und evaluieren verschiedene Arten von Triggern auf Basis von Interaktionsmustern und Kontexterkennung. Unsere Ergebnisse zeigen, dass einfache Interaktionen (z. B. das Einstecken von Kopfhörern) vielversprechend dafür sind, sowohl die Verfügbarkeit als auch die Bereitschaft für eine Lernaktivität zu erfassen. Wir diskutieren, wie ähnliche Interaktionstrigger an individuelle Gewohnheiten angepasst werden können. Im Bereich der Augmentierung von Lerninhalten untersuchen wir zunächst, wie unterhaltsame Alltagsaktivitäten für das Lernen aufbereitet werden können, ohne diese Aktivitäten zu beeinträchtigen. Konkret bewerten wir die Lernerfahrung mit interaktiver Grammatikunterstützung in E-Readern und angepassten Untertiteln für audiovisuelle Medien. Die Teilnehmer:innen unserer Studien fanden, dass die Lernunterstützung ihren Lernprozess erfolgreich förderte. Die Informationslast im Lernsystem sollte auf die aktuellen Bedürfnisse der Lernenden angepasst werden, damit das Flow-Erlebnis nicht beeinträchtigt wird. Die Lernenden brauchen möglicherweise Ermutigung dafür, sich für neuartige, lernoptimierte Konzepte zu entscheiden (z. B. zeitsynchrone Untertitel), anstatt an Gewohnheiten festzuhalten (z. B. Standarduntertitel). Als Nächstes werden in dieser Arbeit die Bedürfnisse und Präferenzen der Lernenden bei der Erstellung ihres eigenen personalisierten Lernmaterials aus ihrem Kontext untersucht. Insbesondere werden Methoden zur automatischen Generierung von Inhalten entwickelt und evaluiert, die Lernmöglichkeiten aus Objekten in der Umgebung des Lernenden generieren. Die Verbindung zum Kontext des Lernenden wird durch aktuelle Technologien wie Objekterkennung und Augmented Reality hergestellt. Wir zeigen anhand mehrerer Nutzerstudien, dass die Lernleistung und das Engagement bei automatisch personalisiertem Lernmaterial mit vordefinierten und manuell erstellten Inhalten vergleichbar sind. Die Ergebnisse zeigen außerdem, dass der Erfolg der Personalisierung vom Aufwand abhängt, der für die Erstellung der Inhalte erforderlich ist, und davon, ob die generierten Materialien den Erwartungen der Lernenden entsprechen. Die verschiedenen Perspektiven, die in dieser Arbeit untersucht werden, bieten neue Einblicke in Herausforderungen und Möglichkeiten, die wir in einem Framework für kontextbewusste ubiquitäre Lerntechnologie zusammenfassen. Die Ergebnisse haben auch allgemeinere Auswirkungen auf die Gestaltung der Interaktion mit personalisierten und kontextbewussten intelligenten Systemen. Beispielsweise ist es bei der automatischen Generierung personalisierter Inhalte wichtig, nicht nur die Korrektheit aus technologischer Sicht zu berücksichtigen, sondern auch, wie die Nutzer die Ergebnisse wahrnehmen

Model-driven Personalisation of Human-Computer Interaction across Ubiquitous Computing Applications

Personalisation is essential to Ubiquitous Computing (Ubicomp), which focuses on a human-centred paradigm aiming to provide interaction with adaptive content, services, and interfaces towards each one of its users, according to the context of the applications’ scenarios. However, the provision of that appropriated personalised interaction is a true challenge due to different reasons, such as the user interests, heterogeneous environments and devices, dynamic user behaviour and data capture. This dissertation focuses on a model-driven personalisation solution that has the main goal of facili-tating the implementation of a personalised human-computer interaction across different Ubicomp scenarios and applications. The research reported here investigates how a generic and interoperable model for personalisation can be used, shared and processed by different applications, among diverse devices, and across different scenarios, studying how it can enrich human-computer interaction. The research started by the definition of a consistent user model with the integration of context to end in a pervasive model for the definition of personalisations across different applications. Besides the model proposal, the other key contributions within the solution are the modelling frame-work, which encapsulates the model and integrates the user profiling module, and a cloud-based platform to pervasively support developers in the implementation of personalisation across different applications and scenarios. This platform provides tools to put end users in control of their data and to support developers through web services based operations implemented on top of a personalisa-tion API, which can also be used independently of the platform for testing purposes, for instance. Several Ubicomp applications prototypes were designed and used to evaluate, at different phases, both the solution as a whole and each one of its components. Some were specially created with the goal of evaluating specific research questions of this work. Others were being developed with a pur-pose other than for personalisation evaluation, but they ended up as personalised prototypes to better address their initial goals. The process of applying the personalisation model to the design of the latter should also work as a proof of concept on the developer side. On the one hand, developers have been probed with the implementation of personalised applications using the proposed solution, or a part of it, to assess how it works and can help them. The usage of our solution by developers was also important to assess how the model and the platform respond to the developers’ needs. On the other hand, some prototypes that implement our model-driven per-sonalisation solution have been selected for end user evaluation. Usually, user testing was conducted at two different stages of the development, using: (1) a non-personalised version; (2) the final per-sonalised version. This procedure allowed us to assess if personalisation improved the human-com-puter interaction. The first stage was also important to know who were the end users and gather interaction data to come up with personalisation proposals for each prototype. Globally, the results of both developers and end users tests were very positive. Finally, this dissertation proposes further work, which is already ongoing, related to the study of a methodology to the implementation and evaluation of personalised applications, supported by the development of three mobile health applications for rehabilitation

Online learning of personalised human activity recognition models from user-provided annotations

PhD ThesisIn Human Activity Recognition (HAR), supervised and semi-supervised training are important tools for devising parametric activity models. For the best modelling performance, large amounts of annotated personalised sample data are typically required. Annotating often represents the bottleneck in the overall modelling process as it usually involves retrospective analysis of experimental ground truth, like video footage. These approaches typically neglect that prospective users of HAR systems are themselves key sources of ground truth for their own activities. This research therefore involves the users of HAR monitors in the annotation process. The process relies solely on users' short term memory and engages with them to parsimoniously provide annotations for their own activities as they unfold. E ects of user input are optimised by using Online Active Learning (OAL) to identify the most critical annotations which are expected to lead to highly optimal HAR model performance gains. Personalised HAR models are trained from user-provided annotations as part of the evaluation, focusing mainly on objective model accuracy. The OAL approach is contrasted with Random Selection (RS) { a naive method which makes uninformed annotation requests. A range of simulation-based annotation scenarios demonstrate that using OAL brings bene ts in terms of HAR model performance over RS. Additionally, a mobile application is implemented and deployed in a naturalistic context to collect annotations from a panel of human participants. The deployment is proof that the method can truly run in online mode and it also shows that considerable HAR model performance gains can be registered even under realistic conditions. The ndings from this research point to the conclusion that online learning from userprovided annotations is a valid solution to the problem of constructing personalised HAR models

Horizon Report Europe - 2014 Schools Edition

The NMC Horizon Project from the New Media Consortium is a long-term investigation launched in 2002 that identifies and describes emerging technologies likely to have a large impact over the coming five years in education around the globe. The NMC Horizon Report Europe: 2014 Schools Edition, the first of its kind for Europe, examines six key trends, six significant challenges and six important developments in educational technology that are very likely to impact educational change processes in European schools over the next five years (2014-2018). The topics within each section were carefully selected by the Horizon Project Europe Expert Panel, a body of 53 experts in European education, technology, and other fields. They come from 22 European countries, as well as international organisations and European networks. Throughout the report, references and links are made to more than 150 European publications (reports, articles, policy documents, blog posts etc.), projects (both EU-funded and national initiatives) and various policy initiatives from all over Europe. The Creative Classrooms multidimensional framework, developed by European Commission’s JRC-IPTS on behalf of DG EAC, was used for analysing the trends, challenges and technologies impacting European schools over the next five years. The analysis reveals that a systemic approach is needed for integrating new technologies in European schools and impacting educational change over the next five years.JRC.J.3-Information Societ

Personalization in cultural heritage: the road travelled and the one ahead

Over the last 20 years, cultural heritage has been a favored domain for personalization research. For years, researchers have experimented with the cutting edge technology of the day; now, with the convergence of internet and wireless technology, and the increasing adoption of the Web as a platform for the publication of information, the visitor is able to exploit cultural heritage material before, during and after the visit, having different goals and requirements in each phase. However, cultural heritage sites have a huge amount of information to present, which must be filtered and personalized in order to enable the individual user to easily access it. Personalization of cultural heritage information requires a system that is able to model the user (e.g., interest, knowledge and other personal characteristics), as well as contextual aspects, select the most appropriate content, and deliver it in the most suitable way. It should be noted that achieving this result is extremely challenging in the case of first-time users, such as tourists who visit a cultural heritage site for the first time (and maybe the only time in their life). In addition, as tourism is a social activity, adapting to the individual is not enough because groups and communities have to be modeled and supported as well, taking into account their mutual interests, previous mutual experience, and requirements. How to model and represent the user(s) and the context of the visit and how to reason with regard to the information that is available are the challenges faced by researchers in personalization of cultural heritage. Notwithstanding the effort invested so far, a definite solution is far from being reached, mainly because new technology and new aspects of personalization are constantly being introduced. This article surveys the research in this area. Starting from the earlier systems, which presented cultural heritage information in kiosks, it summarizes the evolution of personalization techniques in museum web sites, virtual collections and mobile guides, until recent extension of cultural heritage toward the semantic and social web. The paper concludes with current challenges and points out areas where future research is needed

Event and map content personalisation in a mobile and context-aware environment.

Effective methods for information access are of the greatest importance for our modern lives “ particularly with respect to handheld devices. Personalisation is one such method which models a users characteristics to deliver content more focused to the users needs. The emerging area of sophisticated mobile computing devices has started to inspire new forms of personalised systems that include aspects of the persons contextual environment. This thesis seeks to understand the role of personalisation and context, to evaluate the effectiveness of context for content personalisation and to investigate the event and map content domain for mobile usage. The work presented in this thesis has three parts: The first part is a user experiment on context that investigated the contextual attributes of time, location and interest, with respect to participants perception of their usefulness. Results show highly dynamic and interconnected effects of context on participants usefulness ratings. In the second part, these results were applied to create a predictive model of context that was related to attribution theory and then combined with an information retrieval score to create a weighted personalisation model. In the third part of this work, the personalisation model was applied in a mobile experiment. Participants solved situational search tasks using a (i) non-personalized and a (ii) personalized mobile information system, and rating entertainment events based on usefulness. Results showed that the personalised system delivered about 20% more useful content to the mobile user than the non-personalised system, with some indication for reduced search effort in terms of time and the amount of queries per task. The work presented provides evidence for the promising potential of context to facilitate personalised information delivery to users of mobile devices. Overall, it serves as an example of an investigation into the effectiveness of context from multiple angles and provides a potential link to some of the aspects of psychology as a potential source for a deeper understanding of contextual processes in humans

Situated navigation support for heterogeneous large crowds via augmented signage

PhD ThesisNavigating unfamiliar places is a common problem people face, and there is a wealth of commercial and research-based applications particularly for mobile devices that provide support in these settings. While many of these solutions work well on an individual level, they are less well suited for very crowded situations, e.g. sports matches, festivals and fairs, or religious events such as pilgrimages. In a large crowd, attending to a mobile device can be hazardous, the underlying technology might not scale well, and some people might be excluded due to not having access to a mobile device. Public signage does not suffer from these issues, and consequently, people frequently rely on signage in crowded settings. However, a key disadvantage of public signage is that it does not provide personalised navigation support. We have therefore investigated augmented signage as a navigation support system for use in large crowds. This thesis investigates the issues of guidance by augmented displays and how this can be made more suitable for people who navigate in groups in unfamiliar areas. In this context we have undertaken three studies as examples to explore how augmented displays can provide aid to people in crowded places. In the first study, we investigated the question of whether the use of dynamic public signage can help pilgrims count or remember the Tawaf rounds while walking around the Ka’bah. We analysed the current situation in Mecca based on a literature review and a series of interviews with pilgrims, who had completed at least one pilgrimage (already visited Mecca). We then presented a prototypical dynamic signage and reported on a user study we conducted in a realistic setting in order to evaluate the system. The results suggest that dynamic signage may be a feasible option to improve the safety of pilgrims in Mecca. In the second study, we introduced a scalable signage-based approach and present results from a comparison study contrasting two designs for augmented signage with a base approach. The results provide evidence that such a system could be easily useable, may reduce task load, and could improve navigation performance. In the final study, we developed public displays (static and dynamic signage) and investigated the ability of using such displays to assist pilgrims of Mecca to find each other after becoming separated while performing rituals inside the Haram (e.g. Tawaf pillar). Once again here we have addressed the issue through a series of interviews with people who had experienced pilgrimage before. Then we constructed a full idea that allowed us to design the initial system and presented it in a focus group session to gain feedback and redesign the system. Afterwards, we conducted a lab-based user study. The results we obtained suggest that a person can extract information (by reading the dynamic signage), also results showed that users were able to remember their information (whilst completing some distraction tasks), and then they completed the static signs tasks successfully. Generally results showed that the system can indicate people to the right place where they can meet again after becoming separated. In general, these results provided good evidence that augmented signage supported by colour and visual codes might provide considerable help in situations with large and heterogeneous crowds. It might be developed and used in different settings for provisional navigation information and allow multi-users to extract their personalised information individually