User Evaluation Methods for Visual Web Search Interfaces

In recent years, numerous visual Web search interfaces have been developed in the research community. How-ever, the user evaluations of these interfaces have been per-formed using a wide range of methods, making it difficult to compare and verify the relative value of the proposed advancements. In this paper, we survey these evaluation methods, and propose a stepped evaluation and refinement model for the systematic study and enhancement of visual Web search interfaces. We suggest that this stepped model can be generalized to support the evaluation of other in-formation visualization systems that target exploratory or knowledge-centric domains.

From Keyword Search to Exploration: How Result Visualization Aids Discovery on the Web

A key to the Web's success is the power of search. The elegant way in which search results are returned is usually remarkably effective. However, for exploratory search in which users need to learn, discover, and understand novel or complex topics, there is substantial room for improvement. Human computer interaction researchers and web browser designers have developed novel strategies to improve Web search by enabling users to conveniently visualize, manipulate, and organize their Web search results. This monograph offers fresh ways to think about search-related cognitive processes and describes innovative design approaches to browsers and related tools. For instance, while key word search presents users with results for specific information (e.g., what is the capitol of Peru), other methods may let users see and explore the contexts of their requests for information (related or previous work, conflicting information), or the properties that associate groups of information assets (group legal decisions by lead attorney). We also consider the both traditional and novel ways in which these strategies have been evaluated. From our review of cognitive processes, browser design, and evaluations, we reflect on the future opportunities and new paradigms for exploring and interacting with Web search results

Design and Evaluation of User Interfaces for Mobile Web Search

Mobiili tiedonhaku on jatkuvasti kasvava ja monimuotoistuva osa jokapäiväistä tiedonhankintaa. Aikaisemman tutkimuksen mukaan tarvitaan kuitenkin parempia käyttöliittymäratkaisuja tukemaan mobiililaitteilla tapahtuvaa verkkotiedonhakua. Väitöskirjatutkimuksessa suunniteltiin ja toteutettiin kaksi uutta hakukäyttöliittymää, joita arvioitiin käyttäjätutkimuksissa. Ensimmäinen käyttöliittymä perustuu siihen, että hakutulokset luokitellaan ryhmiin niissä esiintyvien avainsanojen perusteella. Käyttäjätutkimusten tulokset osoittavat, että luokittelulla voidaan tukea mobiilikäyttäjien tutkivaa tiedonhakua. Toinen käyttöliittymä antaa hakutulosten yhteydessä yleiskuvan hakulauseen sijaintikohdista tulosdokumenteissa. Vaikkakin menetelmän käyttö vaatii opettelua, käyttäjäarviot osoittavat että se voi auttaa sivuuttamaan huonot hakutulokset, etenkin silloin kun muut hakutulosta kuvaavat tiedot ovat epäselviä. Lisäksi väitöskirjassa tutkittiin aktiivisten mobiili-Internetin käyttäjien tiedontarpeita verkkotiedonhaun käytön ymmärtämiseksi. Tutkimustulosten mukaan hakujen tekeminen ja verkon selaaminen ovat näiden käyttäjien tärkeimpiä tiedonhankintatapoja. Niillä pyritään vastaamaan tiedontarpeisiin heti niiden ilmaantuessa, olipa käyttäjä sitten kotona, liikkeessä tai sosiaalisessa vuorovaikutustilanteessa. Mobiili tiedonhankinta on vahvasti sidoksissa käyttötilanteeseen, mikä tulee huomioida hakukäyttöliittymien suunnittelussa. Tulevaisuuden hakukäyttöliittymät voivat esimerkiksi tukea tiedonhankintaa hyödyntämällä tietoa käyttäjän sijainnista ja aktiviteeteista. Myös epämuodollisten ja tutkivien tiedontarpeiden kasvava rooli asettaa uusia haasteita vuorovaikutuksen suunnittelulle.Mobile Web search is a rapidly growing information seeking activity employed across different locations, situations, and activities. Current mobile search interfaces are based on the ranked result list, dominant in desktop interfaces. Research suggests that new paradigms are needed for better support of mobile searchers. For this dissertation, two such novel search interface techniques were designed, implemented, and evaluated. The first method, a clustering search interface that presents a category- based overview of the results, was studied both in a task-based experiment in a laboratory setting and in a longitudinal field study wherein it was used to address real information needs. The results indicate that clustering can support exploratory search needs when the searcher has trouble defining the information need, requires an overview of the search topic, or is interested in multiple results related to the same topic. The findings informed design guidelines for category-based search interfaces. How and when categorization is presented in the search interface needs to be carefully considered. Categorization methods should be improved, for better response to diverse information needs. Hybrid approaches employing contextually informed clustering, classification, and faceted browsing may offer the best match for user needs. The second presentation method, a visualization of the occurrences of the user s query phrase in a result document, can be incorporated into the ranked result list as an additional, unobtrusive result descriptor. It allows the searcher to see how often the query phrase appears in the result document, enabling the use of various evaluation strategies to assess the relevance of the results. Several iterations of the visualization were studied with users to form an understanding of the potential of this approach. The results suggest that a novel visualization can be useful in ruling out non-relevant results and can assist when the other result descriptors do not provide for a conclusive relevance assessment. However, users familiarity with well-established result descriptors means that users have to learn how to integrate the visualization into their search strategies and reconcile situations in which the visualization is in conflict with other metadata. In addition, the contextual triggers and information behaviors of mobile Internet users were studied, for understanding of the role of Web search as a mobile information seeking activity. The results from this study show that mobile Web search and browsing are important information seeking activities. They are engaged in to resolve emerging information needs as they appear, whether at home, on the go, or in social situations

Metadata visualization of scholarly search results

Studies of online search behaviour have found that searchers often face difficulties formulating queries and exploring the search results sets. These shortcomings may be especially problematic in digital libraries since library searchers employ a wide variety of information-seeking methods (with varying degrees of support), and the corpus to be searched is often more complex than simple textual information. To address these problems, an interactive Web-based library search interface is presented, which has been designed to support strategic retrieval behaviour of library searchers. This system takes advantage of the rich metadata associated with academic documents and employs information visualization techniques to provide searchers with additional information-seeking tools. These tools are designed to facilitate visual and interactive query refinement, search results exploration, and citation navigation. User evaluations illustrate the potential benefits of the design choices in comparison to a list-based digital library search interface

Visualisation des résultats de recherche classifiés en contexte de recherche d’information exploratoire : une évaluation d’utilisabilité

La recherche d’information exploratoire sur le Web présente des défis cognitifs en termes de stratégies cognitives et de tactiques de recherche. Le modèle « question-réponse » des moteurs de recherche actuels est inadéquat pour faciliter les stratégies de recherche d’information exploratoire, assimilables aux stratégies cognitives de l’apprentissage. La visualisation des résultats de recherche est un dispositif qui possède des propriétés graphiques et interactives pertinentes pour le traitement de l’information et l’utilisation de la mémoire et, plus largement de la cognition humaine. Plusieurs recherches ont été menées dans ce contexte de recherche d’information exploratoire, mais aucune n’a distinctement isolé le facteur graphique et interactif de la « visualisation » au sein de son évaluation. L’objectif principal de cette thèse est de vérifier si la visualisation des résultats en contexte de recherche d’information exploratoire témoigne des avantages cognitifs et interactifs pressentis selon ses présupposés théoriques. Pour décrire et déterminer la valeur ajoutée de la visualisation des résultats de recherche dans un contexte de recherche d’information exploratoire sur le Web, cette recherche propose de mesurer son utilisabilité. En la comparant selon les mêmes critères et indicateurs à une interface homologue textuelle, nous postulons que l’interface visuelle atteindra une efficacité, efficience et satisfaction supérieure à l’interface textuelle, dans un contexte de recherche d’information exploratoire. Les mesures objectives de l’efficacité et de l’efficience reposent principalement sur l’analyse des traces de l’interaction des utilisateurs, leur nombre et leur durée. Les mesures subjectives attestant de la satisfaction procurée par l’usage du système dans ce contexte repose sur la perception des utilisateurs par rapport à des critères de perception de la facilité d’utilisation et de l’utilité de l’interface testée et par rapport à des questions plus large sur l’expérience de recherche vécue. Un questionnaire et un entretien ont été passés auprès de chacun des vingt-trois répondants. Leur session de recherche a aussi été enregistré par un logiciel de capture vidéo d’écran. Sur les données des vingt-trois utilisateurs divisés en deux groupes, l’analyse statistique a révélé de faibles différences significatives entre les deux interfaces. Selon les mesures effectuées, l’interface textuelle s’est révélée plus efficace en terme de rappel et de pertinence ; et plus efficiente pour les durées de la recherche d’information. Sur le plan de la satisfaction, les interfaces ont été appréciées toutes deux posivitivement, ne permettant pas de les distinguer pour la grande majorité des métriques. Par contre, au niveau du comportement interactif, des différences notables ont montré que les utilisateurs de l’interface visuelle ont réalisé davantage d’interactions de type exploratoire, et ont procédé à une collecte sélective des résultats de recherche. L’analyse statistique et de contenu sur le critère de l’expérience vécue a permis de démontrer que la visualisation offre l’occasion à l’utilisateur de s’engager davantage dans le processus de recherche d’information en raison de l’impact positif de l’esthétique de l’interface visuelle. De plus, la fonctionnalité de classification a été perçue de manière ambivalente, divisant les candidats peu importe l’interface testée. Enfin, l’analyse des verbatims des « visuelle » a permis d’identifier le besoin de fonctionnalités de rétroaction de l’utilisateur afin de pouvoir communiquer le besoin d’information ou sa pondération des résultats ou des classes, grâce à des modalités interactives de manipulation directe des classes sur un espace graphique.Conducting exploratory searches on the web presents a number of cognitive difficulties as regards search strategies and tactics. The “question-response” model used by the available search engines does not respond adequately to exploratory searches, which are akin to cognitive learning strategies. Visualising search results involves graphic and interactive properties for presenting information that are pertinent for processing and using information, as well as for remembering and, more broadly, for human cognition. Many studies have been conducted in the area of exploratory searches, but none have focussed specifically on the graphic and interactive features of visualisation in their analysis. The principal objective of this thesis is to confirm whether the visualisation of results in the context of exploratory searches offers the cognitive and interactive advantages predicted by conjectural theory. In order to describe and to determine the added value of visualising search results in the context of exploratory web searches, the study proposes to measure its usability. By comparing it to a parallel text interface, using the same criteria and indicators, the likelihood of better efficiency, efficacy, and satisfaction when using a visual interface can be established. The objective measures of efficiency and efficacy are based mainly on the analysis of user interactions, including the number of these interactions and the time they take. Subjective measures of satisfaction in using the system in this context are based on user perception regarding ease of use and the usefulness of the interface tested, and on broader questions concerning the experience of using the search interface. These data were obtained using a questionnaire and a discussion with each participant. Statistical analysis of the data from twenty-three participants divided into two groups showed slightly significant differences between the two interfaces. Analysis of the metrics used showed that the textual interface is more efficient in terms of recall and pertinence, and more efficacious concerning the time needed to search for information. Regarding user satisfaction, both interfaces were seen positively, so that no differences emerged for the great majority of metrics used. However, as regards interactive behaviour, notable differences emerged. Participants using the visual interface had more exploratory interaction, and went on to select and collect pertinent search results. Statistical and content analysis of the experience itself showed that visualisation invites the user to become more involved in the search process, because of the positive effect of a pleasing visual interface. In addition, the classification function was perceived as ambivalent, dividing the participants no matter which interface was used. Finally, analysis of the verbatim reports of participants classed as “visual” indicated the need for a user feedback mechanism in order to communicate information needs or for weighting results or classes, using the interactive function for manipulating classes within a geographic space

Aplicación de técnicas de aprendizaje automático a la gestión y optimización de cachés de teselas para la aceleración de servicios de mapas en las infraestructuras de datos espaciales

La gran proliferación en el uso de servicios de mapas a través de la Web ha motivado la necesidad de disponer de servicios cada vez más escalables. Como respuesta a esta necesidad, los servicios de mapas basados en teselado se han perfilado como una alternativa escalable frente a los servicios de mapas tradicionales, permitiendo la actuación de mecanismos de caché o incluso la prestación del servicio mediante una colección de imágenes pregeneradas. Sin embargo, los requisitos de almacenamiento y tiempo de puesta en marcha de estos servicios resultan a menudo prohibitivos cuando la cartografía a servir cubre una zona geográfica extensa para un elevado número de escalas. Por ello, habitualmente estos servicios se ofrecen recurriendo a cachés parciales que contienen tan solo un subconjunto de la cartografía. Para garantizar una Calidad de Servicio (QoS - Quality of Service) aceptable es necesaria la actuación de adecuadas políticas de mantenimiento y gestión de estas cachés de mapas: 1) Estrategias de población inicial ó seeding de la caché. 2) Algoritmos de carga dinámica ante las peticiones de los usuarios. 3) Políticas de reemplazo de caché. Sin embargo, existe un reducido número de estas estrategias que sean específicas para los servicios de mapas. La mayor parte de estrategias aplicadas a estos servicios son extraídas de otros ámbitos, como los proxies Web tradicionales, las cuáles no tienen en cuenta la componente espacial de los objetos de mapa que gestionan. En la presente tesis se aborda este punto de mejora, diseñando nuevos algoritmos específicos para este dominio de aplicación que permitan optimizar el rendimiento de los servicios de mapas. Dado el elevado número de objetos gestionados por estas cachés y la heterogeneidad de los mismos en cuanto a capas, escalas de representación, etc., se ha hecho un esfuerzo para que las estrategias diseñadas sean automáticas o semi-automáticas, requiriendo la menor intervención humana posible. Así, se han propuesto dos novedosas estrategias para la población inicial de una caché de mapas. Una de ellas utiliza un modelo descriptivo mediante los registros de peticiones pasadas del servicio. La otra se basa en un modelo predictivo para la identificación de fenómenos geográficos directores de las peticiones de los usuarios, parametrizado o bien mediante un análisis regresivo OLS (Ordinary Least Squares) o mediante un sistema inteligente con redes neuronales. Asimismo, se han llevado a cabo importantes contribuciones en relación con las estrategias de reemplazo de estas cachés. Por una parte, se ha propuesto un sistema inteligente basado en redes neuronales, que estima la popularidad de acceso futuro en base a ciertas propiedades de los objetos que gestiona: actualidad de referencia, frecuencia de referencia, y el tamaño de la tesela referenciada. Por otra parte, se ha propuesto una estrategia, bautizada como Spatial-LFU, la cual es una variante de la estrategia Perfect-LFU, simplificada aprovechando la correlación espacial existente entre las peticiones.Departamento de Teoría de la Señal y Comunicaciones e Ingeniería Telemátic

Cognitive Activity Support Tools: Design of the Visual Interface

This dissertation is broadly concerned with interactive computational tools that support the performance of complex cognitive activities, examples of which are analytical reasoning, decision making, problem solving, sense making, forecasting, and learning. Examples of tools that support such activities are visualization-based tools in the areas of: education, information visualization, personal information management, statistics, and health informatics. Such tools enable access to information and data and, through interaction, enable a human-information discourse. In a more specific sense, this dissertation is concerned with the design of the visual interface of these tools. This dissertation presents a large and comprehensive theoretical framework to support research and design. Issues treated herein include interaction design and patterns of interaction for cognitive and epistemic support; analysis of the essential properties of interactive visual representations and their influences on cognitive and perceptual processes; an analysis of the structural components of interaction and how different operational forms of interaction components affect the performance of cognitive activities; an examination of how the information-processing load should be distributed between humans and tools during the performance of complex cognitive activities; and a categorization of common visualizations according to their structure and function, and a discussion of the cognitive utility of each category. This dissertation also includes a chapter that describes the design of a cognitive activity support tool, as guided by the theoretical contributions that comprise the rest of the dissertation. Those that may find this dissertation useful include researchers and practitioners in the areas of data and information visualization, visual analytics, medical and health informatics, data science, journalism, educational technology, and digital games

Enhancing the E-Commerce Experience through Haptic Feedback Interaction

The sense of touch is important in our everyday lives and its absence makes it difficult to explore and manipulate everyday objects. Existing online shopping practice lacks the opportunity for physical evaluation, that people often use and value when making product choices. However, with recent advances in haptic research and technology, it is possible to simulate various physical properties such as heaviness, softness, deformation, and temperature. The research described here investigates the use of haptic feedback interaction to enhance e-commerce product evaluation, particularly haptic weight and texture evaluation. While other properties are equally important, besides being fundamental to the shopping experience of many online products, weight and texture can be simulated using cost-effective devices. Two initial psychophysical experiments were conducted using free motion haptic exploration in order to more closely resemble conventional shopping. One experiment was to measure weight force thresholds and another to measure texture force thresholds. The measurements can provide better understanding of haptic device limitation for online shopping in terms of the availability of different stimuli to represent physical products. The outcomes of the initial psychophysical experimental studies were then used to produce various absolute stimuli that were used in a comparative experimental study to evaluate user experience of haptic product evaluation. Although free haptic exploration was exercised on both psychophysical experiments, results were relatively consistent with previous work on haptic discrimination. The threshold for weight force discrimination represented as downward forces was 10 percent. The threshold for texture force discrimination represented as friction forces was 14.1 percent, when using dynamic coefficient of friction at any level of static coefficient of friction. On the other hand, the comparative experimental study to evaluate user experience of haptic product information indicated that haptic product evaluation does not change user performance significantly. However, although there was an increase in the time taken to complete the task, the number of button click actions tended to decrease. The results showed that haptic product evaluation could significantly increase the confidence of shopping decision. Nevertheless, the availability of haptic product evaluation does not necessarily impose different product choices but it complements other selection criteria such as price and appearance. The research findings from this work are a first step towards exploring haptic-based environments in e-commerce environments. The findings not only lay the foundation for designing online haptic shopping but also provide empirical support to research in this direction

A two-stage framework for designing visual analytics systems to augment organizational analytical processes

A perennially interesting research topic in the field of visual analytics is how to effectively develop systems that support organizational knowledge worker’s decision-making and reasoning processes. The primary objective of a visual analytic system is to facilitate analytical reasoning and discovery of insights through interactive visual interfaces. It also enables the transfer of capability and expertise from where it resides to where it is needed–across individuals, and organizations as necessary. The problem is, however, most domain analytical practices generally vary from organizations to organizations. This leads to the diversified design of visual analytics systems in incorporating domain analytical processes, making it difficult to generalize the success from one domain to another. Exacerbating this problem is the dearth of general models of analytical workflows available to enable such timely and effective designs. To alleviate these problems, this dissertation presents a two-stage framework for informing the design of a visual analytics system. This two-stage design framework builds upon and extends current practices pertaining to analytical workflow and focuses, in particular, on investigating its effect on the design of visual analytics systems for organizational environments. It aims to empower organizations with more systematic and purposeful information analyses through modeling the domain users’ reasoning processes. The first stage in this framework is an Observation and Designing stage, in which a visual analytic system is designed and implemented to abstract and encapsulate general organizational analytical processes, through extensive collaboration with domain users. The second stage is the User-centric Refinement stage, which aims at interactively enriching and refining the already encapsulated domain analysis process based on understanding user’s intentions through analyzing their task behavior. To implement this framework in the process of designing a visual analytics system, this dissertation proposes four general design recommendations that, when followed, empower such systems to bring the users closer to the center of their analytical processes. This dissertation makes three primary contributions: first, it presents a general characterization of the analytical workflow in organizational environments. This characterization fills in the blank of the current lack of such an analytical model and further represents a set of domain analytical tasks that are commonly applicable to various organizations. Secondly, this dissertation describes a two-stage framework for facilitating the domain users’ workflows through integrating their analytical models into interactive visual analytics systems. Finally, this dissertation presents recommendations and suggestions on enriching and refining domain analysis through capturing and analyzing knowledge workers’ analysis processes. To exemplify the generalizability of these design recommendations, this dissertation presents three visual analytics systems that are developed following the proposed recommendations, including Taste for Xerox Corporation, OpsVis for Microsoft, and IRSV for the U.S. Department of Transportation. All of these systems are deployed to domain knowledge workers and are adopted for their analytical practices. Extensive empirical evaluations are further conducted to demonstrate efficacy of these systems in facilitating domain analytical processes