Quantifying diversity in user experience

Evaluation should be integral to any design activity. Evaluation in innovative product development practices however is highly complicated. It often needs to be applied to immature prototypes, while at the same time users’ responses may greatly vary across different individuals and situations. This thesis has focused on methods and tools for inquiring into users’ experiences with interactive products. More specifically, it had three objectives: a) to conceptualize the notion of diversity in subjective judgments of users’ experiences with interactive products, b) to establish empirical evidence for the prevalence of diversity, and c) to provide a number of methodological tools for the study of diversity in the context of product development. Two critical sources of diversity in the context of users’ experiences with interactive products were identified and respective methodological solutions were proposed: a) understanding interpersonal diversity through personal attribute judgments, and b) understanding the dynamics of experience through experience narratives. Personal Attribute Judgments, and in particular, the Repertory Grid Technique, is proposed as an alternative to standardized psychometric scales, in measuring users’ responses to artifacts in the context of parallel design. It is argued that traditional approaches that rely on the a-priori definition of the measures by the researchers have at least two limitations. First, such approaches are inherently limited as researchers might fail to consider a given dimension as relevant for the given product and context, or they might simply lack validated measurement scales for a relevant dimension. Secondly, such approaches assume that participants are able to interpret and position a given statement that is defined by the researcher to their own context. Recent literature has challenged this assumption, suggesting that in certain cases participants are unable to interpret the personal relevance of the statement in their own context, and might instead employ shallow processing, that is respond to surface features of the language rather than attaching personal relevance to the question. In contrast, personal attributes are elicited from each individual respondent, instead of being a-priori imposed by the experimenter, and thus are supposed to be highly relevant to the individual. However, personal attributes require substantially more complex quantitative analysis procedures. It is illustrated that traditional analysis procedures fail to bring out the richness of the personal attribute judgments and two new Multi-Dimensional Scaling procedures that extract multiple complementary views from such datasets are proposed. An alternative approach for the measurement of the dynamics of experience over time is proposed that relies on a) the retrospective elicitation of idiosyncratic selfreports of one’s experiences with a product, the so-called experience narratives, and b) the extraction of generalized knowledge from these narratives through computational content analysis techniques. iScale, a tool that aims at increasing users’ accuracy and effectiveness in recalling their experiences with a product is proposed. iScale uses sketching in imposing a structured process in the reconstruction of one’s experiences from memory. Two different versions of iScale, each grounded in a distinct theory of how people reconstruct emotional experiences from memory, were developed and empirically tested. A computational approach for the extraction of generalized knowledge from experience narratives, that combines traditional coding procedures with computational approaches for assessing the semantic similarity between documents, is proposed and compared with traditional content analysis. Through these two methodological contributions, this thesis argues against averaging in the subjective evaluation of interactive products. It proposes the development of interactive tools that can assist designers in moving across multiple levels of abstractions of empirical data, as design-relevant knowledge might be found on all these levels

Methods to Support the Project Selection Problem With Non-Linear Portfolio Objectives, Time Sensitive Objectives, Time Sensitive Resource Constraints, and Modeling Inadequacies

The United States Air Force relies upon information production activities to gain insight regarding uncertainties affecting important system configuration and in-mission task execution decisions. Constrained resources that prevent the fulfillment of every information production request, multiple information requestors holding different temporal-sensitive objectives, non-constant marginal value preferences, and information-product aging factors that affect the value-of-information complicate the management of these activities. This dissertation reviews project selection research related to these issues and presents novel methods to address these complications. Quantitative experimentation results demonstrate these methods’ significance

Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation

This paper surveys the current state of the art in Natural Language Generation (NLG), defined as the task of generating text or speech from non-linguistic input. A survey of NLG is timely in view of the changes that the field has undergone over the past decade or so, especially in relation to new (usually data-driven) methods, as well as new applications of NLG technology. This survey therefore aims to (a) give an up-to-date synthesis of research on the core tasks in NLG and the architectures adopted in which such tasks are organised; (b) highlight a number of relatively recent research topics that have arisen partly as a result of growing synergies between NLG and other areas of artificial intelligence; (c) draw attention to the challenges in NLG evaluation, relating them to similar challenges faced in other areas of Natural Language Processing, with an emphasis on different evaluation methods and the relationships between them.Comment: Published in Journal of AI Research (JAIR), volume 61, pp 75-170. 118 pages, 8 figures, 1 tabl

Stiffness pathologies in discrete granular systems: bifurcation, neutral equilibrium, and instability in the presence of kinematic constraints

The paper develops the stiffness relationship between the movements and forces among a system of discrete interacting grains. The approach is similar to that used in structural analysis, but the stiffness matrix of granular material is inherently non-symmetric because of the geometrics of particle interactions and of the frictional behavior of the contacts. Internal geometric constraints are imposed by the particles' shapes, in particular, by the surface curvatures of the particles at their points of contact. Moreover, the stiffness relationship is incrementally non-linear, and even small assemblies require the analysis of multiple stiffness branches, with each branch region being a pointed convex cone in displacement-space. These aspects of the particle-level stiffness relationship gives rise to three types of micro-scale failure: neutral equilibrium, bifurcation and path instability, and instability of equilibrium. These three pathologies are defined in the context of four types of displacement constraints, which can be readily analyzed with certain generalized inverses. That is, instability and non-uniqueness are investigated in the presence of kinematic constraints. Bifurcation paths can be either stable or unstable, as determined with the Hill-Bazant-Petryk criterion. Examples of simple granular systems of three, sixteen, and sixty four disks are analyzed. With each system, multiple contacts were assumed to be at the friction limit. Even with these small systems, micro-scale failure is expressed in many different forms, with some systems having hundreds of micro-scale failure modes. The examples suggest that micro-scale failure is pervasive within granular materials, with particle arrangements being in a nearly continual state of instability

INVESTIGATING EVIDENCE-BASED PRACTICES AND INTERVENTIONS USING MULTIFACETED LEARNING THEORY FOR STUDENTS IN A SPECIAL EDUCATION SELF-CONTAINED CLASSROOM

Teachers working in a special education self-contained classroom were required to implement evidence-based practices and interventions, rarely researched in a school setting, with fidelity to meet the needs of students with intellectual disabilities. Evidence-based practices and interventions for students with intellectual disabilities were researched in clinical settings with one to three student participants and without a common evaluation tool. The purpose of this qualitative case study was to use the Tennessee Educator Acceleration Model General Educator Rubric to investigate how experienced teachers used multifaceted learning theory when implementing evidence-based practices and interventions in a diverse special education self-contained classroom to help students access Tennessee state standards. Special education teachers from eight different schools across Tennessee were interviewed and observed using the Tennessee Educator Acceleration Model General Educator Rubric, which I aligned to different learning theories. I found how special education teachers planned activities, used reinforcements, and developed their knowledge of the content and their students to accommodate and modify evidence-based practices and interventions. I observed teachers in special education self-contained classrooms apply 150 (50%) behavioral learning theory strategies, 106 (36%) cognitive learning theory strategies, and 42 (14%) constructivist learning theory strategies. These findings should continue to be explored to further develop a common evaluation tool to monitor the use of multifaceted learning theory in a special education self-contained classroom instead of requiring fidelity of evidence-based practices and interventions

A Pattern Approach to Examine the Design Space of Spatiotemporal Visualization

Pattern language has been widely used in the development of visualization systems. This dissertation applies a pattern language approach to explore the design space of spatiotemporal visualization. The study provides a framework for both designers and novices to communicate, develop, evaluate, and share spatiotemporal visualization design on an abstract level. The touchstone of the work is a pattern language consisting of fifteen design patterns and four categories. In order to validate the design patterns, the researcher created two visualization systems with this framework in mind. The first system displayed the daily routine of human beings via a polygon-based visualization. The second system showed the spatiotemporal patterns of co-occurring hashtags with a spiral map, sunburst diagram, and small multiples. The evaluation results demonstrated the effectiveness of the proposed design patterns to guide design thinking and create novel visualization practices

Data-Driven Shape Analysis and Processing

Data-driven methods play an increasingly important role in discovering geometric, structural, and semantic relationships between 3D shapes in collections, and applying this analysis to support intelligent modeling, editing, and visualization of geometric data. In contrast to traditional approaches, a key feature of data-driven approaches is that they aggregate information from a collection of shapes to improve the analysis and processing of individual shapes. In addition, they are able to learn models that reason about properties and relationships of shapes without relying on hard-coded rules or explicitly programmed instructions. We provide an overview of the main concepts and components of these techniques, and discuss their application to shape classification, segmentation, matching, reconstruction, modeling and exploration, as well as scene analysis and synthesis, through reviewing the literature and relating the existing works with both qualitative and numerical comparisons. We conclude our report with ideas that can inspire future research in data-driven shape analysis and processing.Comment: 10 pages, 19 figure