Balancing Human and Machine Contributions in Human Computation Systems

Many interesting and successful human computation systems leverage the complementary computational strengths of both humans and machines to solve these problems. In this chapter, we examine Human Computation as a type of Human-Computer Collaboration—collaboration involving at least one human and at least one computational agent. We discuss recent advances in the open area of function allocation, and explore how to balance the contributions of humans and machines in computational systems. We then explore how human-computer collaborative strategies can be used to solve problems that are difficult or computationally infeasible for computers or humans alone

Inferring Intent from Interaction with Visualization

Today\u27s state-of-the-art analysis tools combine the human visual system and domain knowledge, with the machine\u27s computational power. The human performs the reasoning, deduction, hypothesis generation, and judgment. The entire burden of learning from the data usually rests squarely on the human user\u27s shoulders. This model, while successful in simple scenarios, is neither scalable nor generalizable. In this thesis, we propose a system that integrates advancements from artificial intelligence within a visualization system to detect the user\u27s goals. At a high level, we use hidden unobservable states to represent goals/intentions of users. We automatically infer these goals from passive observations of the user\u27s actions (e.g., mouse clicks), thereby allowing accurate predictions of future clicks. We evaluate this technique with a crime map and demonstrate that, depending on the type of task, users\u27 clicks appear in our prediction set 79\% -- 97\% of the time. Further analysis shows that we can achieve high prediction accuracy after only a short period (typically after three clicks). Altogether, we show that passive observations of interaction data can reveal valuable information about users\u27 high-level goals, laying the foundation for next-generation visual analytics systems that can automatically learn users\u27 intentions and support the analysis process proactively

Bayesian Quadrature with Prior Information: Modeling and Policies

Quadrature is the problem of estimating intractable integrals. Such integrals regularly arise in engineering and the natural sciences, especially when Bayesian methods are applied; examples include model evidences, normalizing constants and marginal distributions. This dissertation explores Bayesian quadrature, a probabilistic, model-based quadrature method. Specifically, we study different ways in which Bayesian quadrature can be adapted to account for different kinds of prior information one may have about the task. We demonstrate that by taking into account prior knowledge, Bayesian quadrature can outperform commonly used numerical methods that are agnostic to prior knowledge, such as Monte Carlo based integration. We focus on two types of information that are (a) frequently available when faced with an intractable integral and (b) can be (approximately) incorporated into Bayesian quadrature: • Natural bounds on the possible values that the integrand can take, e.g., when the integrand is a probability density function, it must nonnegative everywhere.• Knowledge about how the integral estimate will be used, i.e., for settings where quadrature is a subroutine, different downstream inference tasks can result in different priorities or desiderata for the estimate. These types of prior information are used to inform two aspects of the Bayesian quadrature inference routine: • Modeling: how the belief on the integrand can be tailored to account for the additional information.• Policies: where the integrand will be observed given a constrained budget of observations. This second aspect of Bayesian quadrature, policies for deciding where to observe the integrand, can be framed as an experimental design problem, where an agent must choose locations to evaluate a function of interest so as to maximize some notion of value. We will study the broader area of sequential experimental design, applying ideas from Bayesian decision theory to develop an efficient and nonmyopic policy for general sequential experimental design problems. We consider other sequential experimental design tasks such as Bayesian optimization and active search; in the latter, we focus on facilitating human–computer partnerships with the goal of aiding human agents engaged in data foraging through the use of active search based suggestions and an interactive visual interface. Finally, this dissertation will return to Bayesian quadrature and discuss the batch setting for experimental design, where multiple observations of the function in question are made simultaneously