Analytic Methods for Optimizing Realtime Crowdsourcing

Realtime crowdsourcing research has demonstrated that it is possible to recruit paid crowds within seconds by managing a small, fast-reacting worker pool. Realtime crowds enable crowd-powered systems that respond at interactive speeds: for example, cameras, robots and instant opinion polls. So far, these techniques have mainly been proof-of-concept prototypes: research has not yet attempted to understand how they might work at large scale or optimize their cost/performance trade-offs. In this paper, we use queueing theory to analyze the retainer model for realtime crowdsourcing, in particular its expected wait time and cost to requesters. We provide an algorithm that allows requesters to minimize their cost subject to performance requirements. We then propose and analyze three techniques to improve performance: push notifications, shared retainer pools, and precruitment, which involves recalling retainer workers before a task actually arrives. An experimental validation finds that precruited workers begin a task 500 milliseconds after it is posted, delivering results below the one-second cognitive threshold for an end-user to stay in flow.Comment: Presented at Collective Intelligence conference, 201

Heparinase Immobilization Characterization and Optimization a

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/73755/1/j.1749-6632.1988.tb25880.x.pd

Crowds in two seconds: Enabling realtime crowd-powered interfaces

Interactive systems must respond to user input within seconds. Therefore, to create realtime crowd-powered interfaces, we need to dramatically lower crowd latency. In this paper, we introduce the use of synchronous crowds for on-demand, realtime crowdsourcing. With synchronous crowds, systems can dynamically adapt tasks by leveraging the fact that workers are present at the same time. We develop techniques that recruit synchronous crowds in two seconds and use them to execute complex search tasks in ten seconds. The first technique, the retainer model, pays workers a small wage to wait and respond quickly when asked. We offer empirically derived guidelines for a retainer system that is low-cost and produces on-demand crowds in two seconds. Our second technique, rapid refinement, observes early signs of agreement in synchronous crowds and dynamically narrows the search space to focus on promising directions. This approach produces results that, on average, are of more reliable quality and arrive faster than the fastest crowd member working alone. To explore benefits and limitations of these techniques for interaction, we present three applications: Adrenaline, a crowd-powered camera where workers quickly filter a short video down to the best single moment for a photo; and Puppeteer and A|B, which examine creative generation tasks, communication with workers, and low-latency voting

Alcohol, Tobacco, and Other Drugs: Future Directions for Screening and Intervention in the Emergency Department

This article is a product of a breakout session on injury prevention from the 2009 Academic Emergency Medicine consensus conference on “Public Health in the ED: Screening, Surveillance, and Intervention.” The emergency department (ED) is an important entry portal into the medical care system. Given the epidemiology of substance use among ED patients, the delivery of effective brief interventions (BIs) for alcohol, drug, and tobacco use in the ED has the potential to have a large public health impact. To date, the results of randomized controlled trials of interventional studies in the ED setting for substance use have been mixed in regard to alcohol and understudied in the area of tobacco and other drugs. As a result, there are more questions remaining than answered. The work group developed the following research recommendations that are essential for the field of screening and BI for alcohol, tobacco, and other drugs in the ED. 1) Screening—develop and validate brief and practical screening instruments for ED patients and determine the optimal method for the administration of screening instruments. 2) Key components and delivery methods for intervention—conduct research on the effectiveness of screening, brief intervention, and referral to treatment (SBIRT) in the ED on outcomes (e.g., consumption, associated risk behaviors, and medical psychosocial consequences) including minimum dose needed, key components, optimal delivery method, interventions focused on multiple risk behaviors and tailored based on assessment, and strategies for addressing polysubstance use. 3) Effectiveness among patient subgroups—conduct research to determine which patients are most likely to benefit from a BI for substance use, including research on moderators and mediators of intervention effectiveness, and examine special populations using culturally and developmentally appropriate interventions. 4) Referral strategies—a) promote prospective effectiveness trials to test best strategies to facilitate referrals and access from the ED to preventive services, community resources, and substance abuse and mental health treatment; b) examine impact of available community services; c) examine the role of stigma of referral and follow-up; and d) examine alternatives to specialized treatment referral. 5) Translation—conduct translational and cost-effectiveness research of proven efficacious interventions, with attention to fidelity, to move ED SBIRT from research to practice.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/78664/1/j.1553-2712.2009.00552.x.pd

Tweets as data: Demonstration of TweeQL and TwitInfo

Microblogs such as Twitter are a tremendous repository of user-generated content. Increasingly, we see tweets used as data sources for novel applications such as disaster mapping, brand sentiment analysis, and real-time visualizations. In each scenario, the workflow for processing tweets is ad-hoc, and a lot of unnecessary work goes into repeating common data processing patterns. We introduce TweeQL, a stream query processing language that presents a SQL-like query interface for unstructured tweets to generate structured data for downstream applications. We have built several tools on top of TweeQL, most notably TwitInfo, an event timeline generation and exploration interface that summarizes events as they are discussed on Twitter. Our demonstration will allow the audience to interact with both TweeQL and TwitInfo to convey the value of data embedded in tweets

Crowdsourcing and Human Computation: Systems, Studies and Platforms

Crowdsourcing and human computation are transforming human-computer interaction, and CHI has led the way. The seminal publication in human computation was initially published in CHI in 2004 [1], and the first paper investigating Mechanical Turk as a user study platform has amassed over one hundred citations in two years [5]. However, we are just beginning to stake out a coherent research agenda for the field. This workshop will bring together researchers in the young field of crowdsourcing and human computation and produce three artifacts: a research agenda for the field, a vision for ideal crowdsourcing platforms, and a group-edited bibliography. These resources will be publically disseminated on the web and evolved and maintained by the community

Processing and visualizing the data in tweets

Microblogs such as Twitter provide a valuable stream of diverse user-generated data. While the data extracted from Twitter is generally timely and accurate, the process by which developers extract structured data from the tweet stream is ad-hoc and requires reimplementation of common data manipulation primitives. In this paper, we present two systems for querying and extracting structure from Twitter-embedded data. The first, TweeQL, provides a streaming SQL-like interface to the Twitter API, making common tweet processing tasks simpler. The second, TwitInfo, shows how end-users can interact with and understand aggregated data from the tweet stream, in addition to showcasing the power of the TweeQL language. Together these systems show the richness of content that can be extracted from Twitter

TwitInfo: Aggregating and Visualizing Microblogs for Event Exploration

Microblogs are a tremendous repository of user-generated content about world events. However, for people trying to understand events by querying services like Twitter, a chronological log of posts makes it very difficult to get a detailed understanding of an event. In this paper, we present TwitInfo, a system for visualizing and summarizing events on Twitter. TwitInfo allows users to browse a large collection of tweets using a timeline-based display that highlights peaks of high tweet activity. A novel streaming algorithm automatically discovers these peaks and labels them meaningfully using text from the tweets. Users can drill down to subevents, and explore further via geolocation, sentiment, and popular URLs. We contribute a recall-normalized aggregate sentiment visualization to produce more honest sentiment overviews. An evaluation of the system revealed that users were able to reconstruct meaningful summaries of events in a small amount of time. An interview with a Pulitzer Prize-winning journalist suggested that the system would be especially useful for understanding a long-running event and for identifying eyewitnesses. Quantitatively, our system can identify 80-100% of manually labeled peaks, facilitating a relatively complete view of each event studied

Expansion, Geometry, and Gravity

In general-relativistic cosmological models, the expansion history, matter content, and geometry are closely intertwined. In this brief paper, we clarify the distinction between the effects of geometry and expansion history on the luminosity distance. We show that the cubic correction to the Hubble law, measured recently with high-redshift supernovae, is the first cosmological measurement, apart from the cosmic microwave background, that probes directly the effects of spatial curvature. We illustrate the distinction between geometry and expansion with a toy model for which the supernova results already indicate a curvature radius larger than the Hubble distance.Comment: 4 pages, 1 color figur