Leveraging Mixed Expertise in Crowdsourcing.

Crowdsourcing systems promise to leverage the "wisdom of crowds" to help solve many kinds of problems that are difficult to solve using only computers. Although a crowd of people inherently represents a diversity of skill levels, knowledge, and opinions, crowdsourcing system designers typically view this diversity as noise and effectively cancel it out by aggregating responses. However, we believe that by embracing crowd workers' diverse expertise levels, system designers can better leverage that knowledge to increase the wisdom of crowds. In this thesis, we propose solutions to a limitation of current crowdsourcing approaches: not accounting for a range of expertise levels in the crowd. The current body of work in crowdsourcing does not systematically examine this, suggesting that researchers may not believe the benefits of using mixed expertise warrants the complexities of supporting it. This thesis presents two systems, Escalier and Kurator, to show that leveraging mixed expertise is a worthwhile endeavor because it materially benefits system performance, at scale, for various types of problems. We also demonstrate an effective technique, called expertise layering, to incorporate mixed expertise into crowdsourcing systems. Finally, we show that leveraging mixed expertise enables researchers to use crowdsourcing to address new types of problems.PhDComputer Science and EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/133307/1/afdavid_1.pd

Spartan Daily, December 9, 1981

Volume 77, Issue 67https://scholarworks.sjsu.edu/spartandaily/6843/thumbnail.jp

Xavier University Newswire

https://www.exhibit.xavier.edu/student_newspaper/1472/thumbnail.jp

How Students Access, Filter and Evaluate Digital News: Choices That Shape What They Consume and the Implications for News Literacy Education

Being an informed citizen in the digital age requires the ability to sift through an avalanche of news online and identify content that is credible and diverse. News literacy, a topic with a small but increasing presence in high school and college curricula, is concerned with training students to be discerning news consumers. Assessments of news literacy typically gauge the effects of exposure to news literacy curricula measured through student analysis of media messages selected by researchers. This exploratory, mixed-methods study instead examined how students with no formal news literacy instruction searched for news on a computer using their typical routine, their process of filtering and evaluating news about a topic of interest, and their awareness of their choices when accessing news online that shape what they consume. This study contributes to the understanding of what digital media concepts, cognitive strategies and evaluation criteria warrant targeting or greater emphasis in news literacy curricula. Survey results revealed that participants (n=244) typically spend a significant amount of time consuming video and written news, largely through digital platforms and mostly on a computer. They are mostly information scanners and more often stumble upon news online than seek out specific news of interest. Participants have a strong social interest in news, like to share stories with others, and are often trusting of their social networks and technology to filter the news they consume. Concurrent think-aloud protocols and subsequent interviews with a subset of survey respondents (n=37) found that participants often did not pay close attention to the process by which they accessed and filtered news online, doing so in a state of automaticity instead of thinking critically. When asked to explain the thought processes underlying their news searches, a significant percentage of students lacked a conscious awareness or understanding of the strategies and evaluation criteria that potentially affect the credibility and diversity of news consumed. As a result, students' online news habits often placed them at risk for consuming unreliable news and for adopting a hive mindset or being in a news silo

The BG News January 30, 1979

The BGSU campus student newspaper January 30, 1979. Volume 62 - Issue 166https://scholarworks.bgsu.edu/bg-news/4571/thumbnail.jp

Courting Abolition

This Review of Courting Death offers a different take on two of Professor Carol Steiker and Professor Jordan Steiker\u27s major themes: (1) the tension between effecting meaningful reform and legitimatizing legal façades, and (2) the future of the American death penalty. The Review argues several points, one being that the Model Penal Code may have had a larger pre-Furman impact than the Steikers acknowledge. In addition, the Review expands on some key contributors to the death penalty’s decline that may have been obscured by the all-encompassing nature of the Steikers’ regulation argument — for example, the emergence of unforeseeable exogenous variables (similar to the introduction of DNA evidence into criminal trials in the 1980s), as well as pressure points that exist largely outside of the constitutional regulatory framework, such as lethal injection litigation. Despite these influences, the Review finds the Steikers’ prediction — that, when abolition seems right, it will come by way of a “Furman II” Supreme Court decision — to readily comport with the death penalty’s trajectory over the last fifty years

The BG News February 20, 1979

The BGSU campus student newspaper February 20, 1979. Volume 62 - Issue 179https://scholarworks.bgsu.edu/bg-news/4583/thumbnail.jp

The New Hampshire, Vol. 68, No. 10 (Oct. 11, 1977)

The student publication of the University of New Hampshire

Crowd-powered systems

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 217-237).Crowd-powered systems combine computation with human intelligence, drawn from large groups of people connecting and coordinating online. These hybrid systems enable applications and experiences that neither crowds nor computation could support alone. Unfortunately, crowd work is error-prone and slow, making it difficult to incorporate crowds as first-order building blocks in software systems. I introduce computational techniques that decompose complex tasks into simpler, verifiable steps to improve quality, and optimize work to return results in seconds. These techniques develop crowdsourcing as a platform so that it is reliable and responsive enough to be used in interactive systems. This thesis develops these ideas through a series of crowd-powered systems. The first, Soylent, is a word processor that uses paid micro-contributions to aid writing tasks such as text shortening and proofreading. Using Soylent is like having access to an entire editorial staff as you write. The second system, Adrenaline, is a camera that uses crowds to help amateur photographers capture the exact right moment for a photo. It finds the best smile and catches subjects in mid-air jumps, all in realtime. Moving beyond generic knowledge and paid crowds, I introduce techniques to motivate a social network that has specific expertise, and techniques to data mine crowd activity traces in support of a large number of uncommon user goals. These systems point to a future where social and crowd intelligence are central elements of interaction, software, and computation.by Michael Scott Bernstein.Ph.D