Racial Violence Archive: Public Information System on Incidents of Violence during the Civil Rights Period

We designed a social media interface that allows both scholars and the general public to contribute to an archive on incidents of violence in the South during the Civil Rights Movement. The archive currently contains more than 2,000 entries based on memos/reports generated by state and city administrations, county and court records, news articles, etc. To make this archive more accessible and plentiful, we prototyped a novel web-based system enabling the public to view and search information about events of racial violence in interactive ways (e.g., dynamically filter out incidents using a time slider on the map), and to contribute new event information or add information to existing records, such as related photographs, documents, or details. The usability test showed that our system is well designed to make historical information freely available for use without any restrictions.ye

A Comparative Study of the Role of Examples in Microtask Crowdsourcing for Software Design

Crowdsourcing is gradually becoming an accepted form of work across different disciplines. Not surprisingly, it has attracted the attention of the software engineering community as well. Previous work started exploring the feasibility of crowdsourcing for software design by conducting experiments in which workers from Amazon Mechanical Turk were asked to engage in a set of software design tasks. It was found that, when workers are exposed to examples of previous designs, they generate overall lower quality contributions. The intuition is that, since these experiments displayed all previous contributions as examples to workers, the presence of low quality examples may have negatively influenced workers.This thesis compares the designs produced in the previous experiments to designs obtained in a new experiment in which examples were evaluated against pre-defined quality criteria before being displayed to workers. Only examples that were of sufficient quality were shared with workers, with the hope of stimulating them to provide higher quality designs. We report results from an analysis in which we compare the designs from the current and previous experiments in terms of quantity, diversity of ideas, quality, completeness, perceived task difficulty, and how often workers borrow elements from examples. The major findings are twofold. First, workers who were exposed to sufficient quality examples produced better quality work as compared to workers exposed to all examples. Second, the quality of the designs they produced still did not reach the quality of the designs produced by workers who were not exposed to examples at all

A Comparative Study of the Role of Examples in Microtask Crowdsourcing for Software Design

Crowdsourcing is gradually becoming an accepted form of work across different disciplines. Not surprisingly, it has attracted the attention of the software engineering community as well. Previous work started exploring the feasibility of crowdsourcing for software design by conducting experiments in which workers from Amazon Mechanical Turk were asked to engage in a set of software design tasks. It was found that, when workers are exposed to examples of previous designs, they generate overall lower quality contributions. The intuition is that, since these experiments displayed all previous contributions as examples to workers, the presence of low quality examples may have negatively influenced workers.This thesis compares the designs produced in the previous experiments to designs obtained in a new experiment in which examples were evaluated against pre-defined quality criteria before being displayed to workers. Only examples that were of sufficient quality were shared with workers, with the hope of stimulating them to provide higher quality designs. We report results from an analysis in which we compare the designs from the current and previous experiments in terms of quantity, diversity of ideas, quality, completeness, perceived task difficulty, and how often workers borrow elements from examples. The major findings are twofold. First, workers who were exposed to sufficient quality examples produced better quality work as compared to workers exposed to all examples. Second, the quality of the designs they produced still did not reach the quality of the designs produced by workers who were not exposed to examples at all