Prototype tasks: Improving crowdsourcing results through rapid, iterative task design

Low-quality results have been a long-standing problem on microtask crowdsourcing platforms, driving away requesters and justifying low wages for workers. To date, workers have been blamed for low-quality results: they are said to make as little effort as possible, do not pay attention to detail, and lack expertise. In this paper, we hypothesize that requesters may also be responsible for low-quality work: they launch unclear task designs that confuse even earnest workers, under-specify edge cases, and neglect to include examples. We introduce prototype tasks, a crowdsourcing strategy requiring all new task designs to launch a small number of sample tasks. Workers attempt these tasks and leave feedback, enabling the requester to iterate on the design before publishing it. We report a field experiment in which tasks that underwent prototype task iteration produced higher-quality work results than the original task designs. With this research, we suggest that a simple and rapid iteration cycle can improve crowd work, and we provide empirical evidence that requester “quality” directly impacts result quality

Embedding a Crowd inside a Relay Baton:A Case Study in a Non-Competitive Sporting Activity

This paper presents a digital relay baton that connects long-distance runners with distributed online spectators. Such baton broadcasts athletes’ live locative data to a social network and communicates back remote-crowd support through haptic and audible cheers. Our work takes an exploratory design approach to bring new insights into the design of real-time techno-mediated social support. The prototype was deployed during a 170-mile charity relay race across the UK with 13 participants, 261 on-line supporters, and collected a total of 3153 ‘cheers’. We report on the insights collected during the design and deployment process and identify three fundamental design considerations: the degree of expressiveness afforded by the system design, the context applicability, and the data flow within the social networ

Embedding a Crowd inside a Relay Baton:A Case Study in a Non-Competitive Sporting Activity

This paper presents a digital relay baton that connects long-distance runners with distributed online spectators. Such baton broadcasts athletes’ live locative data to a social network and communicates back remote-crowd support through haptic and audible cheers. Our work takes an exploratory design approach to bring new insights into the design of real-time techno-mediated social support. The prototype was deployed during a 170-mile charity relay race across the UK with 13 participants, 261 on-line supporters, and collected a total of 3153 ‘cheers’. We report on the insights collected during the design and deployment process and identify three fundamental design considerations: the degree of expressiveness afforded by the system design, the context applicability, and the data flow within the social networ

Measuring text simplification with the crowd

Text can often be complex and difficult to read, especially for peo ple with cognitive impairments or low literacy skills. Text simplifi cation is a process that reduces the complexity of both wording and structure in a sentence, while retaining its meaning. However, this is currently a challenging task for machines, and thus, providing effective on-demand text simplification to those who need it re mains an unsolved problem. Even evaluating the simplicity of text remains a challenging problem for both computers, which cannot understand the meaning of text, and humans, who often struggle to agree on what constitutes a good simplification. This paper focuses on the evaluation of English text simplifica tion using the crowd. We show that leveraging crowds can result in a collective decision that is accurate and converges to a consen sus rating. Our results from 2,500 crowd annotations show that the crowd can effectively rate levels of simplicity. This may allow sim plification systems and system builders to get better feedback about how well content is being simplified, as compared to standard mea sures which classify content into ‘simplified ’ or ‘not simplified’ categories. Our study provides evidence that the crowd could be used to evaluate English text simplification, as well as to create simplified text in future work

Integration of accessibility requirements in the design of multimedia user agents interfaces

Mención Internacional en el título de doctorThe continuous increase of multimedia content in the Web, especially video content, is not accompanied by a similar increase of accessibility; there is a lack of synchronized alternatives for the content such as captions, audio description, etc. that allow anyone with or without disability to access such content. This lack of accessibility in video content access is not only due to the lack of alternatives, but also because of the fact that user agents which deliver this content do not provide the necessary means to present them. This fact leads to the noncompliance of the current regulations and legislation in terms of accessibility. This noncompliance could be due to the lack of knowledge, or because of the fact that applying these regulations from an engineering point of view is not trivial. There is a lack of authoring tools and methodological approaches which assist in the development of an accessible product in the Engineering scope as it is the case of the development of a quality user agent which includes accessibility requirements. All these facts, multimedia content’s progressive increase on the Web, accessibility barriers both in the content and in the user agent together with current regulations and legislation regarding accessibility is what has motivated the accomplishment of this Doctoral Thesis. With this Doctoral Thesis, a set of accessibility requirements that a user agent which delivers multimedia content must fulfil is provided. Besides, a workspace is provided following a methodological approach which assists in the design and development of the interface of an accessible user agent which delivers accessible multimedia content. This workspace is composed of an architecture and models following a Model-Based User Interface Development (MBUID) approach and is oriented to be used by designers with knowledge in modeling. Finally, as a support to any professional regardless of their knowledge in modeling and in accessibility, an authoring tool based on models is offered in order to create user agents with accessibility requirements.El continuo incremento del contenido multimedia en la Web, especialmente del contenido vídeo, no va acompañado de un incremento similar de accesibilidad, hay una falta de alternativas sincronizadas al contenido como subtitulado, audiodescripción, etc., que permitan acceder a cualquier persona con y sin discapacidad a dicho contenido. Esta falta de accesibilidad en el acceso al contenido vídeo no solo se debe a la ausencia de alternativas, también es debido a que los agentes de usuario que entregan dicho contenido no proporcionan los medios necesarios para presentarlas. Este hecho da lugar a que no se cumpla la normativa y la legislación vigente en materia de accesibilidad. Dicho incumplimiento, puede ser debido al desconocimiento, o a que aplicar esa normativa desde el punto de vista de la ingeniería no es trivial. Hay una falta de herramientas de autor y de enfoques metodológicos que asistan en el desarrollo de un producto accesible en el ámbito de la Ingeniería, como es el caso del desarrollo de un agente de usuario con calidad que incluya requisitos de accesibilidad. Todos estos hechos, el incremento progresivo del contenido multimedia en la Web, las barreras de accesibilidad tanto en el contenido como en el agente de usuario junto con la normativa y legislación vigente en materia de accesibilidad es lo que ha motivado la realización de esta Tesis Doctoral. Con esta Tesis Doctoral se proporciona el conjunto de requisitos de accesibilidad que debe cumplir un agente de usuario que sirva contenido multimedia accesible. Además se proporciona un espacio de trabajo siguiendo un enfoque metodológico que asista en el diseño y desarrollo de la interfaz de un agente de usuario accesible que sirve contenido multimedia accesible. Este espacio de trabajo está compuesto de una arquitectura y modelos siguiendo el enfoque de Model-Based User Interface Development (MBUID) y está orientado a ser utilizado por diseñadores con conocimientos en modelado. Por último, como recurso de ayuda a cualquier profesional, independientemente de sus conocimientos en modelado y accesibilidad, se ofrece una herramienta de autor basada en modelos para crear agentes de usuario con requisitos de accesibilidad.Programa Oficial de Doctorado en Ciencia y Tecnología InformáticaPresidente: José Antonio Macías Iglesias.- Vocal: Hugo Alexandre Paredes Guede

Textile Society of America Newsletter 28:1 — Spring 2016

Letter from the Editor Volunteer Opportunity: TSA Is Looking for a New Proceedings Editor Letter from the President Textiles Close Up Report: Art of the Zo: Textiles from Myanmar, India, and Bangladesh, Chin Weaving at the Philadelphia Museum of Art R. L. Shep Ethnic Textile Book Award 2015 Nominees Ossabaw Island, Indigo, and Sea Island Cotton: Two Ways to See a Georgia Barrier Island Peer-Review Process Yields Range of Exciting Exhibitions for Biennial Symposium Book Reviews:Symbols of Power: Luxury Items from Islamic Lands, 7th–21st CenturyTextiles of the Banjara: Cloth and Culture of a Wandering TribeThe Handbook of Textile CultureTraditional Weavers of Guatemala: Their Stories, Their LivesDesigning Identity: The Power of Textiles in Late Antiquity Conference Review: 21st Annual Weaving History Conference, 2015 Featured Exhibitions:Heirlooms, Catastrophe, and Survival: The Lace and Sampler Collection of the Palazzo DavanzatiThe Fabric of India, Victoria and Albert Museum, LondonFashion Meets Technology in #techstyle, Museum of Fine Arts, Boston Shibori and Ikat in Mesoamerica International Report:The Centre for Textile Conservation at the University of Glasgow & a New Era for Textile Dye Research in ScotlandThe Philippine Textile Research Institute, Taguig City, Philippines Member News:Member PublicationsMember Workshops and LecturesMember Awards & HonorsMember Exhibitions Conferences & Opportunities Symposium Program [Crosscurrents: Land, Labor, and the Port, 15th Biennial Symposium, Savannah, Georgia, October 19–23, 2016