Globally Optimal Crowdsourcing Quality Management

We study crowdsourcing quality management, that is, given worker responses to a set of tasks, our goal is to jointly estimate the true answers for the tasks, as well as the quality of the workers. Prior work on this problem relies primarily on applying Expectation-Maximization (EM) on the underlying maximum likelihood problem to estimate true answers as well as worker quality. Unfortunately, EM only provides a locally optimal solution rather than a globally optimal one. Other solutions to the problem (that do not leverage EM) fail to provide global optimality guarantees as well. In this paper, we focus on filtering, where tasks require the evaluation of a yes/no predicate, and rating, where tasks elicit integer scores from a finite domain. We design algorithms for finding the global optimal estimates of correct task answers and worker quality for the underlying maximum likelihood problem, and characterize the complexity of these algorithms. Our algorithms conceptually consider all mappings from tasks to true answers (typically a very large number), leveraging two key ideas to reduce, by several orders of magnitude, the number of mappings under consideration, while preserving optimality. We also demonstrate that these algorithms often find more accurate estimates than EM-based algorithms. This paper makes an important contribution towards understanding the inherent complexity of globally optimal crowdsourcing quality management

Effects of errorless learning on velopharyngeal movement control

abstrac

Leveraging Affect Transfer Learning for Behavior Prediction in an Intelligent Tutoring System

In the context of building an intelligent tutoring system (ITS), which improves student learning outcomes by intervention, we set out to improve prediction of student problem outcome. In essence, we want to predict the outcome of a student answering a problem in an ITS from a video feed by analyzing their face and gestures. For this, we present a novel transfer learning facial affect representation and a user-personalized training scheme that unlocks the potential of this representation. We model the temporal structure of video sequences of students solving math problems using a recurrent neural network architecture. Additionally, we extend the largest dataset of student interactions with an intelligent online math tutor by a factor of two. Our final model, coined ATL-BP (Affect Transfer Learning for Behavior Prediction) achieves an increase in mean F-score over state-of-the-art of 45% on this new dataset in the general case and 50% in a more challenging leave-users-out experimental setting when we use a user-personalized training scheme

Improving a gold standard: treating human relevance judgments of MEDLINE document pairs

Given prior human judgments of the condition of an object it is possible to use these judgments to make a maximal likelihood estimate of what future human judgments of the condition of that object will be. However, if one has a reasonably large collection of similar objects and the prior human judgments of a number of judges regarding the condition of each object in the collection, then it is possible to make predictions of future human judgments for the whole collection that are superior to the simple maximal likelihood estimate for each object in isolation. This is possible because the multiple judgments over the collection allow an analysis to determine the relative value of a judge as compared with the other judges in the group and this value can be used to augment or diminish a particular judge’s influence in predicting future judgments. Here we study and compare five different methods for making such improved predictions and show that each is superior to simple maximal likelihood estimates

Affective Man-Machine Interface: Unveiling human emotions through biosignals

As is known for centuries, humans exhibit an electrical profile. This profile is altered through various psychological and physiological processes, which can be measured through biosignals; e.g., electromyography (EMG) and electrodermal activity (EDA). These biosignals can reveal our emotions and, as such, can serve as an advanced man-machine interface (MMI) for empathic consumer products. However, such a MMI requires the correct classification of biosignals to emotion classes. This chapter starts with an introduction on biosignals for emotion detection. Next, a state-of-the-art review is presented on automatic emotion classification. Moreover, guidelines are presented for affective MMI. Subsequently, a research is presented that explores the use of EDA and three facial EMG signals to determine neutral, positive, negative, and mixed emotions, using recordings of 21 people. A range of techniques is tested, which resulted in a generic framework for automated emotion classification with up to 61.31% correct classification of the four emotion classes, without the need of personal profiles. Among various other directives for future research, the results emphasize the need for parallel processing of multiple biosignals

Mapping the beach beneath the street:digital cartography for the playable city

Maps are an important component within many of the playful and gameful experiences designed to turn cities into a playable infrastructures. They take advantage of the fact that the technology used for obtaining accurate spatial information, such as GPS receivers and magnetometers (digital compasses), are now so wide-spread that they are considered as ‘standard’ sensors on mobile phones, which are themselves ubiquitous. Interactive digital maps, therefore, are are widely used by the general public for a variety of purposes. However, despite the rich design history of cartography digital maps typically exhibit a dominant aesthetic that has been de-signed to serve the usability and utility requirements of turn-by-turn urban navigation, which is itself driven by the proliferation of in-car and personal navigation services. The navigation aesthetic is now widespread across almost all spatial applications, even where a be-spoke cartographic product would be better suited. In this chapter we seek to challenge this by exploring novel neo-cartographic ap-proaches to making maps for use within playful and gameful experi-ences designed for the cities. We will examine the potential of de-sign approaches that can producte not only more aesthetically pleasing maps, but also offer the potential for influencing user be-haviour, which can be used to promote emotional engagement and exploration in playable city experiences

Effect of intonation on Cantonese lexical tones

In tonal languages, there are potential conflicts between the F0-based changes due to the coexistence of intonation and lexical tones. In the present study, the interaction of tone and intonation in Cantonese was examined using acoustic and perceptual analyses. The acoustic patterns of tones at the initial, medial, and final positions of questions and statements were measured. Results showed that intonation affects both the F0 level and contour, while the duration of the six tones varied as a function of positions within intonation contexts. All six tones at the final position of questions showed rising F0 contour, regardless of their canonical form. Listeners were overall more accurate in the identification of tones presented within the original carrier than of the same tones in isolation. However, a large proportion of tones 33, 21, 23, and 22 at the final position of questions were misperceived as tone 25 both within the original carrier and as isolated words. These results suggest that although the intonation context provided cues for correct tone identification, the intonation-induced changes in F0 contour cannot always be perceptually compensated for, resulting in some erroneous perception of the identity of Cantonese tone. © 2006 Acoustical Society of America.published_or_final_versio

An odd oxygen framework for wintertime ammonium nitrate aerosol pollution in urban areas: NOx and VOC control as mitigation strategies

Wintertime ammonium nitrate aerosol pollution is a severe air quality issue affecting both developed and rapidly urbanizing regions from Europe to East Asia. In the US, it is acute in western basins subject to inversions that confine pollutants near the surface. Measurements and modeling of a wintertime pollution episode in Salt Lake City, Utah demonstrates that ammonium nitrate is closely related to photochemical ozone through a common parameter, total odd oxygen, Ox,total. We show that the traditional NOx‐VOC framework for evaluating ozone mitigation strategies also applies to ammonium nitrate. Despite being nitrate‐limited, ammonium nitrate aerosol pollution in Salt Lake City is responsive to VOC control and, counterintuitively, not initially responsive to NOx control. We demonstrate simultaneous nitrate limitation and NOx saturation and suggest this phenomenon may be general. This finding may identify an unrecognized control strategy to address a global public health issue in regions with severe winter aerosol pollution