Human-in-the-Loop Learning From Crowdsourcing and Social Media

Computational social studies using public social media data have become more and more popular because of the large amount of user-generated data available. The richness of social media data, coupled with noise and subjectivity, raise significant challenges for computationally studying social issues in a feasible and scalable manner. Machine learning problems are, as a result, often subjective or ambiguous when humans are involved. That is, humans solving the same problems might come to legitimate but completely different conclusions, based on their personal experiences and beliefs. When building supervised learning models, particularly when using crowdsourced training data, multiple annotations per data item are usually reduced to a single label representing ground truth. This inevitably hides a rich source of diversity and subjectivity of opinions about the labels. Label distribution learning associates for each data item a probability distribution over the labels for that item, thus it can preserve diversities of opinions, beliefs, etc. that conventional learning hides or ignores. We propose a humans-in-the-loop learning framework to model and study large volumes of unlabeled subjective social media data with less human effort. We study various annotation tasks given to crowdsourced annotators and methods for aggregating their contributions in a manner that preserves subjectivity and disagreement. We introduce a strategy for learning label distributions with only five-to-ten labels per item by aggregating human-annotated labels over multiple, semantically related data items. We conduct experiments using our learning framework on data related to two subjective social issues (work and employment, and suicide prevention) that touch many people worldwide. Our methods can be applied to a broad variety of problems, particularly social problems. Our experimental results suggest that specific label aggregation methods can help provide reliable representative semantics at the population level

Soft Video Parsing by Label Distribution Learning

In this paper, we tackle the problem of segmenting out a sequence of actions from videos. The videos contain background and actions which are usually composed of ordered sub-actions. We refer the sub-actions and the background as semantic units. Considering the possible overlap between two adjacent semantic units, we utilize label distributions to annotate the various segments in the video. The label distribution covers a certain number of semantic unit labels, representing the degree to which each label describes the video segment. The mapping from a video segment to its label distribution is then learned by a Label Distribution Learning (LDL) algorithm. Based on the LDL model, a soft video parsing method with segmental regular grammars is proposed to construct a tree structure for the video. Each leaf of the tree stands for a video clip of background or sub-action. The proposed method shows promising results on the THUMOS'14 and MSR-II datasets and its computational complexity is much less than the state-of-the-art method

Video sub-action parsing algorithm (ASP) on the frisby-catch action of THUMOS2014 dataset.

1.Please unzip the file codeUploadNew.rar. The code, data and result dirs represent the Matlab version of algorithm (Matlab 2014a is adopted), all the features and groundtruth of dataset and the running result of our ASP method. 2.In dir 'code', a_main.m is the main function. Before you start the a_main.m function. You should place your feature of dataset under the dir 'stipFeatureThumos'. An example of the feature file is included in the `stipFeatureThumos'. 3. In dir 'data', dir 'parameters' includes some parameters of ASP and SP. 'videoGrounTruth' is the ground truth of THUMOS'14: ambiguous segments, framerate and frames of videos, and the ground truth segments.'codeAll' and `confAll' are the BoW coding we've learned. 4.A video showing the result of ASP is attached together with its explanation doc file. 5.The paper 'Soft Video Parsing by Label Distribution Learning' is published at AAAI17. The extended version would be published at Frontiers of Computer Science. If you have any question, please contact [email protected] or [email protected]