4 research outputs found
Fast Dawid-Skene: A Fast Vote Aggregation Scheme for Sentiment Classification
Many real world problems can now be effectively solved using supervised
machine learning. A major roadblock is often the lack of an adequate quantity
of labeled data for training. A possible solution is to assign the task of
labeling data to a crowd, and then infer the true label using aggregation
methods. A well-known approach for aggregation is the Dawid-Skene (DS)
algorithm, which is based on the principle of Expectation-Maximization (EM). We
propose a new simple, yet effective, EM-based algorithm, which can be
interpreted as a `hard' version of DS, that allows much faster convergence
while maintaining similar accuracy in aggregation. We show the use of this
algorithm as a quick and effective technique for online, real-time sentiment
annotation. We also prove that our algorithm converges to the estimated labels
at a linear rate. Our experiments on standard datasets show a significant
speedup in time taken for aggregation - upto 8x over Dawid-Skene and
6x over other fast EM methods, at competitive accuracy performance. The
code for the implementation of the algorithms can be found at
https://github.com/GoodDeeds/Fast-Dawid-SkeneComment: 8 pages, 5 tables, 1 figure, KDD Workshop on Issues of Sentiment
Discovery and Opinion Mining (WISDOM) 201
Towards Better Understanding Attribution Methods
Deep neural networks are very successful on many vision tasks, but hard to
interpret due to their black box nature. To overcome this, various post-hoc
attribution methods have been proposed to identify image regions most
influential to the models' decisions. Evaluating such methods is challenging
since no ground truth attributions exist. We thus propose three novel
evaluation schemes to more reliably measure the faithfulness of those methods,
to make comparisons between them more fair, and to make visual inspection more
systematic. To address faithfulness, we propose a novel evaluation setting
(DiFull) in which we carefully control which parts of the input can influence
the output in order to distinguish possible from impossible attributions. To
address fairness, we note that different methods are applied at different
layers, which skews any comparison, and so evaluate all methods on the same
layers (ML-Att) and discuss how this impacts their performance on quantitative
metrics. For more systematic visualizations, we propose a scheme (AggAtt) to
qualitatively evaluate the methods on complete datasets. We use these
evaluation schemes to study strengths and shortcomings of some widely used
attribution methods. Finally, we propose a post-processing smoothing step that
significantly improves the performance of some attribution methods, and discuss
its applicability.Comment: 30 pages, 31 figures, 2 tables, IEEE/CVF Conference on Computer
Vision and Pattern Recognition (CVPR) 202
Fast Dawid-Skene: A Fast Vote Aggregation Scheme for Sentiment Classification,
Many real world problems can now be effectively solved using supervised machine learning. A major roadblock is often the lack of an adequate quantity of labeled data for training. A possible solution is to assign the task of labeling data to a crowd, and then infer the true label using aggregation methods. A well-known approach for aggregation is the Dawid-Skene (DS) algorithm, which is based on the principle of Expectation-Maximization (EM). We propose a new simple, yet effective, EM-based algorithm, which can be interpreted as a `hard' version of DS, that allows much faster convergence while maintaining similar accuracy in aggregation. We show the use of this algorithm as a quick and effective technique for online, real-time sentiment annotation. We also prove that our algorithm converges to the estimated labels at a linear rate. Our experiments on standard datasets show a significant speedup in time taken for aggregation - upto 8x over Dawid-Skene and 6x over other fast EM methods, at competitive accuracy performance. The code for the implementation of the algorithms can be found at this https UR
Open-WBO-Inc: Approximation Strategies for Incomplete Weighted MaxSAT
Incomplete MaxSAT solving aims to quickly find a solution that attempts to minimize the sum of the weights of unsatisfied soft clauses without providing any optimality guarantees. In this paper, we propose two approximation strategies for improving incomplete weighted MaxSAT solving. In one of the strategies, we cluster the weights and approximate them with a representative weight. In another strategy, we break up the problem of minimizing the sum of weights of unsatisfiable clauses into multiple minimization subproblems. We have implemented these strategies in a tool Open-WBO-Inc. Using the subproblem minimization strategy, Open-WBO-Inc placed first and second in the weighted incomplete tracks in the MaxSAT Evaluation 2018 whereas the strategy based on weight approximation was placed fourth. We compare these strategies with the best incomplete MaxSAT solvers on benchmarks taken from MaxSAT Evaluation 2017 and MaxSAT Evaluation 2018 and show that the strategies proposed are competitive with the best of the solvers