Predicting Panel Ratings for Semantic Characteristics of Lung Nodules

In reading CT scans with potentially malignant lung nodules, radiologists make use of high level information (semantic characteristics) in their analysis. CAD systems can assist radiologists by offering a “second opinion” - predicting these semantic characteristics for lung nodules. In our previous work, we developed such a CAD system, training and testing it on the publicly available Lung Image Database Consortium (LIDC) dataset, which includes semantic annotations by up to four human radiologists for every nodule. However, due to the lack of ground truth and the uncertainty in the dataset, each nodule was viewed as four distinct instances when training the classifier. In this work, we propose a way of predicting the distribution of opinions of the four radiologists using a multiple-label classification algorithm based on belief decision trees. We evaluate our results using a distance-threshold curve and, measuring the area under this curve, obtain 69% accuracy on the testing subset. We conclude that multiple-label classification algorithms are an appropriate method of representing the diagnoses of multiple radiologists on lung CT scans when a single ground truth is not available

Learning from data with uncertain labels by boosting credal classifiers

International audienceIn this article, we investigate supervised learning when training data are associated with uncertain labels. We tackle this problem within the theory of belief functions. Each training pattern x_i is thus associated with a basic belief assignment, representing partial knowledge of its actual class. Here, we propose to use the approach known as boosting to solve the classification problem. We propose a variant of the AdaBoost algorithm where the outputs of the classifiers are interpreted as belief functions. During training, our algorithm estimates the reliability of each classifier to identify patterns from the various classes. During test phase, the outputs of the classifiers are first discounted according to these reliabilities, and then combined using a suitable rule. Experiments conducted on classical datasets show that our algorithm is comparable to AdaBoost in accuracy. Processing EEG data with imperfect labels clearly demonstrates the interest of taking into account the reliability of the labelling, and thus the relevance of our approach

Liberal or Conservative: Evaluation and Classification with Distribution as Ground Truth.

The ability to classify the political leaning of a large number of articles and items is valuable to both academic research and practical applications. The challenge, though, is not only about developing innovative classification algorithms, which constitutes a “classifier” theme in this thesis, but also about how to define the “ground truth” of items’ political leaning, how to elicit labels when labelers do not agree, and how to evaluate classifiers with unreliable labeled data, which constitutes a “ground truth” theme in the thesis. The “ground truth” theme argues for the use of distributions (e.g., 0.6 conservative, 0.4 liberal) instead of labels (e.g, conservative, liberal) as the underlying ground truth of items’ political leaning, where disagreements among labelers are not human errors but rather useful information reflecting the distribution of people’s subjective opinions. Empirical data demonstrate that distributions are dispersed: there are many items upon which labelers simply do not agree. Therefore, mapping distributions into single labels requires more than just majority vote. Also, one can no longer assume the labels from a few labelers are reliable because a different small sample of labelers might yield a very different picture. However, even though individual labeled items are not reliable, simulation suggests that we may still reliably evaluate and rank classifiers, as long as we have a large number of labeled items for evaluation. The optimal way is to obtain one label per item with many items (e.g., 1000~3000) for evaluation. The “classifier” theme proposes the LabelPropagator algorithm that propagates the political leaning of known articles and users to the target nodes in order to classify them. LabelPropagator achieves higher accuracy than the alternative classifiers based on text analysis, suggesting that a relatively small number of labeled people and stories, together with a large number of people to item votes, can be used to classify the other people and items. An article’s source is useful as an input for propagation, while text similarities, users’ friendship, and “href” links to articles are not.PHDInformationUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/97979/1/mrzhou_1.pd