4 research outputs found
The Data Representativeness Criterion: Predicting the Performance of Supervised Classification Based on Data Set Similarity
In a broad range of fields it may be desirable to reuse a supervised
classification algorithm and apply it to a new data set. However,
generalization of such an algorithm and thus achieving a similar classification
performance is only possible when the training data used to build the algorithm
is similar to new unseen data one wishes to apply it to. It is often unknown in
advance how an algorithm will perform on new unseen data, being a crucial
reason for not deploying an algorithm at all. Therefore, tools are needed to
measure the similarity of data sets. In this paper, we propose the Data
Representativeness Criterion (DRC) to determine how representative a training
data set is of a new unseen data set. We present a proof of principle, to see
whether the DRC can quantify the similarity of data sets and whether the DRC
relates to the performance of a supervised classification algorithm. We
compared a number of magnetic resonance imaging (MRI) data sets, ranging from
subtle to severe difference is acquisition parameters. Results indicate that,
based on the similarity of data sets, the DRC is able to give an indication as
to when the performance of a supervised classifier decreases. The strictness of
the DRC can be set by the user, depending on what one considers to be an
acceptable underperformance.Comment: 12 pages, 6 figure
Learning an MR acquisition-invariant representation using Siamese neural networks
Generalization of voxelwise classifiers is hampered by differences between MRI-scanners, e.g. different acquisition protocols and field strengths. To address this limitation, we propose a Siamese neural network (MRAI-NET) that extracts acquisition-invariant feature vectors. These can consequently be used by task-specific methods, such as voxelwise classifiers for tissue segmentation. MRAI-NET is evaluated on both simulated and real patient data. Experiments show that MRAI-NET outperforms both voxelwise classifiers trained on the source data as well as classifiers trained on the limited amount of target scanner data available