In the study of effectiveness and efficiency of an athlete’s performance, intelligent systems can be applied on qualitative approaches and their performance metrics provide useful information on not just the quality of the data, but also reveal issues about the observational criteria and data collection context itself. 2000 executions of two similar exercises, with different levels of complexity, were collected through a single inertial sensor applied on the fencer’s weapon hand. After the signals were split into their key segments through Dynamic Time Warping, the extracted features and respective qualitative evaluations were fed into a Neural Network to learn the patterns that distinguish a good from a bad execution. The performance analysis of the resulting models returned a prediction accuracy of 76.6% and 72.7% for each exercise, but other metrics pointed to the data suffering from high bias. This points towards an imbalance in the qualitative criteria representation of the bad executions, which can be explained by: i) reduced number of samples; ii) ambiguity in the definition of the observation criteria; iii) a single sensor being unable to fully capture the context without taking the actions of the other key body segments into account
