2 research outputs found
Challenges in Annotation of useR Data for UbiquitOUs Systems: Results from the 1st ARDUOUS Workshop
Labelling user data is a central part of the design and evaluation of
pervasive systems that aim to support the user through situation-aware
reasoning. It is essential both in designing and training the system to
recognise and reason about the situation, either through the definition of a
suitable situation model in knowledge-driven applications, or through the
preparation of training data for learning tasks in data-driven models. Hence,
the quality of annotations can have a significant impact on the performance of
the derived systems. Labelling is also vital for validating and quantifying the
performance of applications. In particular, comparative evaluations require the
production of benchmark datasets based on high-quality and consistent
annotations. With pervasive systems relying increasingly on large datasets for
designing and testing models of users' activities, the process of data
labelling is becoming a major concern for the community. In this work we
present a qualitative and quantitative analysis of the challenges associated
with annotation of user data and possible strategies towards addressing these
challenges. The analysis was based on the data gathered during the 1st
International Workshop on Annotation of useR Data for UbiquitOUs Systems
(ARDUOUS) and consisted of brainstorming as well as annotation and
questionnaire data gathered during the talks, poster session, live annotation
session, and discussion session
Labeling subtle conversational interactions within the CONVERSE dataset
The field of Human Action Recognition has ex- panded greatly in previous years, exploring actions and inter- actions between individuals via the use of appearance and depth based pose information. There are numerous datasets that display action classes composed of behaviors that are well defined by their key poses, such as ‘kicking’ and ‘punching’. The CONVERSE dataset presents conversational interaction classes that show little explicit relation to the poses and gestures they exhibit. Such a complex and subtle set of interactions is a novel challenge to the Human Action Recognition community, and one that will push the cutting edge of the field in both machine learning and the understanding of human actions. CONVERSE contains recordings of two person interactions from 7 conversational scenarios, represented as sequences of human skeletal poses captured by the Kinect depth sensor. In this study we discuss a method providing ground truth labelling for the set, and the complexity that comes with defining such annotation. The CONVERSE dataset it made available online