One video is sufficient? Human activity recognition using active video composition

In this paper, we present a novel human activity recogni-tion approach that only requires a single video example per activity. We introduce the paradigm of active video com-position, which enables one-example recognition of com-plex activities. The idea is to automatically create a large number of semi-artificial training videos called composed videos by manipulating an original human activity video. A methodology to automatically compose activity videos hav-ing different backgrounds, translations, scales, actors, and movement structures is described in this paper. Further-more, an active learning algorithm to model the temporal structure of the human activity has been designed, prevent-ing the generation of composed training videos violating the structural constraints of the activity. The intention is to gen-erate composed videos having correct organizations, and take advantage of them for the training of the recognition system. In contrast to previous passive recognition systems relying only on given training videos, our methodology ac-tively composes necessary training videos that the system is expected to observe in its environment. Experimental re-sults illustrate that a single fully labeled video per activity is sufficient for our methodology to reliably recognize human activities by utilizing composed training videos. 1