Sketch-based digital storyboards and floor plans for authoring computer-generated film pre-visuals

Pre-visualisation is an important tool for planning films during the pre-production phase of filmmaking. Existing pre-visualisation authoring tools do not effectively support the user in authoring pre-visualisations without impairing software usability. These tools require the user to either have programming skills, be experienced in modelling and animation, or use drag-and-drop style interfaces. These interaction methods do not intuitively fit with pre-production activities such as floor planning and storyboarding, and existing tools that apply a storyboarding metaphor do not automatically interpret user sketches. The goal of this research was to investigate how sketch-based user interfaces and methods from computer vision could be used for supporting pre-visualisation authoring using a storyboarding approach. The requirements for such a sketch-based storyboarding tool were determined from literature and an interview with Triggerfish Animation Studios. A framework was developed to support sketch-based pre-visualisation authoring using a storyboarding approach. Algorithms for describing user sketches, recognising objects and performing pose estimation were designed to automatically interpret user sketches. A proof of concept prototype implementation of this framework was evaluated in order to assess its usability benefit. It was found that the participants could author pre-visualisations effectively, efficiently and easily. The results of the usability evaluation also showed that the participants were satisfied with the overall design and usability of the prototype tool. The positive and negative findings of the evaluation were interpreted and combined with existing heuristics in order to create a set of guidelines for designing similar sketch-based pre-visualisation authoring tools that apply the storyboarding approach. The successful implementation of the proof of concept prototype tool provides practical evidence of the feasibility of sketch-based pre-visualisation authoring. The positive results from the usability evaluation established that sketch-based interfacing techniques can be used effectively with a storyboarding approach for authoring pre-visualisations without impairing software usability

Object detection with constellations of keypoints

Feature-based object detection methods rely on the discriminative nature of features in order to accurately determine the location of a specific object in a test image. From a set of detected features, non-discriminative features are filtered out by means of a similarity threshold, meaning that if a features is very similar to more than one model feature, it is considered to be non-discriminative. However, in cases where an object consists of repeating patterns the similarity threshold proves inefficient since it considers the majority of detected features to be similar to more than one model feature, i.e., non-discriminative. In the context of one-shot learning we propose a constellation model for enhancing basic feature-based object detection methods, with the aim in utilizing the preserved geometry between features to filter out noisy feature matches. This eliminates the need for the similarity threshold. We evaluate the proposed constellation model whit empirically and numerically modelled feature variance and compare it to a baseline feature model. Model evaluation is performed on a challenging real-world dataset, consisting of logotypes in real-world scenarios. We find that the best variation of the constellation model is the model with empirically determined feature variance, which significantly reduces the number of mismatched features, without significantly affecting detection performance