Inspection and evaluation of artifacts in digital video sources

Streaming digital video content providers such as YouTube, Amazon, Hulu, and Netflix collaborate with production teams to obtain new and old video content. These collaborations lead to an accumulation of video sources, some of which might contain unacceptable visual artifacts. Artifacts may inadvertently enter the video master at any point in the production pipeline, due to any of a number of equipment and user failures. Unfortunately, these artifacts are difficult to detect since no pristine reference exists for comparison. As of now, few automated tools exist that can effectively capture the most common forms of these artifacts. This work studies no-reference video source inspection for generalized artifact detection and subjective quality prediction, which will ultimate inform decisions related to acquisition of new content. Automatically identifying the locations and severities of video artifacts is a difficult problem. We have developed a general method for detecting local artifacts by learning differences in the statistics between distorted and pristine video frames. Our model, which we call the Video Impairment Mapper (VID-MAP), produces a full resolution map of artifact detection probabilities based on comparisons of excitatory and inhibatory convolutional responses. Validation on a large database shows that our method outperforms the previous state-of-the-art of even distortion-specific detectors. A variety of powerful picture quality predictors are available that rely on neuro-statistical models of distortion perception. We extend these principles to video source inspection, by coupling spatial divisive normalization with a series of filterbanks tuned for artifact detection, implemented using a common convolutional framework. We developed the Video Impairment Detection by SParse Error CapTure (VIDSPECT) model, which leverages discriminative sparse dictionaries that are tuned to detect specific artifacts. VIDSPECT is simple, highly generalizable, and yields better accuracy than competing methods. To evaluate the perceived quality of video sources containing artifacts, we built a new digital video database, called the LIVE Video Masters Database, which contains 384 videos affected by the types of artifacts encountered in otherwise pristine digital video sources. We find that VIDSPECT delivers top performance on this database for most artifacts tested, and competitive performance otherwise, using the same basic architecture in all cases.Electrical and Computer Engineerin

Video-based step measurement in sport and daily living.

Current knowledge of tennis player-surface interactions with different court surfaces is limited. The measurement of player step and movement strategy would aid the understanding of tennis player-surface interaction. However, this has not yet been performed: no readily available motion analysis tool is capable of measuring spatio-temporal parameters of gait during match-play tennis. The purpose of this project was to develop, validate and use a motion analysis tool designed to measure player location and foot-surface contacts during match-play tennis.Single camera video footage, obtained from the 2011 Roland Garros Qualifying Tournament, was manually digitised to characterise step and movement strategy during men's and women's forehand groundstrokes. Player movements were consistent with previous notational analyses; however gender differences were highlighted for step frequency. Initial findings were limited by manual analysis, e.g. manual digitising subjectivity and low sample size: an objective and automated system was required.A markerless, view-independent, foot-surface contact identification (FSCi) algorithm was developed. The FSCi algorithm identifies foot-surface contacts in image sequences of gait by quantifying the motion of each foot. The algorithm was validated using standard colour image sequences of walking and running obtained from four unique camera perspectives: output data were compared to three-dimensional motion analysis. The FSCi algorithm identified data for 1243 of 1248 foot-surface contacts; root-mean-square error (RMSE) was 52.2 and 103.4 mm for shod walking and running respectively (all camera perspectives). Findings demonstrated that the FSCi algorithm measured basic, spatio-temporal parameters of walking and running, e.g. step length and step time, without interfering with the activity being observed. Furthermore, analyses were independent of camera view.Video footage obtained from the 2011 ATP World Tour Finals was used to develop a combined player tracking and foot-surface contact identification (PT-FSCi) algorithm. Furthermore, a graphical user interface was developed. The PT-FSCi algorithm was used to analyse twenty match-play tennis rallies: output data were compared to manual digitising. The PT-FSCi algorithm tracked player position and identified data for 832 of 890 foot-surface contacts during match-play tennis. RMSE for player position and foot-surface contacts was 232.9 and 121.9 mm respectively. The calculation of step parameters required manual intervention: this reflected the multi-directional nature of tennis. This represents a limitation to the current algorithm however the segmentation of player movement phases to allow the automatic calculation of step parameters.The analysis of this data indicated that top ranked tennis players can win rallies using movement strategies previously considered to be defensive. Furthermore, step length data indicated that shorter step lengths formed the majority of step strategy. The largest 25% of steps were observed behind the baseline, aligned with deuce and advantage court sidelines. This reflected lunging and turning manoeuvres at lateral extremes of player movement.The single camera system that has resulted from this project will enable the International Tennis Federation to characterise player step and movement strategy during match-play tennis. This will allow a more informed approach to player-surface interaction research. Furthermore, the system has potential to be used for different applications, ranging from sport to surveillance

Visualizing Transmedia Networks: Links, Paths and Peripheries

`Visualizing Transmedia Networks: Links, Paths and Peripheries' examines the increasingly complex rhetorical intersections between narrative and media (`old' and `new') in the creation of transmedia fictions, loosely defined as multisensory and multimodal stories told extensively across a diverse media set. In order to locate the `language' of transmedia expressions, this project calls attention to the formally locatable network structures placed by transmedia producers in disparate media like film, the print novel and video games. Using network visualization software and computational metrics, these structures can be used as data to graph these fictions for both quantitative and qualitative analysis. This study also, however, examines the limits to this approach, arguing that the process of transremediation, where redundancy and multiformity take precedence over networked connection, forms a second axis for understanding transmedia practices, one equally bound to the formation of new modes of meaning and literacy

Proceedings of the 9th international conference on disability, virtual reality and associated technologies (ICDVRAT 2012)

The proceedings of the conferenc

Hands-off Interactive Storytelling in Cinematic Virtual Reality

This is a research by creative practice that aims to explore a form of hands-off interactivity in cinematic virtual reality (CVR). The proposed model for interactive storytelling is based more on intuitive reactions than on conscious decision-making, enhancing diegetic and, thus, narrative immersion. The initial hypothesis states that hands-off interactivity can allow a user to experience a diegesis in which they can avoid being “pulled-back” from the immersion, an interruption of the story produced by the consciousness of explicit interaction and extra-diegetic interfaces. To achieve this, this project uses immersion, spatial storytelling, and dramatically-motivated soundscapes to facilitate and encourage navigation through simultaneous acoustic and dramatic spaces in one immersive environment. Using this setup, the interactive storytelling takes place as users are presented with two simultaneous storylines with their respective protagonists, which happen to be interdependent, influence each other, and are part of one integral story. Users would then be able to freely navigate and alternate between the two storylines – being influenced by strategically designed visual and acoustic diegetic stimuli – and thus play an active role in getting to make sense of the narration. This way, users generate inputs with organic movements around the fixed axis in which CVR uses are placed. This research is strongly focused on creative practice, the generation of creative outputs, and the analysis of the procedures and production workflows, to understand what are the creative and technical challenges for the proposed type of interactive storytelling. The project is also faced from an interdisciplinary approach that, while centred in a filmmaker’s perspective, makes a critical integration of concepts and techniques from other relevant disciplines to approach the expressive challenges proposed by CVR as an experimental medium

Of the Repository: Poetics in a Networked Digital Milieu

This doctoral dissertation is a material and cultural analysis of the entwined histories of the three major North American digital repositories of contemporary avant-garde and experimental poetry: the Electronic Poetry Center at the State University of New York at Buffalo, Kenneth Goldsmith’s UbuWeb, and PennSound at the University of Pennsylvania. The dissertation takes up a media-historical methodology to document the actors, publics, discourses, aesthetics, institutional environments, technological infrastructures, and social relations involved in the production of these open online repositories. The research begins from the premise that, in the study of what writing is, has been, and might be, the discourse of poetics and the figure of the archive fuse together. If, as Kate Eichhorn (2003) argues, “[t]o write in a digital age is to write in the archive,” in this research I ask: What can the composition of archives – their materials, contexts, protocols, and interfaces – teach us about poetics today? Since the mid-1990s, these three repositories have served as a primary means for extending the purview and program of poetics as a contemporary institutional formation. In doing so, the creators of these repositories have utilized them as important media infrastructures for the publication, dissemination, and storage of poetic works and critical analysis on the contemporary production of poetry. Each digital repository is an argument for a specific poetics. Their entwined histories and cultural-technical infrastructures articulate numerous affinities, yet each is distinct in the way it casts a new light on certain critical terms for literary studies. Approaching each in terms of its emphasis on, respectively, access, circulation, and format enables a detailed engagement with the aesthetic, institutional, legal, and technological concerns of the digital repository. Here, this dissertation develops a unique methodology for addressing these complicated structures called digital repositories by emphasizing each case study’s particular bias. Such an engagement opens on to a more general consideration of language, writing, and textuality in networked milieus, and emphasizes the particular affordances that make the digital repository a significant, yet underkacknowledged, archival genre