Extraction and Analysis of Dynamic Conversational Networks from TV Series

Identifying and characterizing the dynamics of modern tv series subplots is an open problem. One way is to study the underlying social network of interactions between the characters. Standard dynamic network extraction methods rely on temporal integration, either over the whole considered period, or as a sequence of several time-slices. However, they turn out to be inappropriate in the case of tv series, because the scenes shown onscreen alternatively focus on parallel storylines, and do not necessarily respect a traditional chronology. In this article, we introduce Narrative Smoothing, a novel network extraction method taking advantage of the plot properties to solve some of their limitations. We apply our method to a corpus of 3 popular series, and compare it to both standard approaches. Narrative smoothing leads to more relevant observations when it comes to the characterization of the protagonists and their relationships, confirming its appropriateness to model the intertwined storylines constituting the plots.Comment: arXiv admin note: substantial text overlap with arXiv:1602.0781

The one comparing narrative social network extraction techniques

Analysing narratives through their social networks is an expanding field in quantitative literary studies. Manually extracting a social network from any narrative can be time consuming, so automatic extraction methods of varying complexity have been developed. However, the effect of different extraction methods on the resulting networks is unknown. Here we model and compare three extraction methods for social networks in narratives: manual extraction, co-occurrence automated extraction and automated extraction using machine learning. Although the manual extraction method produces more precise results in the network analysis, it is highly time consuming. The automatic extraction methods yield comparable results for density, centrality measures and edge weights. Our results provide evidence that automatically-extracted social networks are reliable for many analyses. We also describe which aspects of analysis are not reliable with such a social network. Our findings provide a framework to analyse narratives, which help us improve our understanding of how stories are written and evolve, and how people interact with each other. Index Tenns-social networks, narratives, televisionMichelle Edwards, Jonathan Tuke, Matthew Roughan, Lewis Mitchel

Inspection and evaluation of artifacts in digital video sources

Streaming digital video content providers such as YouTube, Amazon, Hulu, and Netflix collaborate with production teams to obtain new and old video content. These collaborations lead to an accumulation of video sources, some of which might contain unacceptable visual artifacts. Artifacts may inadvertently enter the video master at any point in the production pipeline, due to any of a number of equipment and user failures. Unfortunately, these artifacts are difficult to detect since no pristine reference exists for comparison. As of now, few automated tools exist that can effectively capture the most common forms of these artifacts. This work studies no-reference video source inspection for generalized artifact detection and subjective quality prediction, which will ultimate inform decisions related to acquisition of new content. Automatically identifying the locations and severities of video artifacts is a difficult problem. We have developed a general method for detecting local artifacts by learning differences in the statistics between distorted and pristine video frames. Our model, which we call the Video Impairment Mapper (VID-MAP), produces a full resolution map of artifact detection probabilities based on comparisons of excitatory and inhibatory convolutional responses. Validation on a large database shows that our method outperforms the previous state-of-the-art of even distortion-specific detectors. A variety of powerful picture quality predictors are available that rely on neuro-statistical models of distortion perception. We extend these principles to video source inspection, by coupling spatial divisive normalization with a series of filterbanks tuned for artifact detection, implemented using a common convolutional framework. We developed the Video Impairment Detection by SParse Error CapTure (VIDSPECT) model, which leverages discriminative sparse dictionaries that are tuned to detect specific artifacts. VIDSPECT is simple, highly generalizable, and yields better accuracy than competing methods. To evaluate the perceived quality of video sources containing artifacts, we built a new digital video database, called the LIVE Video Masters Database, which contains 384 videos affected by the types of artifacts encountered in otherwise pristine digital video sources. We find that VIDSPECT delivers top performance on this database for most artifacts tested, and competitive performance otherwise, using the same basic architecture in all cases.Electrical and Computer Engineerin

The One with the Social Network Analysis: the extraction, analysis and modelling of temporal social networks from narratives

Narratives tell us about the people, cultures, and time periods in and about which they were written. Therefore, narrative analysis is a powerful tool for understanding culture. One way to analyse narratives is through their social networks, however extracting the network is a complex task. Manually recording characters and their interactions is an accurate, but time consuming method for narrative social network extraction, however efficient automatic extraction methods may introduce errors. In this thesis, we perform a detailed comparative study of narrative social network extraction techniques, and investigate the effect the techniques have on the analysis of the narrative. We use the 1994–2004 television series Friends as a case study to model and compare extraction techniques. By designing a simulated social network and observation processes resembling different network extraction techniques, we find that automated network extraction methods are reliable for computing many network metrics, but can distort the clustering coefficient. Our comparison of extraction techniques allows for many more narratives to be extracted and analysed efficiently. We also analyse and model the social networks of Friends, to gain new insights into the the series, and what made it successful. We show which are the most important characters and relationships, and through modelling social network features we find the most informative features to predict success. Our analysis of Friends provides an example and a building block for deeper understanding about particular narratives and narratives in general.Thesis (MPhil) -- University of Adelaide, School of Mathematical Sciences, 201

Intelligent Side Information Generation in Distributed Video Coding

Distributed video coding (DVC) reverses the traditional coding paradigm of complex encoders allied with basic decoding to one where the computational cost is largely incurred by the decoder. This is attractive as the proven theoretical work of Wyner-Ziv (WZ) and Slepian-Wolf (SW) shows that the performance by such a system should be exactly the same as a conventional coder. Despite the solid theoretical foundations, current DVC qualitative and quantitative performance falls short of existing conventional coders and there remain crucial limitations. A key constraint governing DVC performance is the quality of side information (SI), a coarse representation of original video frames which are not available at the decoder. Techniques to generate SI have usually been based on linear motion compensated temporal interpolation (LMCTI), though these do not always produce satisfactory SI quality, especially in sequences exhibiting non-linear motion. This thesis presents an intelligent higher order piecewise trajectory temporal interpolation (HOPTTI) framework for SI generation with original contributions that afford better SI quality in comparison to existing LMCTI-based approaches. The major elements in this framework are: (i) a cubic trajectory interpolation algorithm model that significantly improves the accuracy of motion vector estimations; (ii) an adaptive overlapped block motion compensation (AOBMC) model which reduces both blocking and overlapping artefacts in the SI emanating from the block matching algorithm; (iii) the development of an empirical mode switching algorithm; and (iv) an intelligent switching mechanism to construct SI by automatically selecting the best macroblock from the intermediate SI generated by HOPTTI and AOBMC algorithms. Rigorous analysis and evaluation confirms that significant quantitative and perceptual improvements in SI quality are achieved with the new framework

Extraction multimodale de la structure narrative des épisodes de séries télévisées

Nos contributions portent sur l'extraction de la structure narrative d'épisodes de séries télévisées à deux niveaux hiérarchiques. Le premier niveau de structuration consiste à retrouver les transitions entre les scènes à partir d'une analyse de la couleur des images et des locuteurs présents dans les scènes. Nous montrons que l'analyse des locuteurs permet d'améliorer le résultat d'une segmentation en scènes basée sur la couleur. Il est courant de voir plusieurs histoires (ou lignes d'actions) racontées en parallèle dans un même épisode de série télévisée. Ainsi, le deuxième niveau de structuration consiste à regrouper les scènes en histoires. Nous cherchons à désentrelacer les histoires pour pouvoir, par exemple, visualiser les différentes lignes d'actions indépendamment. La principale difficulté consiste à déterminer les descripteurs les plus pertinents permettant de regrouper les scènes appartenant à une même histoire. A ce niveau, nous étudions également l'utilisation de descripteurs provenant des trois modalités différentes précédemment exposées. Nous proposons en outre des méthodes permettant de fusionner les informations provenant de ces trois modalités. Pour répondre à la variabilité de la structure narrative des épisodes de séries télévisées, nous proposons une méthode qui s'adapte à chaque épisode. Elle permet de choisir automatiquement la méthode de regroupement la plus pertinente parmi les différentes méthodes proposées. Enfin, nous avons développé StoViz, un outil de visualisation de la structure d'un épisode de série télévisée (scènes et histoires). Il permet de faciliter la navigation au sein d'un épisode, en montrant les différentes histoires racontées en parallèle dans l'épisode. Il permet également la lecture des épisodes histoire par histoire, et la visualisation d'un court résumé de l'épisode en donnant un aperçu de chaque histoire qui y est racontée.Our contributions concern the extraction of the structure of TV series episodes at two hierarchical levels. The first level of structuring is to find the scene transitions based on the analysis of the color information and the speakers involved in the scenes. We show that the analysis of the speakers improves the result of a color-based segmentation into scenes. It is common to see several stories (or lines of action) told in parallel in a single TV series episode. Thus, the second level of structure is to cluster scenes into stories. We seek to deinterlace the stories in order to visualize the different lines of action independently. The main difficulty is to determine the most relevant descriptors for grouping scenes belonging to the same story. We explore the use of descriptors from the three different modalities described above. We also propose methods to combine these three modalities. To address the variability of the narrative structure of TV series episodes, we propose a method that adapts to each episode. It can automatically select the most relevant clustering method among the various methods we propose. Finally, we developed StoViz, a tool for visualizing the structure of a TV series episode (scenes and stories). It allows an easy browsing of each episode, revealing the different stories told in parallel. It also allows playback of episodes story by story, and visualizing a summary of the episode by providing a short overview of each story

Efficient Analysis in Multimedia Databases

The rapid progress of digital technology has led to a situation where computers have become ubiquitous tools. Now we can find them in almost every environment, be it industrial or even private. With ever increasing performance computers assumed more and more vital tasks in engineering, climate and environmental research, medicine and the content industry. Previously, these tasks could only be accomplished by spending enormous amounts of time and money. By using digital sensor devices, like earth observation satellites, genome sequencers or video cameras, the amount and complexity of data with a spatial or temporal relation has gown enormously. This has led to new challenges for the data analysis and requires the use of modern multimedia databases. This thesis aims at developing efficient techniques for the analysis of complex multimedia objects such as CAD data, time series and videos. It is assumed that the data is modeled by commonly used representations. For example CAD data is represented as a set of voxels, audio and video data is represented as multi-represented, multi-dimensional time series. The main part of this thesis focuses on finding efficient methods for collision queries of complex spatial objects. One way to speed up those queries is to employ a cost-based decompositioning, which uses interval groups to approximate a spatial object. For example, this technique can be used for the Digital Mock-Up (DMU) process, which helps engineers to ensure short product cycles. This thesis defines and discusses a new similarity measure for time series called threshold-similarity. Two time series are considered similar if they expose a similar behavior regarding the transgression of a given threshold value. Another part of the thesis is concerned with the efficient calculation of reverse k-nearest neighbor (RkNN) queries in general metric spaces using conservative and progressive approximations. The aim of such RkNN queries is to determine the impact of single objects on the whole database. At the end, the thesis deals with video retrieval and hierarchical genre classification of music using multiple representations. The practical relevance of the discussed genre classification approach is highlighted with a prototype tool that helps the user to organize large music collections. Both the efficiency and the effectiveness of the presented techniques are thoroughly analyzed. The benefits over traditional approaches are shown by evaluating the new methods on real-world test datasets

Energy efficient enabling technologies for semantic video processing on mobile devices

Semantic object-based processing will play an increasingly important role in future multimedia systems due to the ubiquity of digital multimedia capture/playback technologies and increasing storage capacity. Although the object based paradigm has many undeniable benefits, numerous technical challenges remain before the applications becomes pervasive, particularly on computational constrained mobile devices. A fundamental issue is the ill-posed problem of semantic object segmentation. Furthermore, on battery powered mobile computing devices, the additional algorithmic complexity of semantic object based processing compared to conventional video processing is highly undesirable both from a real-time operation and battery life perspective. This thesis attempts to tackle these issues by firstly constraining the solution space and focusing on the human face as a primary semantic concept of use to users of mobile devices. A novel face detection algorithm is proposed, which from the outset was designed to be amenable to be offloaded from the host microprocessor to dedicated hardware, thereby providing real-time performance and reducing power consumption. The algorithm uses an Artificial Neural Network (ANN), whose topology and weights are evolved via a genetic algorithm (GA). The computational burden of the ANN evaluation is offloaded to a dedicated hardware accelerator, which is capable of processing any evolved network topology. Efficient arithmetic circuitry, which leverages modified Booth recoding, column compressors and carry save adders, is adopted throughout the design. To tackle the increased computational costs associated with object tracking or object based shape encoding, a novel energy efficient binary motion estimation architecture is proposed. Energy is reduced in the proposed motion estimation architecture by minimising the redundant operations inherent in the binary data. Both architectures are shown to compare favourable with the relevant prior art

Beyond Narrative

This book calls for an investigation of the ›borderlands of narrativity‹ — the complex and culturally productive area where the symbolic form of narrative meets other symbolic logics, such as data(base), play, spectacle, or ritual. It opens up a conversation about the ›beyond‹ of narrative, about the myriad constellations in which narrativity interlaces with, rubs against, or morphs into the principles of other forms. To conceptualize these borderlands, the book introduces the notion of »narrative liminality,« which the 16 articles utilize to engage literature, popular culture, digital technology, historical artifacts, and other kinds of texts from a time span of close to 200 years