Characterization of Common Videos with Signatures Extracted from Frame Transition Profiles

People have access to a tremendous amount of video nowadays, both on television and Internet. The amount of video that a viewer has to choose from is so large that it is infeasible for a human to go through it all to find a video of interest. Organizing video into categories will make the process of large number of videos much faster and improves the ease of access. A profile created by observing the rate at which the contents of video frame changes helps in categorization of videos in different types. The experiments we conducted on three types of videos (News, Sports, and Music) show that a profile built on a set of frame transition parameter measurements could be applied to automatically distinguish the types of these videos. We have researched a way to automatically characterize videos into their respected video type, such as a news, music, or sports video clips, by comparing the content value transitions among the video frames. The objective of this research is to see if some measurements extracted from frame transitions are used to show the differences between different categories of videos. In other words, we want to see if such kind of values and measurements can be used to tell different kind of videos or the genre of videos, e.g., with respect to the authors. Our program extracts the statistical data from the video frames based on the histograms of the grayscale pixel intensity changes in the frame transitions. A variety of videos were tested to categorize them using the extracted signatures from these frame transition profiles. The signatures extracted presents a problem of classification that can be addressed using the machine learning algorithms. Time complexity of the evaluation is decreased when compared to other methods in video classification as the video is processed in a single step where all the features are extracted and analysis is performed on the obtained signatures. This provides a simple approach in classifying the videos, additional signatures will be extracted to create a more efficient profiling system to better reveal the nature and characteristics of the video categorization

Video summarisation: A conceptual framework and survey of the state of the art

This is the post-print (final draft post-refereeing) version of the article. Copyright @ 2007 Elsevier Inc.Video summaries provide condensed and succinct representations of the content of a video stream through a combination of still images, video segments, graphical representations and textual descriptors. This paper presents a conceptual framework for video summarisation derived from the research literature and used as a means for surveying the research literature. The framework distinguishes between video summarisation techniques (the methods used to process content from a source video stream to achieve a summarisation of that stream) and video summaries (outputs of video summarisation techniques). Video summarisation techniques are considered within three broad categories: internal (analyse information sourced directly from the video stream), external (analyse information not sourced directly from the video stream) and hybrid (analyse a combination of internal and external information). Video summaries are considered as a function of the type of content they are derived from (object, event, perception or feature based) and the functionality offered to the user for their consumption (interactive or static, personalised or generic). It is argued that video summarisation would benefit from greater incorporation of external information, particularly user based information that is unobtrusively sourced, in order to overcome longstanding challenges such as the semantic gap and providing video summaries that have greater relevance to individual users

Reading the Source Code of Social Ties

Though online social network research has exploded during the past years, not much thought has been given to the exploration of the nature of social links. Online interactions have been interpreted as indicative of one social process or another (e.g., status exchange or trust), often with little systematic justification regarding the relation between observed data and theoretical concept. Our research aims to breach this gap in computational social science by proposing an unsupervised, parameter-free method to discover, with high accuracy, the fundamental domains of interaction occurring in social networks. By applying this method on two online datasets different by scope and type of interaction (aNobii and Flickr) we observe the spontaneous emergence of three domains of interaction representing the exchange of status, knowledge and social support. By finding significant relations between the domains of interaction and classic social network analysis issues (e.g., tie strength, dyadic interaction over time) we show how the network of interactions induced by the extracted domains can be used as a starting point for more nuanced analysis of online social data that may one day incorporate the normative grammar of social interaction. Our methods finds applications in online social media services ranging from recommendation to visual link summarization.Comment: 10 pages, 8 figures, Proceedings of the 2014 ACM conference on Web (WebSci'14

Highly efficient low-level feature extraction for video representation and retrieval.

PhDWitnessing the omnipresence of digital video media, the research community has raised the question of its meaningful use and management. Stored in immense multimedia databases, digital videos need to be retrieved and structured in an intelligent way, relying on the content and the rich semantics involved. Current Content Based Video Indexing and Retrieval systems face the problem of the semantic gap between the simplicity of the available visual features and the richness of user semantics. This work focuses on the issues of efficiency and scalability in video indexing and retrieval to facilitate a video representation model capable of semantic annotation. A highly efficient algorithm for temporal analysis and key-frame extraction is developed. It is based on the prediction information extracted directly from the compressed domain features and the robust scalable analysis in the temporal domain. Furthermore, a hierarchical quantisation of the colour features in the descriptor space is presented. Derived from the extracted set of low-level features, a video representation model that enables semantic annotation and contextual genre classification is designed. Results demonstrate the efficiency and robustness of the temporal analysis algorithm that runs in real time maintaining the high precision and recall of the detection task. Adaptive key-frame extraction and summarisation achieve a good overview of the visual content, while the colour quantisation algorithm efficiently creates hierarchical set of descriptors. Finally, the video representation model, supported by the genre classification algorithm, achieves excellent results in an automatic annotation system by linking the video clips with a limited lexicon of related keywords

Combining Smart Material Platforms and New Computational Tools to Investigate Cell Motility Behavior and Control

Cell-extracellular matrix (ECM) interactions play a critical role in regulating important biological phenomena, including morphogenesis, tissue repair, and disease states. In vivo, cells are subjected to various mechanical, chemical, and electrical cues to collectively guide their functionality within a specific microenvironment. To better understand the mechanisms regulating cell adhesive, differentiation, and motility dynamics, researchers have developed in vitro platforms to synthetically mimic native tissue responses. While important information about cell-ECM interactions have been revealed using these systems, a knowledge gap currently exists regarding how cell responses in static environments relate to the dynamic cell-ECM interaction behaviors observed in vivo. Advances at the intersection of materials science, biophysics, and cell biology have recently enabled the production of dynamic ECM mimics where cells can be exposed to controlled mechanical, electrical or chemical cues to directly decouple cell-ECM related behaviors from cell-cell or cell-environmental factors. Utilization of these dynamic synthetic biomaterials will enable discovery of novel mechanisms fundamental in tissue development, homeostasis, repair, and disease. In this dissertation, the primary goal was to evaluate how mechanical changes in the ECM regulate cell motility and polarization responses. This was accomplished through two major aims: 1) by developing a modular image processing tool that could be applied in complex synthetic in vitro microenvironments to asses cell motility dynamics, and 2) to utilize that tool to advance understanding of mechanobiology and mechanotransduction processes associated with development, wound healing, and disease progression. Therefore, the first portion of this thesis (Chapters 2 and 3) dealt with proof of concept for our newly developed automated cell tracking system, termed ACTIVE (automated contour-based tracking for in vitro environments), while the second portion of this thesis (Chapter 4-7) addressed applying this system in multiple experimental designs to synthesize new knowledge regarding cell-ECM or cell-cell interactions. In Chapter 1, we introduced why cell-ECM interactions are essential for in vivo processes and highlighted the current state of the literature. In Chapter 2, we demonstrated that ACTIVE could achieve greater than 95% segmentation accuracy at multiple cell densities, while improving two-body cell-cell interaction error by up to 43%. In Chapter 3 we showed that ACTIVE could be applied to reveal subtle differences in fibroblast motility atop static wrinkled or static non-wrinkled surfaces at multiple cell densities. In Chapters 4 and 5, we characterized fibroblast motility and intracellular reorganization atop a dynamic shape memory polymer biomaterial, focusing on the role of the Rho-mediated pathway in the observed responses. We then utilized ACTIVE to identify differences in subpopulation dynamics of monoculture versus co-culture endothelial and smooth muscle cells (Chapter 6). In Chapter 7, we applied ACTIVE to investigate E. coli biofilm formation atop poly(dimethylsiloxane) surfaces with varying stiffness and line patterns. Finally, we presented a summary and future work in Chapter 8. Collectively, this work highlights the capabilities of the newly developed ACTIVE tracking system and demonstrates how to synthesize new information about mechanobiology and mechanotransduction processes using dynamic biomaterial platforms

The art of video MashUp: supporting creative users with an innovative and smart application

In this paper, we describe the development of a new and innovative tool of video mashup. This application is an easy to use tool of video editing integrated in a cross-media platform; it works taking the information from a repository of videos and puts into action a process of semi-automatic editing supporting users in the production of video mashup. Doing so it gives vent to their creative side without them being forced to learn how to use a complicated and unlikely new technology. The users will be further helped in building their own editing by the intelligent system working behind the tool: it combines semantic annotation (tags and comments by users), low level features (gradient of color, texture and movements) and high level features (general data distinguishing a movie: actors, director, year of production, etc.) to furnish a pre-elaborated editing users can modify in a very simple way

Ultraviolet Radiation of Hypervelocity Stagnation Flows and Shock/Boundary-Layer Interactions

Shock/boundary-layer interactions can induce flow distortion, create flow separation with loss of control authority, and result in high thermal loads. Correct prediction of the flow structure and heating loads is vital for vehicle survival. However, a recent NATO workshop revealed severe underprediction of thermal loads and discrepancies in the location of separation by simulations of high enthalpy air flows. Due to the coupling between thermochemistry and fluid mechanics, a substantial effort has been placed on the development and validation of thermochemical models. As a result, there is a need for experimental data that are more than mean flow surface measurements. Spatially resolved emission spectra are collected in the post-shock regime of hypervelocity flow over a circular cylinder and a 30-55 degree double wedge. The Hypervelocity Expansion Tube (HET) is used to generate high Mach number, high enthalpy flow (Mach numbers 5 - 7, h₀ = 4 - 8 MJ/kg) with minimal freestream dissociation. The NO γ band (A²Σ⁺ - X²Π) emission is measured in the ultraviolet range of 210-250 nm at downstream locations behind shock waves. Excitation temperatures are extracted from the NO γ emission from spectrum fitting. The result is a temperature relaxation profile that quantifies the state thermal non-equilibrium. Profiles of vibrational band intensity as a function of streamwise distance are used as direct measurements of chemical non-equilibrium in the flow. Cylinder experiments are performed with varying freestream total enthalpy, Mach number, and test gas O₂ mole fraction to examine changes in relaxation profile. Schlieren images are used to accurately measure standoff distance. Temperature measurements are compared against a zero-dimensional state-to-state model. Strategies for spectrum fitting are presented for cases where the gas is not optically thin and for radiation containing multiple electronic states. For freestream mixtures with reduced oxygen mole fraction, an electronic excitation temperature is required to describe the radiation of the NO γ, β (B²Π - X²Π), and δ (C²Π - X²Π) transitions. The creation of electronically excited NO is discussed in the context of measured vibrational band intensities and computed NO(A) number density profiles using a two-temperature reactive Landau-Teller model. Emission spectra are collected in the post bow shock and reattachment shock region of hypervelocity flow over a double wedge. High speed schlieren imaging is performed to investigate facility startup effects and for tracking features in a shock/boundary-layer interaction. Detector exposures occur at select times throughout the flow development process to study temporal changes in thermal and chemical non-equilibrium. Time evolution of temperatures at strategic locations of the flow is obtained from spectrum fitting. Two-temperature calculations of the oblique shock system are compared against the emission results. Radiation data are discussed in the context of recent simulation efforts.</p

Resiliency Assessment and Enhancement of Intrinsic Fingerprinting

Intrinsic fingerprinting is a class of digital forensic technology that can detect traces left in digital multimedia data in order to reveal data processing history and determine data integrity. Many existing intrinsic fingerprinting schemes have implicitly assumed favorable operating conditions whose validity may become uncertain in reality. In order to establish intrinsic fingerprinting as a credible approach to digital multimedia authentication, it is important to understand and enhance its resiliency under unfavorable scenarios. This dissertation addresses various resiliency aspects that can appear in a broad range of intrinsic fingerprints. The first aspect concerns intrinsic fingerprints that are designed to identify a particular component in the processing chain. Such fingerprints are potentially subject to changes due to input content variations and/or post-processing, and it is desirable to ensure their identifiability in such situations. Taking an image-based intrinsic fingerprinting technique for source camera model identification as a representative example, our investigations reveal that the fingerprints have a substantial dependency on image content. Such dependency limits the achievable identification accuracy, which is penalized by a mismatch between training and testing image content. To mitigate such a mismatch, we propose schemes to incorporate image content into training image selection and significantly improve the identification performance. We also consider the effect of post-processing against intrinsic fingerprinting, and study source camera identification based on imaging noise extracted from low-bit-rate compressed videos. While such compression reduces the fingerprint quality, we exploit different compression levels within the same video to achieve more efficient and accurate identification. The second aspect of resiliency addresses anti-forensics, namely, adversarial actions that intentionally manipulate intrinsic fingerprints. We investigate the cost-effectiveness of anti-forensic operations that counteract color interpolation identification. Our analysis pinpoints the inherent vulnerabilities of color interpolation identification, and motivates countermeasures and refined anti-forensic strategies. We also study the anti-forensics of an emerging space-time localization technique for digital recordings based on electrical network frequency analysis. Detection schemes against anti-forensic operations are devised under a mathematical framework. For both problems, game-theoretic approaches are employed to characterize the interplay between forensic analysts and adversaries and to derive optimal strategies. The third aspect regards the resilient and robust representation of intrinsic fingerprints for multiple forensic identification tasks. We propose to use the empirical frequency response as a generic type of intrinsic fingerprint that can facilitate the identification of various linear and shift-invariant (LSI) and non-LSI operations

Automatic Emotion Recognition: Quantifying Dynamics and Structure in Human Behavior.

Emotion is a central part of human interaction, one that has a huge influence on its overall tone and outcome. Today's human-centered interactive technology can greatly benefit from automatic emotion recognition, as the extracted affective information can be used to measure, transmit, and respond to user needs. However, developing such systems is challenging due to the complexity of emotional expressions and their dynamics in terms of the inherent multimodality between audio and visual expressions, as well as the mixed factors of modulation that arise when a person speaks. To overcome these challenges, this thesis presents data-driven approaches that can quantify the underlying dynamics in audio-visual affective behavior. The first set of studies lay the foundation and central motivation of this thesis. We discover that it is crucial to model complex non-linear interactions between audio and visual emotion expressions, and that dynamic emotion patterns can be used in emotion recognition. Next, the understanding of the complex characteristics of emotion from the first set of studies leads us to examine multiple sources of modulation in audio-visual affective behavior. Specifically, we focus on how speech modulates facial displays of emotion. We develop a framework that uses speech signals which alter the temporal dynamics of individual facial regions to temporally segment and classify facial displays of emotion. Finally, we present methods to discover regions of emotionally salient events in a given audio-visual data. We demonstrate that different modalities, such as the upper face, lower face, and speech, express emotion with different timings and time scales, varying for each emotion type. We further extend this idea into another aspect of human behavior: human action events in videos. We show how transition patterns between events can be used for automatically segmenting and classifying action events. Our experimental results on audio-visual datasets show that the proposed systems not only improve performance, but also provide descriptions of how affective behaviors change over time. We conclude this dissertation with the future directions that will innovate three main research topics: machine adaptation for personalized technology, human-human interaction assistant systems, and human-centered multimedia content analysis.PhDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/133459/1/yelinkim_1.pd