Adaptive video delivery using semantics

The diffusion of network appliances such as cellular phones, personal digital assistants and hand-held computers has created the need to personalize the way media content is delivered to the end user. Moreover, recent devices, such as digital radio receivers with graphics displays, and new applications, such as intelligent visual surveillance, require novel forms of video analysis for content adaptation and summarization. To cope with these challenges, we propose an automatic method for the extraction of semantics from video, and we present a framework that exploits these semantics in order to provide adaptive video delivery. First, an algorithm that relies on motion information to extract multiple semantic video objects is proposed. The algorithm operates in two stages. In the first stage, a statistical change detector produces the segmentation of moving objects from the background. This process is robust with regard to camera noise and does not need manual tuning along a sequence or for different sequences. In the second stage, feedbacks between an object partition and a region partition are used to track individual objects along the frames. These interactions allow us to cope with multiple, deformable objects, occlusions, splitting, appearance and disappearance of objects, and complex motion. Subsequently, semantics are used to prioritize visual data in order to improve the performance of adaptive video delivery. The idea behind this approach is to organize the content so that a particular network or device does not inhibit the main content message. Specifically, we propose two new video adaptation strategies. The first strategy combines semantic analysis with a traditional frame-based video encoder. Background simplifications resulting from this approach do not penalize overall quality at low bitrates. The second strategy uses metadata to efficiently encode the main content message. The metadata-based representation of object's shape and motion suffices to convey the meaning and action of a scene when the objects are familiar. The impact of different video adaptation strategies is then quantified with subjective experiments. We ask a panel of human observers to rate the quality of adapted video sequences on a normalized scale. From these results, we further derive an objective quality metric, the semantic peak signal-to-noise ratio (SPSNR), that accounts for different image areas and for their relevance to the observer in order to reflect the focus of attention of the human visual system. At last, we determine the adaptation strategy that provides maximum value for the end user by maximizing the SPSNR for given client resources at the time of delivery. By combining semantic video analysis and adaptive delivery, the solution presented in this dissertation permits the distribution of video in complex media environments and supports a large variety of content-based applications

Advanced Sensing and Control for Connected and Automated Vehicles

Connected and automated vehicles (CAVs) are a transformative technology that is expected to change and improve the safety and efficiency of mobility. As the main functional components of CAVs, advanced sensing technologies and control algorithms, which gather environmental information, process data, and control vehicle motion, are of great importance. The development of novel sensing technologies for CAVs has become a hotspot in recent years. Thanks to improved sensing technologies, CAVs are able to interpret sensory information to further detect obstacles, localize their positions, navigate themselves, and interact with other surrounding vehicles in the dynamic environment. Furthermore, leveraging computer vision and other sensing methods, in-cabin humans’ body activities, facial emotions, and even mental states can also be recognized. Therefore, the aim of this Special Issue has been to gather contributions that illustrate the interest in the sensing and control of CAVs

A Systematic Survey of ML Datasets for Prime CV Research Areas-Media and Metadata

The ever-growing capabilities of computers have enabled pursuing Computer Vision through Machine Learning (i.e., MLCV). ML tools require large amounts of information to learn from (ML datasets). These are costly to produce but have received reduced attention regarding standardization. This prevents the cooperative production and exploitation of these resources, impedes countless synergies, and hinders ML research. No global view exists of the MLCV dataset tissue. Acquiring it is fundamental to enable standardization. We provide an extensive survey of the evolution and current state of MLCV datasets (1994 to 2019) for a set of specific CV areas as well as a quantitative and qualitative analysis of the results. Data were gathered from online scientific databases (e.g., Google Scholar, CiteSeerX). We reveal the heterogeneous plethora that comprises the MLCV dataset tissue; their continuous growth in volume and complexity; the specificities of the evolution of their media and metadata components regarding a range of aspects; and that MLCV progress requires the construction of a global standardized (structuring, manipulating, and sharing) MLCV "library". Accordingly, we formulate a novel interpretation of this dataset collective as a global tissue of synthetic cognitive visual memories and define the immediately necessary steps to advance its standardization and integration

The Importance of the Non-Important Re-Orientations

A multi-layered exploration of three central issues regarding the production of my artistic works. The combination of chance, non-conscious cognition combined with biological, social, and political constraints generate strategies that produce new conceptual readings and reframe modes of existence. Secondly, the mediation of these entanglements through instrumentality complements the ways in which artistic awareness is grounded in multiple body states, that seek the formation of new ways of feeling, seeing, thinking, and re-representing. And border thinking, understood as the superposition of multiple fully complex modes of existence and not in betweens. These intricate clusters of relations provoke nonconventional ways to resist material and conceptual hierarchies that in turn spark the creation of experimental artistic works that reflect multiple positioning in the search for authenticity or autonomy

Visual object category discovery in images and videos

textThe current trend in visual recognition research is to place a strict division between the supervised and unsupervised learning paradigms, which is problematic for two main reasons. On the one hand, supervised methods require training data for each and every category that the system learns; training data may not always be available and is expensive to obtain. On the other hand, unsupervised methods must determine the optimal visual cues and distance metrics that distinguish one category from another to group images into semantically meaningful categories; however, for unlabeled data, these are unknown a priori. I propose a visual category discovery framework that transcends the two paradigms and learns accurate models with few labeled exemplars. The main insight is to automatically focus on the prevalent objects in images and videos, and learn models from them for category grouping, segmentation, and summarization. To implement this idea, I first present a context-aware category discovery framework that discovers novel categories by leveraging context from previously learned categories. I devise a novel object-graph descriptor to model the interaction between a set of known categories and the unknown to-be-discovered categories, and group regions that have similar appearance and similar object-graphs. I then present a collective segmentation framework that simultaneously discovers the segmentations and groupings of objects by leveraging the shared patterns in the unlabeled image collection. It discovers an ensemble of representative instances for each unknown category, and builds top-down models from them to refine the segmentation of the remaining instances. Finally, building on these techniques, I show how to produce compact visual summaries for first-person egocentric videos that focus on the important people and objects. The system leverages novel egocentric and high-level saliency features to predict important regions in the video, and produces a concise visual summary that is driven by those regions. I compare against existing state-of-the-art methods for category discovery and segmentation on several challenging benchmark datasets. I demonstrate that we can discover visual concepts more accurately by focusing on the prevalent objects in images and videos, and show clear advantages of departing from the status quo division between the supervised and unsupervised learning paradigms. The main impact of my thesis is that it lays the groundwork for building large-scale visual discovery systems that can automatically discover visual concepts with minimal human supervision.Electrical and Computer Engineerin

OIL SPILL MODELING FOR IMPROVED RESPONSE TO ARCTIC MARITIME SPILLS: THE PATH FORWARD

Maritime shipping and natural resource development in the Arctic are projected to increase as sea ice coverage decreases, resulting in a greater probability of more and larger oil spills. The increasing risk of Arctic spills emphasizes the need to identify the state-of-the-art oil trajectory and sea ice models and the potential for their integration. The Oil Spill Modeling for Improved Response to Arctic Maritime Spills: The Path Forward (AMSM) project, funded by the Arctic Domain Awareness Center (ADAC), provides a structured approach to gather expert advice to address U.S. Coast Guard (USCG) Federal On-Scene Coordinator (FOSC) core needs for decision-making. The National Oceanic & Atmospheric Administration (NOAA) Office of Response & Restoration (OR&R) provides scientific support to the USCG FOSC during oil spill response. As part of this scientific support, NOAA OR&R supplies decision support models that predict the fate (including chemical and physical weathering) and transport of spilled oil. Oil spill modeling in the Arctic faces many unique challenges including limited availability of environmental data (e.g., currents, wind, ice characteristics) at fine spatial and temporal resolution to feed models. Despite these challenges, OR&R’s modeling products must provide adequate spill trajectory predictions, so that response efforts minimize economic, cultural and environmental impacts, including those to species, habitats and food supplies. The AMSM project addressed the unique needs and challenges associated with Arctic spill response by: (1) identifying state-of-the-art oil spill and sea ice models, (2) recommending new components and algorithms for oil and ice interactions, (3) proposing methods for improving communication of model output uncertainty, and (4) developing methods for coordinating oil and ice modeling efforts

Some of Us Are Looking at the Stars: Japanese Women, Hong Kong Cinema, and Transcultural Fandom

Thesis (Ph.D.) - Indiana University, Communication and Culture, 2011This dissertation offers a historical materialist perspective on the Japanese female fandom of Hong Kong stars that arose in the mid-1980s and peaked in the late 1990s. This fandom was unique among non-diasporic, transnational audiences of Hong Kong cinema for its female composition and its star-centeredness, which together constitute an alternative lens through which to comprehend the meanings and implications of transcultural media fandom. Employing contemporaneous fan-produced writing, film criticism, journalism, and promotional literature for films, media technologies, and transnational travel to Hong Kong in the reconstruction of its discursive surround, the dissertation interrogates the dialectical relationship between fan practices and subjectivities. Through examination of the media discourses that produced this fandom in the Japanese popular imagination, the material means by which fans pursued their interest in Hong Kong stars, and the intersection of fan affect and transnationally-situated experience, the dissertation makes the case for a pragmatics of transcultural fandom that accounts for not only its transnational socio-political context, but also the gender and popular/fan cultural contexts through which it was experienced and understood

Space and place as expressive categories in videogames

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.This thesis sets out to explore some of the ways in which videogames use space as a means of expression. This expression takes place in two registers: representation and embodiment. Representation is understood as a form of expression in which messages and ideas are communicated. Embodiment is understood as a form of expression in which the player is encouraged to take up a particular position in relation to the game. This distinction between representation and embodiment is useful analytically but the thesis attempts to synthesise these modes in order to account for the experience of playing videogames, where representation and embodiment are constantly happening and constantly influencing and shaping each other. Several methods are developed to analyse games in a way that brings these two modes to the fore. The thesis attempts to arrive at a number of spatial aesthetics of videogames by adapting methods from game studies, literary criticism, phenomenology, onomastics (the study of names), cartographic theory, choreography and architectural and urban formation analysis

Location, Movement and Memory: an Ethnographic study of journeys of asylum seekers and refugees from sub-Saharan Africa into Europe and North East England.

This thesis consists of an ethnographic study of asylum seekers and refugees arriving in North East England from sub-Saharan Africa. The ethnography is created by focussing on the life stories of four individuals known to the author. The information gathered is the result of participant observation in the African community on Teesside over a period of ten years. The study is further developed by focussing on one informant in particular. The author engages critically with Gregory Bateson’s theory of double-bind and this becomes the grounded theory for the study. The author reflects on her own experience as a migrant from South Africa to the United Kingdom. This auto-ethnography explains some of the author’s knowledge, experience and commitments and alerts the reader to possible hermeneutical bias. Further depth is added to the study by the author’s report on her own journey (undertaken with the support of a Churchill Trust travelling scholarship) from the refugee camp in Kakuma (northern Kenya) to Cairo, Tripoli and Malta; and by her accounts of social events in the asylum seeker and refugee community on Teesside. The author identifies a number of important factors in relation to sustaining personal identity in the face of traumatic experiences and forced relocation: memory, language, food, belief and hope. These factors are explored in the relevant literature. The implications of these factors for successful integration into the host community, and the tensions to which they give rise, are explored. It is hoped that this study might help the local host community to better understand the experiences and priorities of asylum seekers and refugees arriving on Teesside and that it might also help to inform local government policy regarding how they are treated. This will become increasingly important as larger numbers of asylum seekers and refugees arrive. The study will need to be broadened to encourage the experiences of Syrian refugees (the largest group currently arriving in Teesside)