Adaptive video delivery using semantics

The diffusion of network appliances such as cellular phones, personal digital assistants and hand-held computers has created the need to personalize the way media content is delivered to the end user. Moreover, recent devices, such as digital radio receivers with graphics displays, and new applications, such as intelligent visual surveillance, require novel forms of video analysis for content adaptation and summarization. To cope with these challenges, we propose an automatic method for the extraction of semantics from video, and we present a framework that exploits these semantics in order to provide adaptive video delivery. First, an algorithm that relies on motion information to extract multiple semantic video objects is proposed. The algorithm operates in two stages. In the first stage, a statistical change detector produces the segmentation of moving objects from the background. This process is robust with regard to camera noise and does not need manual tuning along a sequence or for different sequences. In the second stage, feedbacks between an object partition and a region partition are used to track individual objects along the frames. These interactions allow us to cope with multiple, deformable objects, occlusions, splitting, appearance and disappearance of objects, and complex motion. Subsequently, semantics are used to prioritize visual data in order to improve the performance of adaptive video delivery. The idea behind this approach is to organize the content so that a particular network or device does not inhibit the main content message. Specifically, we propose two new video adaptation strategies. The first strategy combines semantic analysis with a traditional frame-based video encoder. Background simplifications resulting from this approach do not penalize overall quality at low bitrates. The second strategy uses metadata to efficiently encode the main content message. The metadata-based representation of object's shape and motion suffices to convey the meaning and action of a scene when the objects are familiar. The impact of different video adaptation strategies is then quantified with subjective experiments. We ask a panel of human observers to rate the quality of adapted video sequences on a normalized scale. From these results, we further derive an objective quality metric, the semantic peak signal-to-noise ratio (SPSNR), that accounts for different image areas and for their relevance to the observer in order to reflect the focus of attention of the human visual system. At last, we determine the adaptation strategy that provides maximum value for the end user by maximizing the SPSNR for given client resources at the time of delivery. By combining semantic video analysis and adaptive delivery, the solution presented in this dissertation permits the distribution of video in complex media environments and supports a large variety of content-based applications

The Sentient Stage: The Theatrical Uncanny in Contemporary Performance

This dissertation identifies and explores a genre of performance I term the “theatrical uncanny.” Intermingling aesthetics and psychology, the uncanny is the realm of “intellectual uncertainty” (for Ernst Jentsch) and “the familiar made strange” (for Sigmund Freud); it is an obscure but palpable disruption of our expectations. The genre of the uncanny has received a great deal of scholarly attention in both film and literature, but has, by contrast, been minimally explored in performance. This neglect is particularly striking considering theatre itself can be viewed as inherently uncanny. The repetitions and representations of performance, its interdependence of the real and the imaginary, imply a kind of ever-present déjà vu. The very pervasiveness of this quality can in fact render it insensible. The genre of uncanny performance is therefore a valuable designation for works that actively elicit this psycho-emotional response. The productions I study are remarkable for their ability to capitalize on theatre’s uncanny potential. I demonstrate that this category of performance is not limited to any particular style or status, locating the effect as potently in popular entertainments (such as the junk opera Shockheaded Peter and punchdrunk’s Sleep No More) as in more esoteric or avant-garde work (like that of artists such as the Quay Brothers and Tadeusz Kantor). Drawing on a variety of methods – including phenomenology, psychoanalysis, and cognitive science – I engage with these performances as sensual experience. Through subjective impression and extensive description, I gradually elucidate the scenographic parameters of the theatrical uncanny. The productions that achieve this effect, while greatly disparate in nature, share certain approaches and techniques in common. They position the spectator (physically, emotionally, perceptually) in an unstable relationship to the objects and bodies of the performance, creating the sensation that the inanimate actually possesses its own unique vitality. Uncanny performance interweaves elements of object theatre, memory theatre, and intermediality, but cannot be encompassed by any of these terms in isolation. These performances question our basic qualifications for declaring something ‘live,’ as the term is used both theoretically and colloquially. They ask what it would mean for a memory to behave as an object, or for an object to have memories. This study is both a critique of how the uncanny works on stage and an attempt to rethink the concept of the uncanny through theatre practice. I argue that the uncanny is best understood as an embodied experience, a feeling mediated through and registered within our flesh. It results from unsettling interactions between our bodies and the matter and space around us. The concrete and present spatial relationships of theatre are ideal for exploring these tensions. Through the materiality of theatre, I offer evidence that the uncanny response, rather than being a marginal or naïve interpretation of the world (as it is sometimes portrayed), is actually a fundamental and profoundly productive state of mind

Robust computational intelligence techniques for visual information processing

The third part is exclusively dedicated to the super-resolution of Magnetic Resonance Images. In one of these works, an algorithm based on the random shifting technique is developed. Besides, we studied noise removal and resolution enhancement simultaneously. To end, the cost function of deep networks has been modified by different combinations of norms in order to improve their training. Finally, the general conclusions of the research are presented and discussed, as well as the possible future research lines that are able to make use of the results obtained in this Ph.D. thesis.This Ph.D. thesis is about image processing by computational intelligence techniques. Firstly, a general overview of this book is carried out, where the motivation, the hypothesis, the objectives, and the methodology employed are described. The use and analysis of different mathematical norms will be our goal. After that, state of the art focused on the applications of the image processing proposals is presented. In addition, the fundamentals of the image modalities, with particular attention to magnetic resonance, and the learning techniques used in this research, mainly based on neural networks, are summarized. To end up, the mathematical framework on which this work is based on, ₚ-norms, is defined. Three different parts associated with image processing techniques follow. The first non-introductory part of this book collects the developments which are about image segmentation. Two of them are applications for video surveillance tasks and try to model the background of a scenario using a specific camera. The other work is centered on the medical field, where the goal of segmenting diabetic wounds of a very heterogeneous dataset is addressed. The second part is focused on the optimization and implementation of new models for curve and surface fitting in two and three dimensions, respectively. The first work presents a parabola fitting algorithm based on the measurement of the distances of the interior and exterior points to the focus and the directrix. The second work changes to an ellipse shape, and it ensembles the information of multiple fitting methods. Last, the ellipsoid problem is addressed in a similar way to the parabola

\u27And I am a Material Girl\u27: How Aesthetics and Material Culture Fashion Femininity in Edith Wharton\u27s The Age of Innocence, from Text to Film

This thesis explores the role of aesthetics and material culture in Edith Wharton’s 1920 novel The Age of Innocence and in Martin Scorsese’s 1993 film adaptation. In Wharton’s Old New York, material opulence is arguably the most essential aspect of culture. Newland Archer is the primary authority on fashion and taste within the narrative, and is thus charged with enforcing standards of socially constructed Victorian femininity with regard to his two romantic interests, May Welland and Ellen Olenska. Scorsese’s film uses mise-en-scène to echo the detail rich design aesthetic found in Wharton’s prose; however, the film’s abandonment of Newland’s distinctly masculine perspective in favor of a female narrator restructures the power dynamics found in Wharton’s narrative. Both the novel and film highlight society’s fetishizing of material culture, a systematic obsession rooted in the oppressive qualities of the Victorian social climate. For both the novel and the film, material opulence is powerful within society because it is the only form of self-expression and individual agency that the characters have access to given the standards of repression, especially for women. Materials can only represent identity and experience and are therefore meaningless. Wharton and Scorsese use their works to criticize the tyranny of materialism during the Victorian period

Photography in the Middle: Dispatches on Media Ecologies and Aesthetics

It’s easy to forget there’s a war on when the front line is everywhere encrypted in plain sight. Gathered in this book’s several chapters are dispatches on the role of photography in a War Universe, a space and time in which photographers such as Hilla Becher, Don McCullin and Eadweard Muybridge exist only insofar as they are a mark of possession, in the sway of larger forces. These photographers are conceptual personae that collectively fabulate a different kind of photography, a paraphotography in which the camera produces negative abyssal flashes or ‘endarkenment.’ In his Vietnam War memoir, Dispatches, Michael Herr imagines a ‘dropped camera’ receiving ‘jumping and falling’ images, images which capture the weird indivisibility of medium and mediated in a time of war. The movies and the war, the photographs and the torn bodies, fused and exchanged. Reporting from the chaos at the middle of things, Herr invokes a kind of writing attuned to this experience. Photography in the Middle, eschewing a high theoretical mode, seeks to exploit the bag of tricks that is the dispatch. The dispatch makes no grand statement about the progress of the war. Cultivating the most perverse implications of its sources, it tries to express what the daily briefing never can. Ports of entry in the script we’re given, odd and hasty little glyphs, unhelpful rips in the cover story, dispatches are futile, dark intuitions, an expeditious inefficacy. They are bleak but necessary responses to an indifferent world in which any action whatever has little noticeable effect

Modular Optical Flow Estimation With Applications To Fluid Dynamics

Optical flow is the apparent motion of intensities in an image sequence. Its estimation has been studied for almost three decades. The results can be used in a wealth of possible applications ranging from scientific applications like experimental fluid dynamics over medical imaging to mobile computer games. The development of a single solution for all optical flow problems seems to be a worthwhile goal. However, in this thesis, we argue that this goal is unlikely to be achieved. We thoroughly motivate this hypothesis with theoretical and practical considerations. Based on the results, we identify two major problems that significantly complicate the research and development of new optical flow algorithms: First, very few reference implementations are publicly available. Second, not all relevant properties of the proposed algorithms are described in literature. In the first part of this thesis, our contribution is to alleviate both problems. First, we discuss a number of algorithm properties which should be known by the user. Second, by decomposing existing optical flow methods into their individual algorithm building blocks, shortly called modules, we propose to individually analyze the properties of each module independently. A large number of existing techniques is composed of relatively few existing modules. By implementing these modules in a software library called Charon and adding tools for the evaluation of the results, we contribute to the accessibility of reference implementations and to the possibility of analyzing algorithms by experiments. In the second part of this thesis, we contribute two modules which are vital for the estimation of fluid flows. They are specifically tuned to the imagery obtained for particle tracking velocimetry (PTV). We call the first module estimatibility measure. It detects those particle locations where fluid motion can be estimated. It is based on the constant position of the center of gravity of the connected components generated by a large number of thresholded versions of the original image. The module only needs a few intuitive parameters. Experiments indicate its robustness with respect to noise with varying mean and variance. To analyze the properties of this module we also provide a framework for simulating the particle image generation. The second module is a motion model based on unsupervised learning via principal component analysis. Training data is provided through Computational Fluid Dynamic (CFD) simulations. The model describes local ensembles of trajectories which can be fitted to the image sequence by means of a similarity measure. Together with a standard similarity measure and a simple optimization scheme we derive a new PTV method. Compared to existing techniques, we obtained superior results with respect to accuracy on real and synthetic sequences with known ground truth. All source code developed during the thesis is available as Open Source following the GNU Lesser General Public License (LGPL)

Aerospace medicine and biology: A continuing bibliography with indexes (supplement 338)

This bibliography lists 139 reports, articles and other documents introduced into the NASA Scientific and Technical Information System during June 1990. Subject coverage includes: aerospace medicine and psychology, life support systems and controlled environments, safety equipment, exobiology and extraterrestrial life, and flight crew behavior and performance

Play Among Books

How does coding change the way we think about architecture? Miro Roman and his AI Alice_ch3n81 develop a playful scenario in which they propose coding as the new literacy of information. They convey knowledge in the form of a project model that links the fields of architecture and information through two interwoven narrative strands in an “infinite flow” of real books

Image categorisation using parallel network constructs: an emulation of early human colour processing and context evaluation

PhD ThesisTraditional geometric scene analysis cannot attempt to address the understanding of human vision. Instead it adopts an algorithmic approach, concentrating on geometric model fitting. Human vision, however, is both quick and accurate but very little is known about how the recognition of objects is performed with such speed and efficiency. It is thought that there must be some process both for coding and storage which can account for these characteristics. In this thesis a more strict emulation of human vision, based on work derived from medical psychology and other fields, is proposed. Human beings must store perceptual information from which to make comparisons, derive structures and classify objects. It is widely thought by cognitive psychologists that some form of symbolic representation is inherent in this storage. Here a mathematical syntax is defined to perform this kind of symbolic description. The symbolic structures must be capable of manipulation and a set of operators is defined for this purpose. The early visual cortex and geniculate body are both inherently parallel in operation and simple in structure. A broadly connectionist emulation of this kind of structure is described, using independent computing elements, which can perform segmentation, re-colouring and generation of the base elements of the description syntax. Primal colour information is then collected by a second network which forms the visual topology, colouring and position information of areas in the image as well as a full description of the scene in terms of a more complex symbolic set. The idea of different visual contexts is introduced and a model is proposed for the accumulation of context rules. This model is then applied to a database of natural images.EPSRC CASE award: Neural Computer Sciences,Southampton