Optimal H.264/AVC video transcoding system

This paper presents an efficient receiver-aware video transcoding system that systematically chooses the optimal transcoding operation from multiple options while meeting network and user constraints. Multi-objective optimization is used to select the best transcoding method that minimizes transcoding complexity and memory usage while ensuring the client constraints of bitrate and requested quality are fulfilled

Adaptation des images et des vidéos pour des utilisateurs multiples dans des environnements hétérogènes

La dernière décennie a connu l'émergence de l'utilisation des équipements mobiles comme les assistants personnels et les téléphones, ainsi que la prolifération des réseaux personnels favorisée par le développement considérable dans les technologies de communications. D'autre part, l'information véhiculée a travers le World Wide Web devient de plus en plus visuelle (images et videos) grâce à la numérisation. Afin de permettre à tous les usagers un accès universel à cette information visuelle dans un environnement caractérisé par la diversité des équipements et l'hétérogénéité des réseaux, il devient nécessaire d'adapter les documents multimédia. L'adaptation consiste à appliquer une ou plusieurs transformations sur un document multimédia. Dans ce cadre, plusieurs travaux ont été élaborés en partant de différentes formulations. Nous pensons qu'un système d'adaptation efficace doit choisir les traitements nécessaires à appliquer sur un document visuel afin de maximiser la satisfaction de l'usager. Il doit considérer conjointement les caractéristiques de cet usager ainsi que les performances de son équipement, la qualité de sa connexion et les conditions de son environnement. La majorité des travaux réalisés dans ce domaine n'ont traité que des cas limités, par exemple ajuster une vidéo pour la capacité d'un réseau donné. Dans la présente recherche, nous proposons une solution globale obtenue à l'aide d'un modèle probabiliste qui utilise les traitements des images et des vidéos et l'extraction des caractéristiques des contenus

A selective approach for energy-aware video content adaptation decision-taking engine in android based smartphone

Rapid advancement of technology and their increasing affordability have transformed mobile devices from a means of communication to tools for socialization, entertainment, work and learning. However, advancement of battery technology and capacity is slow compared to energy need. Viewing content with high quality of experience will consume high power. In limited available energy, normal content adaptation system will decrease the content quality, hence reducing quality of experience. However, there is a need for optimizing content quality of experience (QoE) in a limited available energy. With modification and improvement, content adaptation may solve this issue. The key objective of this research is to propose a framework for energy-aware video content adaptation system to enable video delivery over the Internet. To optimise the QoE while viewing streaming video on a limited available smartphone energy, an algorithm for energy-aware video content adaptation decision-taking engine named EnVADE is proposed. The EnVADE algorithm uses selective mechanism. Selective mechanism means the video segmented into scenes and adaptation process is done based on the selected scenes. Thus, QoE can be improved. To evaluate EnVADE algorithm in term of energy efficiency, an experimental evaluation has been done. Subjective evaluation by selected respondents are also has been made using Absolute Category Rating method as recommended by ITU to evaluate EnVADE algorithm in term of QoE. In both evaluation, comparison with other methods has been made. The results show that the proposed solution is able to increase the viewing time of about 14% compared to MPEG-DASH which is an official international standard and widely used streaming method. In term of QoE subjective test, EnVADE algorithm score surpasses the score of other video streaming method. Therefore, EnVADE framework and algorithm has proven its capability as an alternative technique to stream video content with higher QoE and lower energy consumption

Semantischer Schutz und Personalisierung von Videoinhalten. PIAF: MPEG-kompatibles Multimedia-Adaptierungs-Framework zur Bewahrung der vom Nutzer wahrgenommenen Qualität.

UME is the notion that a user should receive informative adapted content anytime and anywhere. Personalization of videos, which adapts their content according to user preferences, is a vital aspect of achieving the UME vision. User preferences can be translated into several types of constraints that must be considered by the adaptation process, including semantic constraints directly related to the content of the video. To deal with these semantic constraints, a fine-grained adaptation, which can go down to the level of video objects, is necessary. The overall goal of this adaptation process is to provide users with adapted content that maximizes their Quality of Experience (QoE). This QoE depends at the same time on the level of the user's satisfaction in perceiving the adapted content, the amount of knowledge assimilated by the user, and the adaptation execution time. In video adaptation frameworks, the Adaptation Decision Taking Engine (ADTE), which can be considered as the "brain" of the adaptation engine, is responsible for achieving this goal. The task of the ADTE is challenging as many adaptation operations can satisfy the same semantic constraint, and thus arising in several feasible adaptation plans. Indeed, for each entity undergoing the adaptation process, the ADTE must decide on the adequate adaptation operator that satisfies the user's preferences while maximizing his/her quality of experience. The first challenge to achieve in this is to objectively measure the quality of the adapted video, taking into consideration the multiple aspects of the QoE. The second challenge is to assess beforehand this quality in order to choose the most appropriate adaptation plan among all possible plans. The third challenge is to resolve conflicting or overlapping semantic constraints, in particular conflicts arising from constraints expressed by owner's intellectual property rights about the modification of the content. In this thesis, we tackled the aforementioned challenges by proposing a Utility Function (UF), which integrates semantic concerns with user's perceptual considerations. This UF models the relationships among adaptation operations, user preferences, and the quality of the video content. We integrated this UF into an ADTE. This ADTE performs a multi-level piecewise reasoning to choose the adaptation plan that maximizes the user-perceived quality. Furthermore, we included intellectual property rights in the adaptation process. Thereby, we modeled content owner constraints. We dealt with the problem of conflicting user and owner constraints by mapping it to a known optimization problem. Moreover, we developed the SVCAT, which produces structural and high-level semantic annotation according to an original object-based video content model. We modeled as well the user's preferences proposing extensions to MPEG-7 and MPEG-21. All the developed contributions were carried out as part of a coherent framework called PIAF. PIAF is a complete modular MPEG standard compliant framework that covers the whole process of semantic video adaptation. We validated this research with qualitative and quantitative evaluations, which assess the performance and the efficiency of the proposed adaptation decision-taking engine within PIAF. The experimental results show that the proposed UF has a high correlation with subjective video quality evaluation.Der Begriff "Universal Multimedia Experience" (UME) beschreibt die Vision, dass ein Nutzer nach seinen individuellen Vorlieben zugeschnittene Videoinhalte konsumieren kann. In dieser Dissertation werden im UME nun auch semantische Constraints berücksichtigt, welche direkt mit der Konsumierung der Videoinhalte verbunden sind. Dabei soll die Qualität der Videoerfahrung für den Nutzer maximiert werden. Diese Qualität ist in der Dissertation durch die Benutzerzufriedenheit bei der Wahrnehmung der Veränderung der Videos repräsentiert. Die Veränderung der Videos wird durch eine Videoadaptierung erzeugt, z.B. durch die Löschung oder Veränderung von Szenen, Objekten, welche einem semantischen Constraints nicht entsprechen. Kern der Videoadaptierung ist die "Adaptation Decision Taking Engine" (ADTE). Sie bestimmt die Operatoren, welche die semantischen Constraints auflösen, und berechnet dann mögliche Adaptierungspläne, die auf dem Video angewandt werden sollen. Weiterhin muss die ADTE für jeden Adaptierungsschritt anhand der Operatoren bestimmen, wie die Vorlieben des Nutzers berücksichtigt werden können. Die zweite Herausforderung ist die Beurteilung und Maximierung der Qualität eines adaptierten Videos. Die dritte Herausforderung ist die Berücksichtigung sich widersprechender semantischer Constraints. Dies betrifft insbesondere solche, die mit Urheberrechten in Verbindung stehen. In dieser Dissertation werden die oben genannten Herausforderungen mit Hilfe eines "Personalized video Adaptation Framework" (PIAF) gelöst, welche auf den "Moving Picture Expert Group" (MPEG)-Standard MPEG-7 und MPEG-21 basieren. PIAF ist ein Framework, welches den gesamten Prozess der Videoadaptierung umfasst. Es modelliert den Zusammenhang zwischen den Adaptierungsoperatoren, den Vorlieben der Nutzer und der Qualität der Videos. Weiterhin wird das Problem der optimalen Auswahl eines Adaptierungsplans für die maximale Qualität der Videos untersucht. Dafür wird eine Utility Funktion (UF) definiert und in der ADTE eingesetzt, welche die semantischen Constraints mit den vom Nutzer ausgedrückten Vorlieben vereint. Weiterhin ist das "Semantic Video Content Annotation Tool" (SVCAT) entwickelt worden, um strukturelle und semantische Annotationen durchzuführen. Ebenso sind die Vorlieben der Nutzer mit MPEG-7 und MPEG-21 Deskriptoren berücksichtigt worden. Die Entwicklung dieser Software-Werkzeuge und Algorithmen ist notwendig, um ein vollständiges und modulares Framework zu erhalten. Dadurch deckt PIAF den kompletten Bereich der semantischen Videoadaptierung ab. Das ADTE ist in qualitativen und quantitativen Evaluationen validiert worden. Die Ergebnisse der Evaluation zeigen unter anderem, dass die UF im Bereich Qualität eine hohe Korrelation mit der subjektiven Wahrnehmung von ausgewählten Nutzern aufweist

A scalable approach to video summarization and adaptation

Tesis doctoral inédita. Universidad Autónoma de Madrid, Escuela Politécnica Superior, octubre de 201

Adaptive video delivery using semantics

The diffusion of network appliances such as cellular phones, personal digital assistants and hand-held computers has created the need to personalize the way media content is delivered to the end user. Moreover, recent devices, such as digital radio receivers with graphics displays, and new applications, such as intelligent visual surveillance, require novel forms of video analysis for content adaptation and summarization. To cope with these challenges, we propose an automatic method for the extraction of semantics from video, and we present a framework that exploits these semantics in order to provide adaptive video delivery. First, an algorithm that relies on motion information to extract multiple semantic video objects is proposed. The algorithm operates in two stages. In the first stage, a statistical change detector produces the segmentation of moving objects from the background. This process is robust with regard to camera noise and does not need manual tuning along a sequence or for different sequences. In the second stage, feedbacks between an object partition and a region partition are used to track individual objects along the frames. These interactions allow us to cope with multiple, deformable objects, occlusions, splitting, appearance and disappearance of objects, and complex motion. Subsequently, semantics are used to prioritize visual data in order to improve the performance of adaptive video delivery. The idea behind this approach is to organize the content so that a particular network or device does not inhibit the main content message. Specifically, we propose two new video adaptation strategies. The first strategy combines semantic analysis with a traditional frame-based video encoder. Background simplifications resulting from this approach do not penalize overall quality at low bitrates. The second strategy uses metadata to efficiently encode the main content message. The metadata-based representation of object's shape and motion suffices to convey the meaning and action of a scene when the objects are familiar. The impact of different video adaptation strategies is then quantified with subjective experiments. We ask a panel of human observers to rate the quality of adapted video sequences on a normalized scale. From these results, we further derive an objective quality metric, the semantic peak signal-to-noise ratio (SPSNR), that accounts for different image areas and for their relevance to the observer in order to reflect the focus of attention of the human visual system. At last, we determine the adaptation strategy that provides maximum value for the end user by maximizing the SPSNR for given client resources at the time of delivery. By combining semantic video analysis and adaptive delivery, the solution presented in this dissertation permits the distribution of video in complex media environments and supports a large variety of content-based applications

Contributions to multimedia adaptation within the MPEG-21 framework

Tesis doctoral inédita. Universidad Autónoma de Madrid, Escuela Politécnica Superior, octubre de 201

Investigation Report on Universal Multimedia Access

Universal Multimedia Access (UMA) refers to the ability to access by any user to the desired multimedia content(s) over any type of network with any device from anywhere and anytime. UMA is a key framework for multimedia content delivery service using metadata. This investigation report analyzes the state-of-the-art technologies in UMA and tries to identify the key issues of UMA. The state-of-the-art in multimedia content adaptation, an overview of the standards that supports the UMA framework, potential privacy problems in UMA systems and some new UMA applications are presented in this report. This report also provides challenges that still remain to be resolved in UMA to make clear the potential key problems in UMA and determine which ones to solve