MPEG-4 natural video coding - An overview

This paper describes the MPEG-4 standard, as defined in ISO/IEC 14496-2. The MPEG-4 visual standard is developed to provide users a new level of interaction with visual contents. It provides technologies to view, access and manipulate objects rather than pixels, with great error robustness at a large range of bit-rates. Application areas range from digital television, streaming video, to mobile multimedia and games. The MPEG-4 natural video standard consists of a collection of tools that support these application areas. The standard provides tools for shape coding, motion estimation and compensation, texture coding, error resilience, sprite coding and scalability. Conformance points in the form of object types, profiles and levels, provide the basis for interoperability. Shape coding can be performed in binary mode, where the shape of each object is described by a binary mask, or in gray scale mode, where the shape is described in a form similar to an alpha channel, allowing transparency, and reducing aliasing. Motion compensation is block based, with appropriate modifications for object boundaries. The block size can be 16×16, or 8×8, with half pixel resolution. MPEG-4 also provides a mode for overlapped motion compensation. Texture coding is based in 8×8 DCT, with appropriate modifications for object boundary blocks. Coefficient prediction is possible to improve coding efficiency. Static textures can be encoded using a wavelet transform. Error resilience is provided by resynchronization markers, data partitioning, header extension codes, and reversible variable length codes. Scalability is provided for both spatial and temporal resolution enhancement. MPEG-4 provides scalability on an object basis, with the restriction that the object shape has to be rectangular. MPEG-4 conformance points are defined at the Simple Profile, the Core Profile, and the Main Profile. Simple Profile and Core Profiles address typical scene sizes of QCIF and CIF size, with bit-rates of 64, 128, 384 and 2 Mbit/s. Main Profile addresses a typical scene sizes of CIF, ITU-R 601 and HD, with bit-rates at 2, 15 and 38.4 Mbit/s

Behavioral analysis of real-time systems with interdependent tasks

The analysis presented in this thesis considers the problem of processing a media stream by a system consisting of a chain of given off-the-shelf software components, executed on a scarce-resource embedded platform. Each component corresponds to a task, and the communication between tasks is buffered. The essential requirement on the physical platform is cost-effectiveness, and the requirement on the system is robustness. These requirements lead to minimizing the resources made available to the system to the limit that it remains robust. In the context of this work the robustness criteria for a system are meeting the system real-time constraints. The real-time constraints come from the fact that media processing systems must display audio/video information at a certain rate in order to avoid audio/video artefacts. This implies that the degree in which the real-time constraints are met directly influences the quality of service provided by the system. To ensure that the timing constraints are met, the chain is provided with a guaranteed resource budget. Within the chain, the tasks are scheduled using fixed priority scheduling. Due to the dependencies between the tasks, and their different behaviors, it is difficult to predict the behavior of the chain. Hence, it is difficult to determine the minimum needed resource budget, to predict chain response time, to minimize buffer sizes and context switch overhead, and to reason about chain composition. The current practice in the domain of media processing systems lacks a theoretical underpinning that helps designers and developers beyond intuition and experience. Such a theory is also needed to control the chain behavior at design time, to make sure timing requirements are met even in overload situations. This thesis provides an underlying theory that helps engineers to reason rigorously about system behavior and associated resource needs. It starts from the experimental observation that, under certain conditions, a media processing chain assumes a repetitive behavior, the stable phase, after a finite initial phase. Starting from this observation a theoretical model for the execution of streaming chains in media processing systems is built. The general strategy is to analyze streaming systems in an incremental manner starting from a simple theoretical case, to realistic streaming chains that include branching and more complex types of components. 213 214 The approach allows calculating the execution order of the components in a chain, expressed as a trace of actions taken by each component. The analysis formally proves that the behavior of the chain can be expressed as a unique trace, which, under certain conditions imposed at chain design time, assumes a repetitive pattern after a finite prefix. The trace is completely determined by the individual traces of the components, the computation times of the component actions, the topology of the chain, the capacities of the communication buffers, and the static priorities of the components. Given the computation times for each action in the trace, the associated schedule can be derived. The unique trace of actions and the schedule prove an excellent starting point for further analysis. Start times and response times of the individual components and the complete chain are immediately available. The number of context switches, and the position of the context switches in the component traces, which is an indicator for their overhead cost, can be extracted from the trace. Aside of that, the theory provides corollaries showing how to design the system such that each of the above parameters can be optimized. Also, given the individual traces of the components and the channel constraints, the minimum necessary and sufficient capacities for each buffer in the chain are calculated. Finally, the analysis shows the conditions to be imposed at design time such that even in overload situations the chain satisfies (during an infinite suffix of the unique trace) its real-time constraints that influence directly the quality of service provided

Workload model for video decoding and its applications

Ph.DDOCTOR OF PHILOSOPH

A robust mechanism for adaptive scheduling of multimedia applications

We propose an adaptive scheduling technique to schedule highly dynamic multimedia tasks on a CPU. We use a combination of two techniques: the first one is a feedback mechanism to track the resource requirements of the tasks based on local observations. The second one is a mechanism that operates with a global visibility, reclaiming unused bandwidth. The combination proves very effective: resource reclaiming increases the robustness of the feedback, while the identification of the correct bandwidth made by the feedback increases the effectiveness of the reclamation. We offer both theoretical results and an extensive experimental validation of the approach

Algorithms for compression of high dynamic range images and video

The recent advances in sensor and display technologies have brought upon the High Dynamic Range (HDR) imaging capability. The modern multiple exposure HDR sensors can achieve the dynamic range of 100-120 dB and LED and OLED display devices have contrast ratios of 10^5:1 to 10^6:1. Despite the above advances in technology the image/video compression algorithms and associated hardware are yet based on Standard Dynamic Range (SDR) technology, i.e. they operate within an effective dynamic range of up to 70 dB for 8 bit gamma corrected images. Further the existing infrastructure for content distribution is also designed for SDR, which creates interoperability problems with true HDR capture and display equipment. The current solutions for the above problem include tone mapping the HDR content to fit SDR. However this approach leads to image quality associated problems, when strong dynamic range compression is applied. Even though some HDR-only solutions have been proposed in literature, they are not interoperable with current SDR infrastructure and are thus typically used in closed systems. Given the above observations a research gap was identified in the need for efficient algorithms for the compression of still images and video, which are capable of storing full dynamic range and colour gamut of HDR images and at the same time backward compatible with existing SDR infrastructure. To improve the usability of SDR content it is vital that any such algorithms should accommodate different tone mapping operators, including those that are spatially non-uniform. In the course of the research presented in this thesis a novel two layer CODEC architecture is introduced for both HDR image and video coding. Further a universal and computationally efficient approximation of the tone mapping operator is developed and presented. It is shown that the use of perceptually uniform colourspaces for internal representation of pixel data enables improved compression efficiency of the algorithms. Further proposed novel approaches to the compression of metadata for the tone mapping operator is shown to improve compression performance for low bitrate video content. Multiple compression algorithms are designed, implemented and compared and quality-complexity trade-offs are identified. Finally practical aspects of implementing the developed algorithms are explored by automating the design space exploration flow and integrating the high level systems design framework with domain specific tools for synthesis and simulation of multiprocessor systems. The directions for further work are also presented

Soluções de broadcast para redes 4G

Mestrado em Engenharia Electrónica e de TelecomunicaçõesA primeira difusão de conteúdos video e audio teve um forte impacto no quotidiano da população que assistiu a uma revolução nos modelos de transmissão de informação e de entretenimento. A evolução desde então foi significativa, e já na era digital, encontramo-nos face a uma nova sub-elevação da metodologia e do conceito subjacentes à transmissão de conteudos multimédia. O mundo actual apresenta, contudo, diferentes requisitos, de entre os quais se destacam a procura pela alta definição e mobilidade. A mobilidade tem sido um particular foco de atenção por parte dos operadores que exploram agora modelos para entregar uma vasta gama de serviços que sejam atractivos para os utilizadores. Esta dissertação apresenta um sumário das tecnologias emergentes de broadcast que se distinguem nas várias partes do mundo com a sua particular incidência geográfica, características e cenários de aplicação. É ainda apresentada uma arquitectura 4G abordando assuntos inerentes à mobilidade e qualidade de serviço com particular incidência nos aspectos relacionados com a integração de uma tecnologia de broadcast particular. Para avaliação da arquitectura proposta foram efectuados estudos com base num equipamento de broadcast na sua versão comercial, permitindo desta forma obter uma análise que ilustra o que os operadores podem esperar do estado actual dos dispositivos. Os resultados permitiram retirar ilações sobre o comportamento de um equipamento considerado como um produto final a disponibilizar aos operadores, quando integrado num ambiente 4G com suporte de mobilidade e QoS. Nomeadamente é discutida a sua aplicabildiade tendo em linha de conta as desvantagens introduzidas pelas características inerentes à própria tecnologia.Broadcast of video and audio through analogical television completely changed the paradigm of information and entertainment divulgation. Today, in the “digital era”, the Analogue Switch Off revolution is being held. Manufacturers and operators already show concerns regarding the support of mobility, quality of experience and of service. Delivering competitive High Definition contents and providing solutions for the average “on-the-move” user are two of the most important issues to be dealt by the service providers, which are also within the analysis scope of this work. This dissertation presents an overview on the most relevant broadcast technologies which are assumed to be of relative acceptance in their respective target market. It presents their main characteristics and applicability. 4G architectural concepts are also analyzed, closely dealing with mobility and quality of service provisioning, with particular focus on the seamless integration of broadcast technologies. As a mean to evaluate the feasibility of integrating broadcast technologies with 4G architectures, a performance evaluation study was performed using commercial equipment. In this way a several set of considerations constructed illustrating the features and functionalities which operators can expect or disregard from professional commercial broadcasting devices. Results allow the withdrawing of conclusions concerning the integration of a final broadcasting solution when incorporated within a 4G environment with QoS and mobility support. Its applicability is evaluated having in mind the performance drawbacks introduced by the specific technology, and generalized towards the gathering of more general conclusions which consider the main characteristics of the commercial broadcasting devices

On Transmission System Design for Wireless Broadcasting

This thesis considers aspects related to the design and standardisation of transmission systems for wireless broadcasting, comprising terrestrial and mobile reception. The purpose is to identify which factors influence the technical decisions and what issues could be better considered in the design process in order to assess different use cases, service scenarios and end-user quality. Further, the necessity of cross-layer optimisation for efficient data transmission is emphasised and means to take this into consideration are suggested. The work is mainly related terrestrial and mobile digital video broadcasting systems but many of the findings can be generalised also to other transmission systems and design processes. The work has led to three main conclusions. First, it is discovered that there are no sufficiently accurate error criteria for measuring the subjective perceived audiovisual quality that could be utilised in transmission system design. Means for designing new error criteria for mobile TV (television) services are suggested and similar work related to other services is recommended. Second, it is suggested that in addition to commercial requirements there should be technical requirements setting the frame work for the design process of a new transmission system. The technical requirements should include the assessed reception conditions, technical quality of service and service functionalities. Reception conditions comprise radio channel models, receiver types and antenna types. Technical quality of service consists of bandwidth, timeliness and reliability. Of these, the thesis focuses on radio channel models and errorcriteria (reliability) as two of the most important design challenges and provides means to optimise transmission parameters based on these. Third, the thesis argues that the most favourable development for wireless broadcasting would be a single system suitable for all scenarios of wireless broadcasting. It is claimed that there are no major technical obstacles to achieve this and that the recently published second generation digital terrestrial television broadcasting system provides a good basis. The challenges and opportunities of a universal wireless broadcasting system are discussed mainly from technical but briefly also from commercial and regulatory aspectSiirretty Doriast

Smart PIN: performance and cost-oriented context-aware personal information network

The next generation of networks will involve interconnection of heterogeneous individual networks such as WPAN, WLAN, WMAN and Cellular network, adopting the IP as common infrastructural protocol and providing virtually always-connected network. Furthermore, there are many devices which enable easy acquisition and storage of information as pictures, movies, emails, etc. Therefore, the information overload and divergent content’s characteristics make it difficult for users to handle their data in manual way. Consequently, there is a need for personalised automatic services which would enable data exchange across heterogeneous network and devices. To support these personalised services, user centric approaches for data delivery across the heterogeneous network are also required. In this context, this thesis proposes Smart PIN - a novel performance and cost-oriented context-aware Personal Information Network. Smart PIN's architecture is detailed including its network, service and management components. Within the service component, two novel schemes for efficient delivery of context and content data are proposed: Multimedia Data Replication Scheme (MDRS) and Quality-oriented Algorithm for Multiple-source Multimedia Delivery (QAMMD). MDRS supports efficient data accessibility among distributed devices using data replication which is based on a utility function and a minimum data set. QAMMD employs a buffer underflow avoidance scheme for streaming, which achieves high multimedia quality without content adaptation to network conditions. Simulation models for MDRS and QAMMD were built which are based on various heterogeneous network scenarios. Additionally a multiple-source streaming based on QAMMS was implemented as a prototype and tested in an emulated network environment. Comparative tests show that MDRS and QAMMD perform significantly better than other approaches