182 research outputs found
Exploring Processor and Memory Architectures for Multimedia
Multimedia has become one of the cornerstones of our 21st century society and, when combined with mobility, has enabled a tremendous evolution of our society. However, joining these two concepts introduces many technical challenges. These range from having sufficient performance for handling multimedia content to having the battery stamina for acceptable mobile usage. When taking a projection of where we are heading, we see these issues becoming ever more challenging by increased mobility as well as advancements in multimedia content, such as introduction of stereoscopic 3D and augmented reality. The increased performance needs for handling multimedia come not only from an ongoing step-up in resolution going from QVGA (320x240) to Full HD (1920x1080) a 27x increase in less than half a decade. On top of this, there is also codec evolution (MPEG-2 to H.264 AVC) that adds to the computational load increase. To meet these performance challenges there has been processing and memory architecture advances (SIMD, out-of-order superscalarity, multicore processing and heterogeneous multilevel memories) in the mobile domain, in conjunction with ever increasing operating frequencies (200MHz to 2GHz) and on-chip memory sizes (128KB to 2-3MB). At the same time there is an increase in requirements for mobility, placing higher demands on battery-powered systems despite the steady increase in battery capacity (500 to 2000mAh). This leaves negative net result in-terms of battery capacity versus performance advances. In order to make optimal use of these architectural advances and to meet the power limitations in mobile systems, there is a need for taking an overall approach on how to best utilize these systems. The right trade-off between performance and power is crucial. On top of these constraints, the flexibility aspects of the system need to be addressed. All this makes it very important to reach the right architectural balance in the system. The first goal for this thesis is to examine multimedia applications and propose a flexible solution that can meet the architectural requirements in a mobile system. Secondly, propose an automated methodology of optimally mapping multimedia data and instructions to a heterogeneous multilevel memory subsystem. The proposed methodology uses constraint programming for solving a multidimensional optimization problem. Results from this work indicate that using todayâs most advanced mobile processor technology together with a multi-level heterogeneous on-chip memory subsystem can meet the performance requirements for handling multimedia. By utilizing the automated optimal memory mapping method presented in this thesis lower total power consumption can be achieved, whilst performance for multimedia applications is improved, by employing enhanced memory management. This is achieved through reduced external accesses and better reuse of memory objects. This automatic method shows high accuracy, up to 90%, for predicting multimedia memory accesses for a given architecture
Efficiency in audio processing : filter banks and transcoding
Audio transcoding is the conversion of digital audio from one compressed form A to another compressed form B, where A and B have different compression properties, such as a different bit-rate, sampling frequency or compression method. This is typically achieved by decoding A to an intermediate uncompressed form, and then encoding it to B. A significant portion of the involved computational effort pertains to operating the synthesis filter bank, which is an important processing block in the decoding stage, and the analysis filter bank, which is an important processing block in the encoding stage. This thesis presents methods for efficient implementations of filter banks and audio transcoders, and is separated into two main parts. In the first part, a new class of Frequency Response Masking (FRM) filter banks is introduced. These filter banks are usually characterized by comprising a tree-structured cascade of subfilters, which have small individual filter lengths. Methods of complexity reduction are proposed for the scenarios when the filter banks are operated in single-rate mode, and when they are operated in multirate mode; and for the scenarios when the input signal is real-valued, and when it is complex-valued. An efficient variable bandwidth FRM filter bank is designed by using signed-powers-of-two reduction of its subfilter coefficients. Our design has a complexity an order lower than that of an octave filter bank with the same specifications. In the second part, the audio transcoding process is analyzed. Audio transcoding is modeled as a cascaded quantization process, and the cascaded quantization of an input signal is analyzed under different conditions, for the MPEG 1 Layer 2 and MP3 compression methods. One condition is the input-to-output delay of the transcoder, which is known to have an impact on the audio quality of the transcoded material. Methods to reduce the error in a cascaded quantization process are also proposed. An ultra-fast MP3 transcoder that requires only integer operations is proposed and implemented in software. Our implementation shows an improvement by a factor of 5 to 16 over other best known transcoders in terms of execution speed
DRM analysis using a simulator of multiprocessor embedded system
Mestrado em Engenharia Electrónica e TelecomunicaçÔesOs sistemas multiprocessador são uma tecnologia emergente. O projecto
Hijdra, que estĂĄ a ser desenvolvido na âNXP semiconductors
Researchâ Ă© um sistema multiprocessador de tempo real que corre aplicaçÔes
com constrangimentos do tipo âhardâ e âsoftâ. Nestes sistemas,
os processadores comunicam atravĂ©s de uma rede de silĂcio. As aplicaçÔes
que correm no sistema multiprocessador consistem em mĂșltiplas
tarefas que correm em processadores embutidos. Achar soluçÔes para o
mapeamento das tarefas é o maior problema destes sistemas. Uma aplicação
para este sistema que tem vindo a ser estudada Ă© o âCar Radioâ.
Esta dissertação diz respeito a uma aplicação de rådio digital (DRM)
na arquitectura Hijdra. Neste contexto, uma aplicação de um receptor de
DRM foi estudada. Um modelo de anĂĄlise de âData Flowâ foi extraĂdo a
partir da aplicação, foi estudada a latĂȘncia introduzida na rede de silĂcio
pela introdução de um novo processador (acelerador de Viterbi) e foi
estudada a possibilidade do mapeamento das vårias tarefas da aplicação
em diferentes processadores a correr em paralelo.
Muitas estratégias ainda ficaram por definir a fim de optimizar o
desempenho da aplicação do receptor de DRM de modo a esta poder
trabalhar de uma forma mais eficaz.
ABSTRACT: Multiprocessor systems are an emerging technology. The Hijdra project
being developed at NXP semiconductors Research is a
multiprocessor system running with both hard and soft real time streaming
media jobs. These jobs consist of multiple tasks running on
embedded multiprocessors. Finding good solutions for job mapping is the
main problem of these systems. One application which has being studied
for Hijdra is the âCar Radioâ.
This thesis concerns the study of a digital radio receptor application
(DRM) in Hijdra architecture. In this context, a data flow model of analysis
was extracted from the application, the latency introduced by the
addition of a new tile (Viterbi accelerator) and eventual speed gains were
studied and the possibility of mapping the different tasks of the application
in different processors was foreseen.
Many strategies were yet to be defined in order to optimize the application
performance so it can work more effectively in the multiprocessor
system
Quality of Service Controlled Multimedia Transport Protocol
PhDThis research looks at the design of an open transport protocol that supports a range of
services including multimedia over low data-rate networks. Low data-rate multimedia
applications require a system that provides quality of service (QoS) assurance and flexibility.
One promising field is the area of content-based coding. Content-based systems use an array
of protocols to select the optimum set of coding algorithms. A content-based transport
protocol integrates a content-based application to a transmission network.
General transport protocols form a bottleneck in low data-rate multimedia
communicationbsy limiting throughpuot r by not maintainingt iming requirementsT. his work
presents an original model of a transport protocol that eliminates the bottleneck by
introducing a flexible yet efficient algorithm that uses an open approach to flexibility and
holistic architectureto promoteQ oS.T he flexibility andt ransparenccyo mesi n the form of a
fixed syntaxt hat providesa seto f transportp rotocols emanticsT. he mediaQ oSi s maintained
by defining a generic descriptor. Overall, the structure of the protocol is based on a single
adaptablea lgorithm that supportsa pplication independencen, etwork independencea nd
quality of service.
The transportp rotocol was evaluatedth rougha set of assessmentos:f f-line; off-line
for a specific application; and on-line for a specific application. Application contexts used
MPEG-4 test material where the on-line assessmenuts eda modified MPEG-4 pl; yer. The
performanceo f the QoSc ontrolledt ransportp rotocoli s often bettert hano thers chemews hen
appropriateQ oS controlledm anagemenatl gorithmsa re selectedT. his is shownf irst for an
off-line assessmenwt here the performancei s compared between the QoS controlled
multiplexer,a n emulatedM PEG-4F lexMux multiplexers chemea, ndt he targetr equirements.
The performanceis also shownt o be better in a real environmentw hen the QoS controlled
multiplexeri s comparedw ith the real MPEG-4F lexMux scheme
Platforms for handling and development of audiovisual data
EstĂĄgio realizado na MOG Solutions e orientado por VĂtor TeixeiraTese de mestrado integrado. Engenharia InformĂĄtca e Computação. Faculdade de Engenharia. Universidade do Porto. 200
Algorithms and methods for video transcoding.
Video transcoding is the process of dynamic video adaptation. Dynamic video adaptation can be defined as the process of converting video from one format to another, changing the bit rate, frame rate or resolution of the encoded video, which is mainly necessitated by the end user requirements. H.264 has been the predominantly used video compression standard for the last 15 years. HEVC (High Efficiency Video Coding) is the latest video compression standard finalised in 2013, which is an improvement over H.264 video compression standard. HEVC performs significantly better than H.264 in terms of the Rate-Distortion performance. As H.264 has been widely used in the last decade, a large amount of video content exists in H.264 format. There is a need to convert H.264 video content to HEVC format to achieve better Rate-Distortion performance and to support legacy video formats on newer devices. However, the computational complexity of HEVC encoder is 2-10 times higher than that of H.264 encoder. This makes it necessary to develop low complexity video transcoding algorithms to transcode from H.264 to HEVC format. This research work proposes low complexity algorithms for H.264 to HEVC video transcoding. The proposed algorithms reduce the computational complexity of H.264 to HEVC video transcoding significantly, with negligible loss in Rate-Distortion performance. This work proposes three different video transcoding algorithms. The MV-based mode merge algorithm uses the block mode and MV variances to estimate the split/non-split decision as part of the HEVC block prediction process. The conditional probability-based mode mapping algorithm models HEVC blocks of sizes 16Ă16 and lower as a function of H.264 block modes, H.264 and HEVC Quantisation Parameters (QP). The motion-compensated MB residual-based mode mapping algorithm makes the split/non-split decision based on content-adaptive classification models. With a combination of the proposed set of algorithms, the computational complexity of the HEVC encoder is reduced by around 60%, with negligible loss in Rate-Distortion performance, outperforming existing state-of-art algorithms by 20-25% in terms of computational complexity. The proposed algorithms can be used in computation-constrained video transcoding applications, to support video format conversion in smart devices, migration of large-scale H.264 video content from host servers to HEVC, cloud computing-based transcoding applications, and also to support high quality videos over bandwidth-constrained networks
Bi & tri dimensional scene description and composition in the MPEG-4 standard
MPEG-4 is a new ISO/IEC standard being developed by MPEG (Moving Picture Experts Group). The standard is to be released in November 1998 and version 1 will be an International Standard in January 1999 The MPEG-4 standard addresses the new demands that arise in a world in which more and more audio-visual material is exchanged in digital form MPEG-4 addresses the coding of objects of various types. Not only traditional video and audio frames, but also natural video and audio objects as well as textures, text, 2- and 3-dimensional graphic primitives, and synthetic music
and sound effects.
Using MPEG-4 to reconstruct an audio-visual scene at a terminal, it is hence no longer sufficient to encode the raw audio-visual data and transmit it, as MPEG-2 does m order to synchronize video and audio. In MPEG-4, all objects are multiplexed together at the encoder and transported to the terminal Once de-multiplexed, these objects are composed at the terminal to construct and present to the end user a meaningful audio-visual scene. The placement of these elementary audio-visual objects in space and time is described in the scene description of a scene. While the action of putting these objects together in the same representation space is the composition of audio-visual objects.
My research was concerned with the scene description and composition of the audio-visual objects that are defined in an audio-visual scene Scene descriptions are coded independently irom sticams related to primitive audio-visual objects. The set of parameters belonging to the scene description are differentiated from the parameters that are used to improve the coding efficiency of an object. While the independent coding of different objects may achieve a higher compression rate, it also brings the ability to manipulate content at the terminal. This allows the modification of the scene description parameters without having to decode the primitive audio-visual objects themselves. This approach allows the development of a syntax that describes the spatio-temporal relationships of audio-visual scene objects. The behaviours of objects and their response to user inputs can thus also be represented in the scene description, allowing richer audio-visual content to be delivered as an MPEG-4 stream
HbbTV-compliant Platform for Hybrid Media Delivery and Synchronization on Single- and Multi-Device Scenarios
[EN] The combination of broadcast and broadband
(hybrid) technologies for delivering TV related
media contents can bring fascinating opportunities. It is
motivated by the large amount and diversity of media contents,
together with the ubiquity and multiple connectivity capabilities
of modern consumption devices. This paper presents an
end-to-end platform for the preparation, delivery, and synchronized
consumption of related hybrid (broadcast/broadband)
media contents on a single device and/or on multiple close-by
devices (i.e., a multi-device scenario). It is compatible with the
latest version of the Hybrid Broadcast Broadband TV (HbbTV)
standard (version 2.0.1). Additionally, it provides adaptive and
efficient solutions for key issues not specified in that standard,
but that are necessary to successfully deploy hybrid and multidevice
media services. Moreover, apart from MPEG-DASH and
HTML5, which are the broadband technologies adopted by
HbbTV, the platform also provides support for using HTTP Live
Streaming and Real-time Transport Protocol and its companion
RTP Control Protocol broadband technologies. The presented
platform can provide support for many hybrid media services.
In this paper, in order to evaluate it, the use case of multi-device
and multi-view TV service has been selected. The results of both
objective and subjective assessments have been very satisfactory,
in terms of performance (stability, smooth playout, delays,
and sync accuracy), usability of the platform, usefulness of
its functionalities, and the awaken interest in these kinds of
platforms.This work was supported in part by the "Fondo Europeo de Desarrollo Regional" and in part by the Spanish Ministry of Economy and Competitiveness through R&D&I Support Program under Grant TEC2013-45492-R.Boronat, F.; Marfil-Reguero, D.; Montagud, M.; Pastor Castillo, FJ. (2017). HbbTV-compliant Platform for Hybrid Media Delivery and Synchronization on Single- and Multi-Device Scenarios. IEEE Transactions on Broadcasting. 1-26. https://doi.org/10.1109/TBC.2017.2781124S12
Recommended from our members
Multimedia delivery in the future internet
The term âNetworked Mediaâ implies that all kinds of media including text, image, 3D graphics, audio
and video are produced, distributed, shared, managed and consumed on-line through various networks,
like the Internet, Fiber, WiFi, WiMAX, GPRS, 3G and so on, in a convergent manner [1]. This white
paper is the contribution of the Media Delivery Platform (MDP) cluster and aims to cover the Networked
challenges of the Networked Media in the transition to the Future of the Internet.
Internet has evolved and changed the way we work and live. End users of the Internet have been confronted
with a bewildering range of media, services and applications and of technological innovations concerning
media formats, wireless networks, terminal types and capabilities. And there is little evidence that the pace
of this innovation is slowing. Today, over one billion of users access the Internet on regular basis, more
than 100 million users have downloaded at least one (multi)media file and over 47 millions of them do so
regularly, searching in more than 160 Exabytes1 of content. In the near future these numbers are expected
to exponentially rise. It is expected that the Internet content will be increased by at least a factor of 6, rising
to more than 990 Exabytes before 2012, fuelled mainly by the users themselves. Moreover, it is envisaged
that in a near- to mid-term future, the Internet will provide the means to share and distribute (new)
multimedia content and services with superior quality and striking flexibility, in a trusted and personalized
way, improving citizensâ quality of life, working conditions, edutainment and safety.
In this evolving environment, new transport protocols, new multimedia encoding schemes, cross-layer inthe
network adaptation, machine-to-machine communication (including RFIDs), rich 3D content as well as
community networks and the use of peer-to-peer (P2P) overlays are expected to generate new models of
interaction and cooperation, and be able to support enhanced perceived quality-of-experience (PQoE) and
innovative applications âon the moveâ, like virtual collaboration environments, personalised services/
media, virtual sport groups, on-line gaming, edutainment. In this context, the interaction with content
combined with interactive/multimedia search capabilities across distributed repositories, opportunistic P2P
networks and the dynamic adaptation to the characteristics of diverse mobile terminals are expected to
contribute towards such a vision.
Based on work that has taken place in a number of EC co-funded projects, in Framework Program 6 (FP6)
and Framework Program 7 (FP7), a group of experts and technology visionaries have voluntarily
contributed in this white paper aiming to describe the status, the state-of-the art, the challenges and the way
ahead in the area of Content Aware media delivery platforms
- âŠ