International audienceThe progressive needs for using video streaming for different applications in varied domains have created a new set of heterogeneous environments. In order to get the best performance of such environments, video streaming has to be adapted to the different parameters that characterize these environments, namely: bandwidth, CPU, GPU, screen resolution, etc. In this paper, we define a new web service, named Wava (Web service for Automatic Video Data Flows Adaptation). Wava allows multimedia platform to adjust the adaptation at two levels: the static level during initialization and the dynamic level according to the fluctuation of the environment

C. Jung

C. Prehofer

G. Zhai

J.-G. Kim

Y. Wang

Crossref

WAVA: A New Web Service for Automatic Video Data Flow Adaptation in Heterogeneous Collaborative Environments

Aupet, Jean-Baptiste

Kassab, Rami

Lapayre, Jean-Christophe

HAL-CentraleSupelec

WAVA: a New Web Service for Automatic Video DataFlow Adaptation in Heterogeneous CollaborativeEnvironmentsJean-Baptiste Aupet, Rami Kassab, Jean-Christophe LapayreTo cite this version:Jean-Baptiste Aupet, Rami Kassab, Jean-Christophe Lapayre. WAVA: a New Web Servicefor Automatic Video Data Flow Adaptation in Heterogeneous Collaborative Environments.CDVE’09: 6th Int. Conf. on Cooperative Design, Visualization and Engineering, 2009, Lux-embourg. pp.280–288, 2009. <hal-00563302>HAL Id: hal-00563302https://hal.archives-ouvertes.fr/hal-00563302Submitted on 4 Feb 2011HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestine´e au de´poˆt et a` la diffusion de documentsscientifiques de niveau recherche, publie´s ou non,e´manant des e´tablissements d’enseignement et derecherche franc¸ais ou e´trangers, des laboratoirespublics ou prive´s.WAVA: a New Web Service for Automatic VideoData Flow Adaptation in HeterogeneousCollaborative EnvironmentsJ.-B. Aupet, R. Kassab, and J.-C. LapayreLIFC - EA 4269, Universite´ de Franche-Comte´,16, Rte de Gray - 25030 BESANCON Cedex, France.[jbaupet,rkassab,jclapayre]@lifc.univ-fcomte.frWeb page: http://lifc.univ-fcomte.frAbstract. The progressive needs of video streaming for different ap-plications in varied domains have created a new set of heterogeneousenvironments especially in Virtual Collaborative Environments. In or-der to get the best performance of such environments, video streaminghas to be adapted to the different parameters that characterize theseenvironments, namely: bandwidth, CPU, GPU, screen resolution, etc.In this paper, we define a new Web Service, named Wava (Web Servicefor Automatic Video Data Flows Adaptation). Wava allows multimediaplatforms and collaborative application servers to adjust the adaptationat two levels: at the static level during initialization and at the dynamiclevel according to the variation of the environment.Keywords : Adaptation, Asynchronous, Collaborative Environments, Multime-dia, Real-time, Streaming, Synchronous, Terminal, Video, Web service.1 IntroductionMultimedia environments providing access to media contents from heterogeneousenvironments are commonly called UMA ”Universal Multimedia Access”. In het-erogeneous environments systems clients differ in their hardware and softwarecapabilities and limitations. These differences might be found on the commu-nication level, for example the variation of bandwidth from 50 kb to 20 Mb.Variations may also involve the terminal specifications e.g. CPU, memory, dis-play capabilities and GPU encoding-decoding capacities. Therefore when wetalk about video adaptation systems we should take all conditions into account.By heterogeneous applications we mean the variation of usage, entertainmentor work (for example tele-medicine) and programming languages. These appli-cations are distinguished beetween two types: synchronous and asynchronousapplications. In order to solve these differences, and because we mostly work onthe Internet, XML files and Web Services are involved.In the first section of this paper we present the state of the art in this domain.In the following section, we define our new Web Service Wava (Web Service forAutomatic Video Adaptation) which allows platforms to adapt the stream attwo levels: at the static level (during initialization) and at the dynamic levelaccording to the variation of the environment, and its performance. The lastsection presents the conclusion and future work.2 State of the ArtThe development of multimedia applications puts in evidence the necessity ofnew research. The arrival of HD video by the Internet and the increase of multipleformats of video generate a lot of studies in each part of this domain. In thissection, we focus our study on adapatative systems and especially on UMA(Universal Media Access) systems based (Wang 2007).2.1 Adaptation with the UMA ModelThe General architecture of the UMA system consists of four main componentsdescribed in (Wang 2007) and illustrated in the figure 1:Fig. 1. UMA Adaptation System Fig. 2. System ChartMultimedia serverResearch which focuses on the adaptation aspect performed by the multimediaserver (Prehofer 1999) considers the server is responsible of coding and formingthe high quality Scalable Video bitstream (Sprljan 2005). That means that itcan be scaled (adapted) to be in adequacy with specifications. In addition, theserver can adapt the video. However, the adaptation process consumes largeramount of calculation power than a real time system can actually bear. In orderto further adapt the video bitstream in a convenient way, the middleware has todecode and then to recode the bitstream to the adapted format. Therefore, theserver transmits specific bitstream description helping the middleware to avoidthe decoding-recoding step. The server role in adaptation system can be sum-marized to, forming scalable bitstream (Sprljan 2005), performing adaptation,and transmitting video information with the SVC (figure 2).Middleware.The main purpose of middleware is the adaptation process, it receives the scal-able bitstream, operates the adaptation and then sends the scaled bitstreamto the terminal. In real time systems, studies are concentrated on middlewareand proposed adaptation solutions like UF (Utility Function) (Chang 2002),(Kim 2005), (Wang 2007) or gBSD (generic Bitstream Synt. Desc.) (Iqbal 2007).The main factor used in the adaptation process is the constraints (Feedback)received from the terminal. The feedback consists of terminal-transmitted infor-mation necessary for helping the middleware in the adaptation process to choosethe right video bitstream.Communication systemThe revolution of communication systems in the last decade has created variousmedia, paved the way for real-time Collaborative systems. From this viewpoint,it is very important to analyze the communication mediums for the adaptation.Terminals can use a medium to communicate and pass to an other. This stepis the main reason for performing the adaptation. For example, a mobile clientcould change his Internet connection from GPRS to Wi-Fi (Ruiz 2002). If thesystem allows a client to notify the system of these changes, it is called anadaptable system. If the system is able to notify itself about the changes of theenvironment, it is called an adaptive system.TerminalIn addition to decoding the video, terminals have one of the most importanttasks in adaptation systems that is creating feedback. Terminal application hasto calculate the feedback and send it to the middleware or the server, which inturn should make the right decision. Many factors can influence the feedback,such as computation capacity of the terminal or free resources to store video andothers. The most important resources to take into account are: Bandwidth andits variation, the available RAM and Hard disk capacity to store buffered video,the CPU capacity and load, ang GPU Capacity for Displayable Resolution andcodecs supported Software.Feedback is the major factor to choose the adapted video, many techniquesare used to calculate it: produced as a mark from the terminal, created eitherby testing the hardware of a terminal or by using mathematical methods suchas PSNR (Wang 2004) or VQA (Zhai 2008) (see part 2.4).2.2 Use of adaptation in video applicationsCollaborative environments are used more and more due to the increase of datatransmission capacity. Now a user can easily be noticed by chat or voice-chat, andmore and more by video-conferencing. Video applications vary from entertain-ment purposes like watching films on VOD and channels of television but also formedical applications, such as video conferencing for Telediagnosis(Jung 2005),e-learning. . . These applications are classified in two main categories, real time(synchronous) video applications and stored video (asynchronous) applications.Stored video applications Thess kind of applications are easier to perform,there is no need to consider the time factor in all parts of the system. A videois available on the server and the application has to adapt it with terminalcapabilities.Real time applications These applications need special solutions to ensurethe time contract and quality of service are respected. Collaborative applicationsmust ensure synchronous data exchange and consistency. The obvious the goalof the middleware is: saving time for the adaptation process.2.3 Multidimensional video adaptationIn terms of performance, the adaptation process has three main factors or threemain dimensions which are: time, resolution, and quality. And they are referredas: temporal, spatial and SNR, respectively:– Temporal (Kropfberger 2004) and (Leopold 2003): Frame rate (from 1 up to30 fps fig. 3a.)– Spatial: Screen and GPU Resolution capabilities (figure 3b.)– SNR: Signal-to-Noise Ratio (Low, mid, high): Extended for video by PSNRand VQA in depth.Fig. 3. Bandwidth consumption depending on: a. Frame rate and b. Resolution2.4 PSNR and VQA in depthPSNR: Peak Signal to Noise Ratio is a method to calculate the noiseratio of a signal depending on the measurement of MSE (Mean Squared error)(Wang 2004). However, PSNR does not compute the real performance of a videostream. For this reason, it is not applicable to measure the performance of themultidimensional video adaptation process. Furthermore, it has been proventhat there can be different videos with different qualities that have the samePSNR. These experiments was correlated by HVS (Human Visual System) in(Zhai 2008), which depends on human notice for giving the result. HVS is alsocalled perceptually VQA.VQA: Video Quality Assessment is a value that is calculated in subjec-tive(HVS) or objective (mathematical) manner (Zhai 2008). In this paper, wefocus on the objective method in calculating VQA, because it can be calculatedmathematically. The differences between these VQA types are the amount ofvideo features sent with the adapted video.3 A New Web Service for Automatic Video Adaptation:WAVAWhen users have to communicate, they do not have to be impacted by theweaknesses of the others. Clients does not feel the latency of the adaptationprocess. But hey can not determine themselves the optimal feedback all the time.With regard to this, we propose a solution that automatically and periodicallydetects the changes of the environment without user’s interaction. In order toprovide transparent processing, enhance the performance and facilitate the usageof the system, we propose the following architecture.3.1 WAVA ArchitectureIn the beginning of our work, we used a client-server JAVA application, wherethe client sends his mark manually and the server reacts according to this mark.We automatized this process by making a simple detection of the load of theCPU and the network congestion. In another experience, we created a client ap-plication that reads the terminal performances (CPU frequency and load, Screenresolution, network bandwidth) and sends the corresponding mark to the server.The latency generated by the calculation did not allow an efficient and satis-factory work. Furthermore in a collaborative environments, heterogeneous clientcan connect to the system. The best solution to resolve this problem is the use ofWeb Services. Because of their portability and independence from programminglanguages, Web Services can handle any kind of platforms. Furthermore, we savethe feedback calculation time of the terminal. When the client chooses a video,the collaborative system automatically detects terminals performances with theWAVA marks and sends back adapted videos.Fig. 4. WAVA ProcessingSo we propose a new Web Service design that detects each terminal propertyand calculates the related mark. Depending on this mark, the server decideswhether it sends a high or low quality video matching the terminal’s exigences.The quality is determined by the decrease of the resolution and frame rate.It is possible to integrate this system in a web page with a script that readsthe terminal performance and sends an XML file in a Virtual Awareness Cardformat like in (Aupet 2009) to the WAVA web service. Subsequently WAVA usesan algorithm to produce the corresponding mark. At the media server we havetwo choices. The first is to make a video pool that contains all the videos inall the codec formats, which match with all the marks: the same video will berepeated for all formats. The second choice is to use one video source that wouldbe converted in real-time either by the server or by the middleware to adapt itto the marks received from the Web Service. We have chosen the second solutionfor the video conferences, where we have the camera as the only source. So wedirectly adapt the stream to match the feedback.3.2 WAVA ProcessingThe Mark represents the terminal performance. We classify CPUs, graphic cards,resolutions, and bandwidth into groups. Each group contains the individualsthat perform in an identical manner in terms of video streaming. Then, a markis attached to each group. We apply this design by using a database which storesdevices attaching to their marks. The database is automatically filled with theunknown devices of terminal by giving a mark calculated with the average markof the terminal performance. We distinguish two categories of mark: the staticmark that does not change during a session, we call them Global Marks (Mg).This mark refers to CPU (Mcpu), resolution (Mres), and graphic card (Mgrph),and obviously remain unchangeable during one session. Mg is calculated as theminimum value of Mcpu, Mres and Mgrph. The second category of mark is thedynamic mark that refers to the bandwidth (Mbw) that changes during the videostreaming session. In order to get the final mark (M), we calculate the minimumvalue between Mg and Mbw.Fig. 5. Wava AlgorithmWe always use the smallest value because we need the system work optimally:we adapt our collaborative system to the weakest value. After calculating themark, WAVA sends it and saves the global mark. Periodically WAVA receivesthe bandwidth information of users, calculate the new Mbw, then calculates themark which remains the minimum between the Mg and the updated Mbw.3.3 WAVA PerformanceIn order to test our system, we make a pool of videos in different formats to testseveral terminals. With these videos we have tested the proportion of packetloss during the streaming transfer. We used the Mpeg-H.264 codec to producethese videos in five different qualities. We changed the resolution and frame rate.Results are in the figure 6a.Periodically the bandwidth mark is sent and the new adaptation mark is up-dated to provide the best video quality that fits the real conditions of the user.We describe a scenario-based test (fig. 7) to show the decrease of packets lossby using WAVA Web Service. We can show the technical performance of WAVAin figure 6b. In facts, when the adaptation is well done, participants can workefficiently.Fig. 6. a.Packets loss - b.Wava PerformanceFig. 7. Scenario-based Test4 Conclusion and future workIn this paper, we showed that video adaptation is one of the main matters ofmultimedia development in heterogeneous collaborative environments. Thereforeresearche has been conducted in this domain which focused on the UMA model.We proposed a new Web Service based system for automatic video adaptation(WAVA). The Web Service is used to calculate the best adaptation that matchesthe system capabilities, for both, static parameters (during initialization) anddynamic parameters depending on the variation of the environment. We imple-mented and tested our system, and we exposed the performance results. Wavasignificantly improves the use of mobile terminals for collaborative work.Our future work focuses on how to test Wava not only with a mark for eachterminal, but also for a whole collaborative environment. We imagine a processto adapt media during a session of medical tele-diagnosis between remote mobilepractitioners: we are working on new distributed algorithms for shared markscalculation and optimising adaptation to each participant.5 AcknowledgementsThe authors thank the European Community (European FEDER) for financingthis work by the InterRegIV project SERVASTIC : Teleneurology in MobileEnvironments.References(Aupet 2009) J.-B. Aupet, N. Elmarzouqi, E. Garcia, and J.-C. Lapayre, Virtual Aware-ness Card for adaptability in Collaborative Virtual Environments, In SETIT 2009,5th Int. Conf. on Sciences of Electronic, Technologies of Information and Telecom-munications, pp.243, Mar.(Chang 2002) Shih-Fu Chang, Optimal Video Adaptation and Skimming Using aUtility-Based Framework, Proc. IWDC-2002, pp.20 ,Capri Island, Italy.(Iqbal 2006) Razib Iqbal, Shervin Shirmohammadi and Chris Joslin, MPEG-21 BasedTemporal Adaptation of Live H.264 Video, 8th IEEE ISM’06, pp.457–464, Dec.(Iqbal 2007) Razib Iqbal, Shervin Shirmohammadi, and Abdulmotaleb El Saddik, AFramework for MPEG-21 DIA Based Adaptation and Perceptual Encryption ofH.264 Video, Multimedia computing and networking, vol.6504(03), pp.1–12, (2007).(Jung 2005) Chulho Jung, Sanghee Lee, and Eunseok Lee, Dynamic Adaptive Architec-ture for Self-adaptation in VideoConferencing System, School of Information andCommunication Engineering, LNCS 3768, Springer-Verlag, pp.36–47, Heidelberg.(Kim 2005) Jae-Gon Kim,Yong Wang,Shih-Fu Chang,Hyung-Myung Kim, An optimalframework of video adaptation and its application to rate adaptation transcoding,ETRI Journal, vol.27(4), pp.341–354, Aug.(Kropfberger 2004) Michael Kropfberger, Klaus Leopold, and Hermann Hellwagner,Quality Variations of Different Priority-based Temporal Video Adaptation Algo-rithms, IEEE Workshop on Multimedia Signal Processing No6, pp.183–186, Sept.(Leopold 2003) Klaus Leopold, Hermann Hellwagner, Michael Kropfbereer, QualityControlled Temporal Video Adaptation, SPIE proceedings series Congress Internetmultimedia management systems IV, Vol. 5242, pp.163–174, Sept.(Prehofer 1999) Christian Prehofer, Wolfgang Kellerer, Robert Hirschfeld, HendrikBerndt, Katsuya Kawamura, An Architecture Supporting Adaptation and Evolu-tion in Fourth Generation Mobile Communication Systems, Journal of Communi-cations and Networks, vol.4, pp.336–343, Jan.(Ruiz 2002) Pedro M. Ruiz, Emilio Garcia, Improving User-Perceived QoS in Mobileand Wireless IP Networks Using Real-Time Adaptative Multimedia Applications,Proceedings of PIMRC2002, pp.1467–1471, Sept.(Sprljan 2005) N. Sprljan, M. Mrak, G. C. K. Abhayaratne, E. Izquierdo, A ScalableCoding Framework for Efficient Video Adaptation. 6th WIAMIS, Vol.1, Apr.(Wang 2004) Yong Wang, Shih-Fu Chang, Alexander C. Loui, Subjective Preferenceof Spatio-Temporal Rate in Video Adaptation Using Multi-Dimensional ScalableCoding, IEEE ICME, vol.3, p.1719–1722, Jun.(Wang 2007) Yong Wang, Jae-Gon Kim, Shih-Fu Chang, and Hyung-Myung Kim,Utility-Based Video Adaptation for Universal Multimedia Access (UMA) andContent-Based Utility Function Prediction for Real-Time Video Transcoding, IEEETransactions on Multimedia, vol.9(2), pp.213–220, Feb.(Zhai 2008) Guangtao Zhai, Jianfei Cai, Weisi Lin, Xiaokang Yang, Wenjun Zhang,Three Dimensional Scalable Video Adaptation via User-end Perceptual QualityAssessment, IEEE Transactions on Broadcasting, vol.54, pp.719–727,Sept.

{WAVA}: a New Web Service for Automatic Video Data Flow Adaptation in Heterogeneous Collaborative Environments

Abstract. The progressive needs of video streaming for different ap-plications in varied domains have created a new set of heterogeneous environments especially in Virtual Collaborative Environments. In or-der to get the best performance of such environments, video streaming has to be adapted to the different parameters that characterize these environments, namely: bandwidth, CPU, GPU, screen resolution, etc. In this paper, we define a new Web Service, named Wava (Web Servic

J. -b. Aupet

R. Kassab

J. -c. Lapayre

CiteSeerX

Wava: a new web service for automatic video data flow adaptation in heterogeneous collaborative environments

HAL - Université de Franche-Comté

http://hal-onera.archives-ouvertes.fr/docs/00/56/33/02/PDF/akl09_ip.pdf

{WAVA}: a New Web Service for Automatic Video Data Flow Adaptation in Heterogeneous Collaborative Environments

Abstract

Similar works

Full text

Available Versions

Crossref

HAL-CentraleSupelec

CiteSeerX

HAL - Université de Franche-Comté