The SNE group is an active participant in the CineGrid initiative. We built the first integrated CineGrid portal. In this report we describe the rationale behind the architecture of our solution. We provide a component-wise description of design and functionality

de Laat, C.

Grosso, P.

Knopper, S.

Koning, R.

Roodhart, J.

Search4Dev

Amsterdam CineGrid Exchange: A distributed high-quality digital media solution

NARCIS 

International Migration, Integration and Social Cohesion online publications

UvA-DARE is a service provided by the library of the University of Amsterdam (https://dare.uva.nl)UvA-DARE (Digital Academic Repository)Amsterdam CineGrid Exchange: A distributed high-quality digital media solutionKnopper, S.; Koning, R.; Roodhart, J.; Grosso, P.; de Laat, C.Publication date2009Document VersionFinal published versionLink to publicationCitation for published version (APA):Knopper, S., Koning, R., Roodhart, J., Grosso, P., & de Laat, C. (2009). Amsterdam CineGridExchange: A distributed high-quality digital media solution. (SNE technical report; No. SNE-UVA-2009-1). Fac. Natuurwetenschappen Wiskunde en Informatica.http://staff.science.uva.nl/~grosso/Publications/AmsCineGridExchange.pdfGeneral rightsIt is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s)and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an opencontent license (like Creative Commons).Disclaimer/Complaints regulationsIf you believe that digital publication of certain material infringes any of your rights or (privacy) interests, pleaselet the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the materialinaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letterto: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. Youwill be contacted as soon as possible.Download date:14 Apr 2023Universiteit van Amsterdam System and Network EngineeringAmsterdam CineGrid ExchangeA distributed high-quality digital media solutionSander Knopper,Ralph Koning, Jeroen Roodhart, Paola Grosso and Cees de LaatJuly 30, 2009AbstractThe SNE group is an active participant in the CineGridinitiative. We built the first integrated CineGrid portal.In this report we describe the rationale behind the archi-tecture of our solution. We provide a component-wisedescription of design and functionality.SNE technical report SNE-UVA-2009-01http://www.science.uva.nl/research/sne/reports/1 IntroductionCineGrid[1] is an organization focused on the distribution of very-high-quality digital media over photonic networks. It does this by stimulatingresearch, development and demonstrations within its community. This com-munity consists of researchers in the academic world but also some compa-nies such as film studios.The very-high-quality digital media we’re currently working with is the 4kstandard. 4K offers a resolution of 4096 by 2160 pixels. Table 1 provides ansummary of the characteristics of various media formats.Format X Y Rate Color Frame Frame(fps) (bits/px) (px) (MB)720p HD 1280 720 60 24 921600 2.81080p HD 1920 1080 30 24 2073600 6.22k 2048 1080 24 / 48 36 2211840 10SHD 3840 2160 30 24 8294400 254k 4096 2160 24 36 8847360 40Table 1: Overview of current media formatsCompared to the 1080p HD standard, which is nowadays quite common forhome cinema systems, the resolution of 4k is 4 times larger.Format Flow Stream(MB/s) (Gbit/s)720p HD 170 1.31080p HD 190 1.52k 240 / 480 1.9 / 3.8SHD 750 6.04k 960 7.6Table 2: Network requirements of the various media formatsTable 2 reports the network requirements for transport of each of theseformats. 4K results in a bandwidth of 7.6 Gbit/s (compared to 1.5 Gbit/sfor 1080p); this requires a quite advanced network infrastructure.2 The CineGrid ExchangeThe CineGrid Exchange is the central distribution point in the CineGridcollaboration. The world-wide version is under development. When com-2pleted, it will provide a distributed storage system for digital video contentacross the world.The Exchange relies on the existence of metadata associated with the con-tent. These metadata would include the author, title of the video for in-stance, but also some more technical information like resolution, color depth,etc. This information allows the system to automatically place the videointo the storage system and make arrangements for the video to pop up inthe user interface. It is possible to store the video in different formats (interms of resolution and encoding for instance) and on different sites. In theend, a CineGrid user is not aware of the distributed nature of the storage.He would just be browsing on a website searching for content of his likingand - provided that he has the technical means to receive the stream- startdisplaying the content on his screen.A major technical requirements for the Exchange is extensibility. For ex-ample, different encodings require different media players; mechanisms thatallow to add new types of players related to new encodings seamlessly are amust. The playback functionality has to be fully pluggable. The same holdsfor the storage back-end, so also this part of the system is fully pluggable.3 ArchitectureThere are 3 distinct subsystems in the prototype system we’ve developed:• the web portal• the metadata server• the stream serverFig.1 shows the overall architecture of our Exchange. A metadata andstream server form a pair within a single streaming server. We can havemultiple sites where digital content is publicly available, but there’s only aneed for a single web portal. The web portal retrieves its data from themetadata servers at the different sites and when playback is requested, itnotifies the corresponding streaming server.The global workflow is as follow:1. a user browses the portal. The portal contains information about allthe available content. The portal has access to a database filled withinformation it gets from the metadata server.2. the user selects a movie and decides he wants to display it.3Figure 1: Architectural overview of the Amsterdam CineGrid Exchange3. the portal communicates with the stream server which will then takecare of streaming the content from disk to the display.In the following sections we give a more detailed overview of the three com-ponents.3.1 Streaming serverThe streaming server manages the streaming of content to the display chosenby the user. It supports multiple media players and a queueing algorithmto make sure different media request do not interfere with each other.Its functionalities are easily accessible by XML-RPC. It provides all thestandard functions expected from a streaming server: it can start playingcontent, it allows to skip to another movie, it can stop playing the currentmovie and provide the status. This is accomplished by the use of threads.The whole server is built around the scheduler, which schedules playback forthe destinations (displays) available and holds queues. It periodically checksfor movies that are finished playing and then frees that object and putsanother movie in its place after which it will start that movie. The algorithmbehind the scheduler is quite simple, but can easily be changed. It holds4an array of queues indexed by destination, so each destination (display) hasits own queue. Within this queue are the player objects, these are objectsalready holding all the information needed to start playing the movie whenthe scheduler asks them to. When a new movie is being added, the schedulerchecks the destination and looks if there’s already a queue available for it. Ifthere is, it simply puts the movie at the end of the queue, otherwise it createsa new queue for this destination and puts the movie in it while immediatelygiving it a signal to start playing. A periodic check loops through all thequeues looking for movies that have stopped playing, while also cleaning upany empty queues.When the scheduler starts playback for a specific player object, a new threadstarts. This thread sets some initial variables, usually the parameters for themedia player program, and after that it forks itself. In the forked process itstarts the right media player with the correct parameters. At this point themedia player takes care of actually displaying the content and the threadwill exit eventually. The original process (before the fork) will simply waitfor the new forked proces to finish, after which it will set the internal stateto “finished playing” which will be noticed by the next periodic check of thescheduler.Player objects are actually derived classes from the abstract player class.Each derived class is written to work with a single media player. Currentlythere is a class that works with SAGE[2] (a system used to stream contentto tiled displays) and one that works with VLC. Adding support for newmedia players is relatively easy, simply create a new class derived from theabstract player class and fill in the required methods to properly call themedia player executable.3.2 Metadata serverThe metadata server’s main goal is to gather all the available informationabout the content and save it as if it were a database. A very simple onethough, as you can’t execute complex queries on it, but you can request allthe available information.It consists of 4 main components:• XML-RPC server, which communicates with the outer world• metadata database, which holds all data about the content• metadata consumer, which converts available metadata to an internalformat• file system watcher, which keeps track of file changes5The XML-RPC server main functionality is to send the complete databaseto other parties. Beside that, it can also be used to retrieve the date of thelast update, so another party can figure out whether they need to updateor not.The metadata database is just a thread-safe container for single metadataobjects. Basically it’s a large array with some added functionality to makesure operations from the XML-RPC server and the file system watcher don’tcause data inconsistencies. This is important, since each of them runs in itsown thread.The interface of the metadata consumer is described in an abstract class, thisallows for multiple methods of harvesting metadata. Currently there’s onlyone implementation, which is a regular file system based one. In simply readsXML files of a mounted file system and converts the data to the internalrepresentation that will be put into the database.The file system watcher is an optional but quite useful component. It onlyworks in combination with the file system metadata consumer describedabove. It watches the file system where the content is placed for modifica-tions. This could be when a new file has been created, an existing file hasbeen modified or a file has been deleted. If such a modification occurs, thefile system watcher notifies the metadata consumer to re-read the metadatathus effectively updating the database. The file system watcher also has anabstract interface, currently only an inotify implementation exists for useunder GNU/Linux, new implementations are relatively easy to write.3.3 Web portalThe web portal is the only component the user sees and works with directly.It periodically checks for any updates the metadata servers might have, itthen fetches all the metadata from that server and does this for all associ-ated servers. It will then put all information into a local SQL database forfast lookup and support for complex queries, mostly required by the searchfunctionality.When a user enters the portal it sees the recently added movies along witha search field on top of the page. The user can search for specific keywordsin the title of the movie for example, but also filter the output by author orvideo format.After choosing a movie, a new page shows all the information about themovie and most importantly, all the formats the movie is available in. For-mats are a combination of resolution and encoding, this lets low-bandwidthusers also make use of the system for example. When the preferred format is6known, a page where the destination can be supplied shows. At the momentthis is a combo box filled with pre-defined values by the administrator. Inother words, filled with known displays.The final step consists of the portal looking up the corresponding streamingserver of the movie after which it will send an XML-RPC request to startplayback or at least put it in a queue. The user sees a confirmation of whathappened.4 About languages, libraries and frameworksThe metadata and streaming server are completely written in C++, makingextensive use of the C++ STL for strings, arrays, queues etc. Both pro-grams come with an autotools-based build system, providing compile-timedetection of available libraries. Such as inotify for example.The SAGE[2] library is used by bvmplayer, a media player to display un-compressed or DXT encoded content.VLC[3] is a media player, used to display content in all other encodings.xmlrpc-c is an XML-RPC library for C/C++ which is heavily used in boththe metadata and streaming server.Django[5] is a Python-based web framework which we’ve used for the webportal.75 DemonstrationsThe preliminary version of this architecture has also been used at NortelNetworks Advanced Technology Summit (ATS) and SuperComputing 2008(SC08)[6]. The preliminary version consisted of a very early implementationof the streaming server and no metadata server. The metadata had to beentered manually through the admin interface into the SQL database.At ATS the focus of the demo was to demonstrate streaming very highquality video content using Provider Backbone Transport (PBT)[10] over atransatlantic link from our servers in Amsterdam to a display in Ottawa totest the Quality of Service properties of PBT. We used the Web portal toinitiate the video streams and to select the proper display. We got somegood results from this.A week later at SuperComputing 2008 (SC08)[6] the same version of theportal was used as a part of a HPDMnet[12] demo. It involved streaminghigh quality content from our servers in Amsterdam to a display at theCANARIE[11] booth at SC08. The portal seemed to work very well, howeverthere seemed to be networking problems resulting in major packet loss whichcould not be resolved during the event.The architecture in its current form has been demonstrated at the Visual-ization Symposium in june 2009[13]. This included the metadata server, thenew streaming server and an updated version of the web portal. Duringthis demo people could select the video they wanted to see from the portaland stream it to the tiled panel display next to it. This worked very welland there were positive responses on the general look en feel, however whenstarting a stream people got confused selecting the proper streaming sourceand destination display. This is was because not all the streaming and dis-play nodes in the list were functional at the time and you also had to beaware of the capabilities of the nodes, therefore this has to be improved orpreferably automated in the future.A poster has also been made about the architecture in its current form. Thisposter was demonstrated during the poster sessions of the Terena Network-ing Conference 2009[14].6 Conclusion and future workAlthough there’s still quite a lot to be done, the overall foundations that thisarchitecture provides lead to a usable digital content exchange. No matterwhat direction the CineGrid collaboration decides to go, the architecture8is relatively easy to extend and modify to adhere to new standards andprocedures.Some of the improvements that could be implemented in the future include:• Persistent database. The metadata server holds the metadata databasein-memory currently. This is fast, but could lead to difficulties withlarge databases.• Incremental updates. It would be nice if a client can ask for updatesin the metadata database since the last update or a specific date.Currently, the metadata server always sends the entire database.• iRODS integration. iRODS is the Integrated Rule-Oriented Data Sys-tem [7]. It will manage the storage in the distributed Exchange. Theintegration with iRODS regards metadata as well as playback.• Use of NDL. NDL [8] the Network Description Language - is a series ofRDF schema that allow better usage of the network,and in particularof circuit switched photonic networks, as the ones used in CineGrid.NDL can be used to determine available bandwith so the schedulercan schedule better.• Security. XML-RPC connections aren’t restricted in any way cur-rently, implement some sort of security system possibly also usingiRODS.References[1] CineGridhttp://www.cinegrid.org[2] Scalable Adaptive Graphics Environmenthttp://www.evl.uic.edu/cavern/sage/index.php[3] VLC, the cross-platform open-source multimedia framework, player andserverhttp://www.videolan.org/vlc/[4] XML-RPC for C and C++, a lightweight RPC library based on XMLand HTTPhttp://xmlrpc-c.sourceforge.net/[5] Django, The Web framework for perfectionists with deadlineshttp://www.djangoproject.com/[6] SuperComputing 2008http://sc08.supercomputing.org/9[7] iRODS, the Integrated Rule-Oriented Data Systemhttp://www.irods.org[8] The Network Description Language http://www.science.uva.nl/research/sne/ndl[9] Nortel Networks http://www.nortel.com/[10] Provider Backbone Transport http://www.nortel.com/pbt[11] CANARIE http://www.canarie.ca/[12] HPDMnet http://www.hpdmnet.net/[13] Mini-symposium: Networked Visualization for e-Sciencehttp://staff.science.uva.nl/ delaat/symp-2009-06-19/[14] Terena Networking Conference 2009 http://tnc2009.terena.org/10

English

UvA-DARE is a service provided by the library of the University of Amsterdam (https://dare.uva.nl)UvA-DARE (Digital Academic Repository)Amsterdam CineGrid Exchange: A distributed high-quality digital media solutionKnopper, S.; Koning, R.; Roodhart, J.; Grosso, P.; de Laat, C.Publication date2009Document VersionFinal published versionLink to publicationCitation for published version (APA):Knopper, S., Koning, R., Roodhart, J., Grosso, P., & de Laat, C. (2009). Amsterdam CineGridExchange: A distributed high-quality digital media solution. (SNE technical report; No. SNE-UVA-2009-1). Fac. Natuurwetenschappen Wiskunde en Informatica.http://staff.science.uva.nl/~grosso/Publications/AmsCineGridExchange.pdfGeneral rightsIt is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s)and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an opencontent license (like Creative Commons).Disclaimer/Complaints regulationsIf you believe that digital publication of certain material infringes any of your rights or (privacy) interests, pleaselet the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the materialinaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letterto: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. Youwill be contacted as soon as possible.Download date:10 Feb 2023Universiteit van Amsterdam System and Network EngineeringAmsterdam CineGrid ExchangeA distributed high-quality digital media solutionSander Knopper,Ralph Koning, Jeroen Roodhart, Paola Grosso and Cees de LaatJuly 30, 2009AbstractThe SNE group is an active participant in the CineGridinitiative. We built the first integrated CineGrid portal.In this report we describe the rationale behind the archi-tecture of our solution. We provide a component-wisedescription of design and functionality.SNE technical report SNE-UVA-2009-01http://www.science.uva.nl/research/sne/reports/1 IntroductionCineGrid[1] is an organization focused on the distribution of very-high-quality digital media over photonic networks. It does this by stimulatingresearch, development and demonstrations within its community. This com-munity consists of researchers in the academic world but also some compa-nies such as film studios.The very-high-quality digital media we’re currently working with is the 4kstandard. 4K offers a resolution of 4096 by 2160 pixels. Table 1 provides ansummary of the characteristics of various media formats.Format X Y Rate Color Frame Frame(fps) (bits/px) (px) (MB)720p HD 1280 720 60 24 921600 2.81080p HD 1920 1080 30 24 2073600 6.22k 2048 1080 24 / 48 36 2211840 10SHD 3840 2160 30 24 8294400 254k 4096 2160 24 36 8847360 40Table 1: Overview of current media formatsCompared to the 1080p HD standard, which is nowadays quite common forhome cinema systems, the resolution of 4k is 4 times larger.Format Flow Stream(MB/s) (Gbit/s)720p HD 170 1.31080p HD 190 1.52k 240 / 480 1.9 / 3.8SHD 750 6.04k 960 7.6Table 2: Network requirements of the various media formatsTable 2 reports the network requirements for transport of each of theseformats. 4K results in a bandwidth of 7.6 Gbit/s (compared to 1.5 Gbit/sfor 1080p); this requires a quite advanced network infrastructure.2 The CineGrid ExchangeThe CineGrid Exchange is the central distribution point in the CineGridcollaboration. The world-wide version is under development. When com-2pleted, it will provide a distributed storage system for digital video contentacross the world.The Exchange relies on the existence of metadata associated with the con-tent. These metadata would include the author, title of the video for in-stance, but also some more technical information like resolution, color depth,etc. This information allows the system to automatically place the videointo the storage system and make arrangements for the video to pop up inthe user interface. It is possible to store the video in different formats (interms of resolution and encoding for instance) and on different sites. In theend, a CineGrid user is not aware of the distributed nature of the storage.He would just be browsing on a website searching for content of his likingand - provided that he has the technical means to receive the stream- startdisplaying the content on his screen.A major technical requirements for the Exchange is extensibility. For ex-ample, different encodings require different media players; mechanisms thatallow to add new types of players related to new encodings seamlessly are amust. The playback functionality has to be fully pluggable. The same holdsfor the storage back-end, so also this part of the system is fully pluggable.3 ArchitectureThere are 3 distinct subsystems in the prototype system we’ve developed:• the web portal• the metadata server• the stream serverFig.1 shows the overall architecture of our Exchange. A metadata andstream server form a pair within a single streaming server. We can havemultiple sites where digital content is publicly available, but there’s only aneed for a single web portal. The web portal retrieves its data from themetadata servers at the different sites and when playback is requested, itnotifies the corresponding streaming server.The global workflow is as follow:1. a user browses the portal. The portal contains information about allthe available content. The portal has access to a database filled withinformation it gets from the metadata server.2. the user selects a movie and decides he wants to display it.3Figure 1: Architectural overview of the Amsterdam CineGrid Exchange3. the portal communicates with the stream server which will then takecare of streaming the content from disk to the display.In the following sections we give a more detailed overview of the three com-ponents.3.1 Streaming serverThe streaming server manages the streaming of content to the display chosenby the user. It supports multiple media players and a queueing algorithmto make sure different media request do not interfere with each other.Its functionalities are easily accessible by XML-RPC. It provides all thestandard functions expected from a streaming server: it can start playingcontent, it allows to skip to another movie, it can stop playing the currentmovie and provide the status. This is accomplished by the use of threads.The whole server is built around the scheduler, which schedules playback forthe destinations (displays) available and holds queues. It periodically checksfor movies that are finished playing and then frees that object and putsanother movie in its place after which it will start that movie. The algorithmbehind the scheduler is quite simple, but can easily be changed. It holds4an array of queues indexed by destination, so each destination (display) hasits own queue. Within this queue are the player objects, these are objectsalready holding all the information needed to start playing the movie whenthe scheduler asks them to. When a new movie is being added, the schedulerchecks the destination and looks if there’s already a queue available for it. Ifthere is, it simply puts the movie at the end of the queue, otherwise it createsa new queue for this destination and puts the movie in it while immediatelygiving it a signal to start playing. A periodic check loops through all thequeues looking for movies that have stopped playing, while also cleaning upany empty queues.When the scheduler starts playback for a specific player object, a new threadstarts. This thread sets some initial variables, usually the parameters for themedia player program, and after that it forks itself. In the forked process itstarts the right media player with the correct parameters. At this point themedia player takes care of actually displaying the content and the threadwill exit eventually. The original process (before the fork) will simply waitfor the new forked proces to finish, after which it will set the internal stateto “finished playing” which will be noticed by the next periodic check of thescheduler.Player objects are actually derived classes from the abstract player class.Each derived class is written to work with a single media player. Currentlythere is a class that works with SAGE[2] (a system used to stream contentto tiled displays) and one that works with VLC. Adding support for newmedia players is relatively easy, simply create a new class derived from theabstract player class and fill in the required methods to properly call themedia player executable.3.2 Metadata serverThe metadata server’s main goal is to gather all the available informationabout the content and save it as if it were a database. A very simple onethough, as you can’t execute complex queries on it, but you can request allthe available information.It consists of 4 main components:• XML-RPC server, which communicates with the outer world• metadata database, which holds all data about the content• metadata consumer, which converts available metadata to an internalformat• file system watcher, which keeps track of file changes5The XML-RPC server main functionality is to send the complete databaseto other parties. Beside that, it can also be used to retrieve the date of thelast update, so another party can figure out whether they need to updateor not.The metadata database is just a thread-safe container for single metadataobjects. Basically it’s a large array with some added functionality to makesure operations from the XML-RPC server and the file system watcher don’tcause data inconsistencies. This is important, since each of them runs in itsown thread.The interface of the metadata consumer is described in an abstract class, thisallows for multiple methods of harvesting metadata. Currently there’s onlyone implementation, which is a regular file system based one. In simply readsXML files of a mounted file system and converts the data to the internalrepresentation that will be put into the database.The file system watcher is an optional but quite useful component. It onlyworks in combination with the file system metadata consumer describedabove. It watches the file system where the content is placed for modifica-tions. This could be when a new file has been created, an existing file hasbeen modified or a file has been deleted. If such a modification occurs, thefile system watcher notifies the metadata consumer to re-read the metadatathus effectively updating the database. The file system watcher also has anabstract interface, currently only an inotify implementation exists for useunder GNU/Linux, new implementations are relatively easy to write.3.3 Web portalThe web portal is the only component the user sees and works with directly.It periodically checks for any updates the metadata servers might have, itthen fetches all the metadata from that server and does this for all associ-ated servers. It will then put all information into a local SQL database forfast lookup and support for complex queries, mostly required by the searchfunctionality.When a user enters the portal it sees the recently added movies along witha search field on top of the page. The user can search for specific keywordsin the title of the movie for example, but also filter the output by author orvideo format.After choosing a movie, a new page shows all the information about themovie and most importantly, all the formats the movie is available in. For-mats are a combination of resolution and encoding, this lets low-bandwidthusers also make use of the system for example. When the preferred format is6known, a page where the destination can be supplied shows. At the momentthis is a combo box filled with pre-defined values by the administrator. Inother words, filled with known displays.The final step consists of the portal looking up the corresponding streamingserver of the movie after which it will send an XML-RPC request to startplayback or at least put it in a queue. The user sees a confirmation of whathappened.4 About languages, libraries and frameworksThe metadata and streaming server are completely written in C++, makingextensive use of the C++ STL for strings, arrays, queues etc. Both pro-grams come with an autotools-based build system, providing compile-timedetection of available libraries. Such as inotify for example.The SAGE[2] library is used by bvmplayer, a media player to display un-compressed or DXT encoded content.VLC[3] is a media player, used to display content in all other encodings.xmlrpc-c is an XML-RPC library for C/C++ which is heavily used in boththe metadata and streaming server.Django[5] is a Python-based web framework which we’ve used for the webportal.75 DemonstrationsThe preliminary version of this architecture has also been used at NortelNetworks Advanced Technology Summit (ATS) and SuperComputing 2008(SC08)[6]. The preliminary version consisted of a very early implementationof the streaming server and no metadata server. The metadata had to beentered manually through the admin interface into the SQL database.At ATS the focus of the demo was to demonstrate streaming very highquality video content using Provider Backbone Transport (PBT)[10] over atransatlantic link from our servers in Amsterdam to a display in Ottawa totest the Quality of Service properties of PBT. We used the Web portal toinitiate the video streams and to select the proper display. We got somegood results from this.A week later at SuperComputing 2008 (SC08)[6] the same version of theportal was used as a part of a HPDMnet[12] demo. It involved streaminghigh quality content from our servers in Amsterdam to a display at theCANARIE[11] booth at SC08. The portal seemed to work very well, howeverthere seemed to be networking problems resulting in major packet loss whichcould not be resolved during the event.The architecture in its current form has been demonstrated at the Visual-ization Symposium in june 2009[13]. This included the metadata server, thenew streaming server and an updated version of the web portal. Duringthis demo people could select the video they wanted to see from the portaland stream it to the tiled panel display next to it. This worked very welland there were positive responses on the general look en feel, however whenstarting a stream people got confused selecting the proper streaming sourceand destination display. This is was because not all the streaming and dis-play nodes in the list were functional at the time and you also had to beaware of the capabilities of the nodes, therefore this has to be improved orpreferably automated in the future.A poster has also been made about the architecture in its current form. Thisposter was demonstrated during the poster sessions of the Terena Network-ing Conference 2009[14].6 Conclusion and future workAlthough there’s still quite a lot to be done, the overall foundations that thisarchitecture provides lead to a usable digital content exchange. No matterwhat direction the CineGrid collaboration decides to go, the architecture8is relatively easy to extend and modify to adhere to new standards andprocedures.Some of the improvements that could be implemented in the future include:• Persistent database. The metadata server holds the metadata databasein-memory currently. This is fast, but could lead to difficulties withlarge databases.• Incremental updates. It would be nice if a client can ask for updatesin the metadata database since the last update or a specific date.Currently, the metadata server always sends the entire database.• iRODS integration. iRODS is the Integrated Rule-Oriented Data Sys-tem [7]. It will manage the storage in the distributed Exchange. Theintegration with iRODS regards metadata as well as playback.• Use of NDL. NDL [8] the Network Description Language - is a series ofRDF schema that allow better usage of the network,and in particularof circuit switched photonic networks, as the ones used in CineGrid.NDL can be used to determine available bandwith so the schedulercan schedule better.• Security. XML-RPC connections aren’t restricted in any way cur-rently, implement some sort of security system possibly also usingiRODS.References[1] CineGridhttp://www.cinegrid.org[2] Scalable Adaptive Graphics Environmenthttp://www.evl.uic.edu/cavern/sage/index.php[3] VLC, the cross-platform open-source multimedia framework, player andserverhttp://www.videolan.org/vlc/[4] XML-RPC for C and C++, a lightweight RPC library based on XMLand HTTPhttp://xmlrpc-c.sourceforge.net/[5] Django, The Web framework for perfectionists with deadlineshttp://www.djangoproject.com/[6] SuperComputing 2008http://sc08.supercomputing.org/9[7] iRODS, the Integrated Rule-Oriented Data Systemhttp://www.irods.org[8] The Network Description Language http://www.science.uva.nl/research/sne/ndl[9] Nortel Networks http://www.nortel.com/[10] Provider Backbone Transport http://www.nortel.com/pbt[11] CANARIE http://www.canarie.ca/[12] HPDMnet http://www.hpdmnet.net/[13] Mini-symposium: Networked Visualization for e-Sciencehttp://staff.science.uva.nl/ delaat/symp-2009-06-19/[14] Terena Networking Conference 2009 http://tnc2009.terena.org/10

UvA-DARE

Amsterdam CineGrid Exchange A distributed high-quality digital media solution

Amsterdam CineGrid Exchange A distributed high-quality digital media solution

Abstract

Similar works

Full text

Available Versions

Search4Dev

NARCIS

International Migration, Integration and Social Cohesion online publications

International Migration, Integration and Social Cohesion online publications

UvA-DARE