This paper reports on research investigating the current ability of HTML5 to play video in mobile browsers. Smartphones and the Mobile Internet are rapidly becoming an important platform for access to information anytime and anywhere. HTML5, the new HTML standard incorporates features like video playback that have been previously dependent on third-party browser plug-ins but there are no browsers that currently provide 100% support for HTML5. All the tests reported in this paper were carried out using smartphones with screen sizes 3.0 to 4.8 inches and the ability to replay videos of a range of formats, move directly to time points in the video and display closed captions were investigated. Key findings were that: video cannot be started programmatically; only selecting on the screen can trigger playback; no visual elements sitting over the &lt;video&gt; will receive click events while the video is visible (playing or paused); there are many HTML5 video players but MediaElement.js was found to currently be the open source player satisfying the greatest number of requirements

Draffan, E.A.

Li, Yunjia

Wald, Mike

Wei, Jing

English

Southampton (e-Prints Soton)

HTML5 Video on Mobile Browsers   Mike Wald, E.A. Draffan, Yunjia Li, Wei Jing ECS  University of Southampton Southampton, UK m.wald@ecs.soton.ac.uk  Abstract—This paper reports on research investigating the current ability of HTML5 to play video in mobile browsers. Smartphones and the Mobile Internet are rapidly becoming an important platform for access to information anytime and anywhere. HTML5, the new HTML standard incorporates features like video playback that have been previously dependent on third-party browser plug-ins but there are no browsers that currently provide 100% support for HTML5. All the tests reported in this paper were carried out using smartphones with screen sizes 3.0 to 4.8 inches and the ability to replay videos of a range of formats, move directly to time points in the video and display closed captions were investigated. Key findings were that: video cannot be started programmatically; only selecting on the screen can trigger playback; no visual elements sitting over the <video> will receive click events while the video is visible (playing or paused); there are many HTML5 video players but MediaElement.js was found to currently be the open source player satisfying the greatest number of requirements. Keywords- HTML5, video, mobile browser I.   INTRODUCTION This paper reports on research investigating the current ability of HTML5 to play video in mobile browsers. Smartphones and the Mobile Internet are rapidly becoming an important platform for access to information anytime and anywhere. Ofcom communication market research [1], [2] has shown that 40% of UK adults are now smartphone users and 40% of these thought their phones are more important for accessing the internet than any other devices, with 22% users watching TV/film content, 37% taking photos/videos and 35% listening to music. In the first quarter of 2012, 36% of the world’s mobile phone shipments were smartphones, compared to 25% the year before [3]. Smartphones powered by the Android and iOS mobile operating systems accounted for more than 80% of smartphones shipped in the first quarter of 2012 [4]. However, there are different versions of Android and according to the number of Android devices that have accessed Google Play during a 14-day period ending on November 1st 2012,  Android version 2 had 69.3 % of the Android market, version 3 had 1.8 % and version 4 had 28.5% % [5]. With new devices launched with Android 4, its market share will rise with the sales, but Android 2.3 Gingerbread will still be the dominant Android operating system for some time [7].    II.  HTML5 <video> support in mobile browsers HTML5 is the fifth major revision of HTML. Steve Jobs [6] stated that “Flash is no longer necessary to watch video or consume any kind of web content….” and “new open standards created in the mobile era, such as HTML5, will win.” Frank, [7] states that “H.264 is becoming the industry standard for playing video & is the market leader at 80%. WebM has little traction (less than 2%). Ogg Theora hasn’t grown significantly. Flash is no longer  being developed for mobile”. Dilger [8] states that “Mozilla's director of research Andreas Gal has proposed enabling mobile H.264 video decoding via hardware or the underlying operating system, signalling the end to the group's war on the Apple-led H.264 video codec.” Safari 6 [9] on the desktop has opened their API on HTML5 <track> and HTML5 media synchronization in June 2012, and those changes also will happen in iOS 6. The latest versions of Apple Safari, Google Chrome, Mozilla Firefox, Opera and Internet Explorer on the desktop all support many HTML5 features while the mobile web browsers, which come pre-installed on iPhones, and Android phones also all embrace HTML5.  There are no browsers that provide 100% support for HTML5. HTML5test shows the mobile support score of HTML5 during the last four years and at the time of writing Opera Mobile leads with a score of 406 out of a possible 500  with Chrome next with 390, Firefox mobile with 388 and then iOS with 386 [10]. The HTML5 standard incorporates features like video playback that have been previously dependent on third-party browser plug-ins such as Adobe Flash, Microsoft Silverlight. Also, HTML5 is designed, as much as possible, to be backward compatible with existing web browsers. New features build on existing features and allow users to provide fallback content for older browsers.  As there are no systematic tests for the video compatibility on different mobile browsers, experiments were carried out to find out which video codecs could be played in the major browsers. Another choice the designer needs to make is which codec(s) their mobile system will use. There are, at this time, three codecs supported by current browsers in HTML5:  MP4 or H.264; Ogg/Theora; WebM. Unfortunately, no one codec is supported by all the major players, so for making technology to be seen by the largest group of people, video should be converted into at least two formats. Long Tail Video research Group state [11] that roughly 74% of the market (browsers and devices) now supports HTML5 with Chrome/Firefox/Opera supporting the WebM format and Chrome/IE/Safari supporting the MP4 format. They have not included the Ogg video format in their tests as they considered that Ogg is not widely used and has lower quality than MP4 and WebM. However for mobile players such high quality may not be required; therefore, all these there current popular video formats were tested. All the tests were carried out using mobile phones with 3.0 to 4.8 inches screen sizes and the main test platforms and browsers are shown in Table I. III.  HTML5 Video Playable Test The test results in Tables II, III and IV show if a video file created with a certain codec could be played in a particular browser on a particular mobile phone. The test results show that: There is currently no video codec format that works with all mobile browsers but MP4 seems compatible with most “default” browsers, i.e. Android Native browser in Android phones and Safari in iPhones; Opera Mini could not play any video; WebM is not well supported by the native players on mobile browsers; Most mobile phones play the video in full screen mode by default and users cannot quit the full screen unless stopping the video but Firefox is an exception. IV.  HTML5 Video Seekable Compatibility in Mobile Browsers Seeking a time point in the video is an important function in any player design. In the test page used for this research there were 3 time points the video could jump to. The test results in Tables V, VI and VII show if a video file with a certain codec could jump to these time points in a particular browser on the same mobile phones used in the playable test. “NA” means the video couldn’t be played and so the test could not be carried out. The test results show that if the video can be played, it can jump to a time point. V.  WEBVTT SUPPORT IN MOBILE BROWSERS HTML5 not only allows videos to play natively, but also includes the track tag for captions which are very important for the accessibility of videos on the Web. WebVTT (Web Video Text Tracks) [12] is only in the WHATWG specification, but the recently established W3C Web Media Text Tracks Community Group [13] should introduce a WebVTT specification to the W3C soon. Files captioned using WebVTT were used to investigate whether the captions could be displayed on the platforms, browsers and devices used for the previous tests and Table VIII present the results which showed that there were no mobile browsers which supported WebVTT natively. There are many HTML5 video players on the Web, VideoJS, PopcornJS, MediaElement.js, JWPlayer, Kaltura Player, etc. [14]. Some of browsers indeed support WebVTT on desktop browsers. But they are all “polyfills”, which means the support is not native and they use JavaScript and CSS to somehow present WebVTT together with the video. In the next section MediaElement.js is tested on iPhones, but it still failed to display the captions on mobile phones although it can successfully display WebVTT on the iPad. VI.  MEDIAELEMENT.JS SUPPORT IN MOBILE BROWSERS The challenge for HTML5 video/audio is that the system needs to embed different players based on not only the media types, but also the platforms. Since Flash is not well supported on mobile platforms the designer must consider the HTML5 native player and control it through JavaScript. MediaElement.js is a “fall forward” html player [15], which means it is based on the HTML5 native player (See Fig. 1). However, if the browser doesn’t support HTML5, MediaElement.js will embed the self-developed Flash and Silverlight. While both JW player and MediaElement.js come top in the newest player comparison [16] JW player is not an open source player. Fig. 2 shows the Browser and Device support in MediaElement.js. All the tests with the formats and browsers and devices described in sections 3, 4, 5 were repeated with the MediaElement.js player and the results are presented in tables IX – XV. The YouTube format was also investigated.  D/L stands for ‘Download’ which means that the player does not support the video and suggests the user choose other software or device to read the file(s) [17]. MediaElement.js makes the video more accessible; users can see captions fluently in both Android 2.3 Opera Mobile browser and Android 4.04 native browser.         TABLE I HTML5 VIDEO SUPPORT TEST PLATFORMS OS   Android 2.3   Android 4.0   iOS 5    Browser Opera Mobile   Opera Mobile   Safari  Opera Mini   Opera Mini   Chrome  Firefox   Chrome   Opera Mini  Android native browser  Firefox      Android native browser    TABLE II HTML5 VIDEO COMPATIBILITY IN ANDROID 2.3 Browser   MEPG4H.264  Ogg Theora  WebMVP8  opera mobile 12.0   Yes   No   No  opera mini 7.0.29   No   No   No  Firefox 14.0.1   No   Yes*   Yes *  Android inner browser   Yes   No   No  *Unlike other browsers, the video will not enter full screen when playing, and there is no full screen control in the player in Firefox 14.0.1. Test Phones were HTC Design S, Android 2.3.5; Sony SK17i, Android 2.3.4; Samsung Galaxy S II, Android version 2.3.3; HTC G10, Android 2.3.5.  TABLE III HTML5 VIDEO COMPATIBILITY IN ANDROID 4.0.4 Browser   MEPG4 H.264  Ogg Theora  WebMVP8  opera mobile 12.0  Yes1   No  No  opera mini 7.0.29   No   No   No  Chrome 18.0   Yes   No   Yes  Firefox 14.0.1   No   Yes2   Yes2  Android inner browser   Yes   No   No  1. The video will not enter full screen when playing, and also there is no control bar.  2. The video will not enter full screen when playing, and there is no full screen control in the player in Firefox 14.0.1. Test Phone: Samsung Galaxy S III  TABLE IV HTML5 VIDEO COMPATIBILITY IN IOS 5.1.1 Browser   MEPG4H.264  Ogg Theora  WebMVP8  Safari 5   Yes   No   No  Chrome 21.0   Yes   No   No  Opera Mini 7.0   No   No   No  Test Phones: iPhone 4 & 4S    TABLE V HTML5 VIDEO SEEKABLE COMPATIBILITY IN ANDROID 2.3 Browser   MP4 H.264  Ogg Theora  WebM VP8  YouTube  opera mobile 12.0  Yes   NA   NA   Yes  opera mini 7.0.29  NA   NA   NA   NA  Firefox 14.0.1  NA   Yes   Yes   NA  Android inner browser  Yes   NA   NA   Yes   TABLE VII HTML5 VIDEO SEEKABLE COMPATIBILITY IN IOS 5.1.1 Browser   MEPG4H.264  Ogg Theora  WebMVP8  Safari 5   Yes   NA   NA  Chrome 21.0   Yes   NA   NA  Opera Mini 7.0   NA   NA   NA   TABLE VI HTML5 VIDEO SEEKABLE COMPATIBILITY IN ANDROID 4.0.4 Browser   MEPG4 H.264  Ogg Theora  WebMVP8  opera mobile 12.0  Yes   NA   NA  opera mini 7.0.29   NA   NA   NA  Chrome 18.0   Yes   NA   Yes  Firefox 14.0.1   NA   Yes   Yes  Android inner browser   Yes   NA   NA   TABLE VIII WEBVTT SUPPORT IN DIFFERENT MOBILE BROWSERS OS   Browser   MP4 H.264  Ogg Theora  WebM VP8  Android 2.3 Opera Mobile 12.0   No   N/A   N/A  Opera Mini 7.0.29   N/A   N/A   N/A  Firefox 14.0.1   N/A   No   No  Android Native Browser  No   N/A   N/A  Android 4.0.4  Opera Mobile 12.0   No   N/A   N/A  Opera Mini 7.0.29   N/A   N/A   N/A  Chrome 18.0   No   N/A   No  Firefox 14.0.1   N/A   No   No  Android Native Browser  No   N/A   N/A  iPhone iOS 5 Safari 5   No   N/A   N/A  Chrome 21.0   No   N/A   N/A  Opera Mini 7.0.29   N/A   N/A   N/A  “N/A” means a certain format of the video file cannot be played   TABLE XV WEBVTT SUPPORT IN MEDIAELEMENT.JS IN DIFFERENT MOBILE BROWSERS OS   Browser   MP4 H.264  Ogg Theora  WebM VP8  Android 2.3 Opera Mobile 12.0   Yes   N/A   N/A  Opera Mini 7.0.29   N/A   N/A   N/A  Firefox 14.0.1   N/A   No   No  Android Native Browser   No   N/A   N/A  Android 4.0.4  Opera Mobile 12.0   No   N/A   N/A  Opera Mini 7.0.29   N/A   N/A   N/A  Chrome 18.0   No   N/A   No  Firefox 14.0.1   N/A   No   No  Android Native Browser   Yes   N/A   N/A  iPhone iOS 5 Safari 5   No   N/A   N/A  Chrome 21.0   No   N/A   N/A  Opera Mini 7.0.29   N/A   N/A   N/A   TABLE X ANDROID 4.0.4 VIDEO COMPATIBILITY IN MEDIAELEMENT.JS PLAYER Browser   MP4 H.264  Ogg Theora  WebM VP8  YouTube  opera mobile 12.0  Yes*   D/L  D/L  No  opera mini 7.0.29  D/L  D/L  D/L  No  chrome 18.0   Yes*   D/L  Yes*   Yes  Firefox 14.0.1  NA   Yes*   Yes*   NA  Android inner browser  Yes*   D/L  D/L  Yes*  * The video can be played but will not enter full screen. Also, there have no full screen button.   TABLE IX ANDROID 2.3.4 VIDEO COMPATIBILITY IN MEDIAELEMENT.JS Browser   MP4 H.264  Ogg Theora  WebM VP8  YouTube  opera mobile 12.0  Yesa      D/L  D/L  opera mini 7.0.29  D/L  D/L  D/L  D/L  Firefox 14.0.1  No   Yes   Yes b   Yes c  Android inner browser  Yes   D/L  D/L   Yes  a When users change phone direction, the video will restart in full screen. bThe video will be played in a new size as the device, but not in full screen mode. Once users change the device direction, the size and the direction of video will not change. c. Users must download a new plugin to support YouTube before you play the video Google Chrome is not compatible with Android 2.3.   TABLE XI IOS 5.1.1 VIDEO COMPATIBILITY IN MEDIAELEMENT.JS PLAYER Browser   MP4 H.264  Ogg Theora  WebM VP8  YouTube  Safari 5   Yes   D/L  D/L  Yes*  Chrome 21.0   Yes   D/L  D/L  Yes*  Opera Mini 7.0  D/L  D/L  D/L  No  *YouTube video will appear on loading and does not play   TABLE XIII HTML5 VIDEO SEEKABLE COMPATIBILITY IN ANDROID 4.0.4 Browser   MP4 H.264  Ogg Theora  WebM VP8  YouTube  opera mobile 12.0  Yes   NA   NA   Yes  opera mini 7.0.29  NA   NA   NA   NA  chrome 18.0   Yes   NA   Yes   Yes  Firefox 14.0.1  NA   Yes   Yes   NA  Android inner browser  Yes   NA   NA   Yes   TABLE XII HTML5 VIDEO SEEKABLE COMPATIBILITY IN ANDROID 2.3 Browser   MEPG4H.264  Ogg Theora  WebMVP8  opera mobile 12.0   Yes   NA   NA  opera mini 7.0.29   NA   NA   NA  Firefox 14.0.1   No   Yes   Yes  Android inner browser   Yes   NA   NA   TABLE XIV HTML5 VIDEO SEEKABLE COMPATIBILITY IN IOS 5.1.1 Browser   MP4 H.264  Ogg Theora  WebM VP8  YouTube  Safari 5   Yes   NA   NA   Yes  Chrome 21.0   Yes   NA   NA   Yes  Opera Mini 7.0  NA   NA   NA   NA   Figure 1 How MediaElement.js works [15]  VII.  CONCLUSION HTML5 has been designed to be backward compatible with older versions of HTML. The test for HTML5 video tag and WebVTT on mobile browsers is important. Developers should be clear about which video formats could be played in each target device. On the server side, the web applications need to host the best format that is compatible with the target device. Video sharing services, such as YouTube, need to be adaptive enough to deliver videos to different devices and browsers. Captions important for the accessibility of videos, (especially on mobile platforms where viewing areas are small and audio output not necessarily clear) are once again dependent on the browser support and may not always work. With each update or release of new versions of browsers, new codecs might be supported and old formats might be abandoned. WebVTT is currently not well supported on mobile browsers. Pfeiffer [14], a WebVTT main developer, states that IE, Chrome, Safari, Opera and Firefox are all trying to support it in their new editions. In order to ensure the best possible user experience on iOS devices, Apple (2012) [18] has placed some stringent controls on how video can be viewed and controlled on these devices. All HTML5 video players try and work around these issues in the best way possible, but the following limitations still persist: Video is always playing in full screen mode in the native playback application and full-screen videos can only display the default controls. Therefore, users cannot view any context on the iPhone. Videos cannot be started programmatically, only clicking on the screen can trigger playback. Additionally, no visual elements sitting over the <video> will receive click events while the video is visible (playing or paused). Users can also only use the built-in control bar or the device controls to adjust volume. For future work it will be necessary to repeat the experiments to ensure the results reflect any changes to browsers, platforms and devices. REFERENCES [1]  Ofcom. Communications Market Report 2012. p221-234. http://stakeholders.ofcom.org.uk/market-data-research/market-data/communications-market-reports/cmr12/uk/ [2]  Ofcom. Smartphones are an important means of accessing the internet. http://stakeholders.ofcom.org.uk/market-data-research/market-data/communications-market-reports/cmr12/internet-web/uk-4.03  Figure 2 MediaElement.js Browser and Device support [17] [3]  Anthony, S. Smartphones set to become the fastest spreading technology in human history. http://www.extremetech.com/computing/129058-smartphones-set-to-become-the-fastest-spreading-technology-in-human-history [4]  International Data Corporation. Android- and iOS-Powered Smartphones Expand Their Share of the Market in the First Quarter, According to IDC. http://www.idc.com/getdoc.jsp?containerId=prUS23503312 [5]  Android Open Source Project Group. Dashboards. http://developer.android.com/about/dashboards/index.html [6]  Jobs, S. Thoughts on Flash. http://www.apple.com/hotnews/thoughts-on-flash/ [7]  Sinton, Frank.  HTML5 Based Video http://blog.mefeedia.com/html5-dec-2011  [8]  Dilger, D.E. Mozilla considers H.264 video support after Google's WebM fails to gain traction. http://appleinsider.com/articles/12/03/14/mozilla_considers_h264_video_support_after_googles_vp8_fails_to_gain_traction.html [9]  Apple Developer.  Safari HTML5.  https://developer.apple.com/technologies/safari/html5.html [10]  Sights. The HTML5 Test -  HOW WELL DOES YOUR BROWSER SUPPORT HTML5? http://html5test.com/results/mobile.html [11]  Long Tail Video. The State Of HTML5 Video. http://www.longtailvideo.com/html5 [12]  Hickson, I. WebVTT Living Standard —  Last Updated 11 September 2012. http://dev.w3.org/html5/webvtt/ [13]  Singer, D.  Web Media Text Tracks Community Group.  http://www.w3.org/community/texttracks/ [14]   Pfeiffer, S. WebVTT support in browsers. http://www.w3.org/community/texttracks/2012/08/23/webvtt-support-in-browsers/  [15]  Dyer, J. MediaElement.js Released. http://blog.mediaelementjs.com/2010/07/mediaelementjs-released/ [16]  praegnanz Group. HTML5 Video Player Comparison.  http://praegnanz.de/html5video/  [17]  Dyer, J.  MediaElement.js. http://mediaelementjs.com/ [18]   Safari Developer Library. Safari HTML5 Audio and Video Guide.   http://developer.apple.com/library/safari/#documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/ControllingMediaWithJavaScript/ControllingMediaWithJavaScript.html#//apple_ref/doc/uid/TP40009523 

HTML5 video on mobile browsers

Abstract

Similar works

Full text

Available Versions

Southampton (e-Prints Soton)