November 9, 2010

Rich media reduces battery life. Not Flash, but the content it enables

Filed under: Flash,HTML5 — Steve @ 11:12 AM

Disclaimer – I use the Flash Platform to develop applications. I’m also excited by recent developments HTML5, CSS3 and JS. This is purely an attempt at balancing out the argument that Flash and not the types of content it drives is responsible for additional battery usage on devices. The problem is implementation.

In its review of the Macbook Air recently Ars Technica stated that using Flash resulted in a 33% reduction in battery life, a figure that has since been used by a number of anti-Flash campaigners and others to justify its demise in favour of a combination of HTML5, CSS3 and Javascript.

I thought I’d take a sample of some HTML5/CSS3/JS demos and measure the effects they place on my CPU. I was doing these tests on a Intel Core i5 iMac using Safari 5 – on a mobile device CPU usage is only part of the battery life equation but the harder you work the CPU, the less battery life you can expect.

A few points to begin with:

  • I’m not trying to make some sweeping statement about the real-world effects these results would have on battery life, nor am I going to pretend this was in any way scientific, it’s merely to demonstrate that rich content doesn’t come for free, regardless of the technology driving it
  • I’m not attempting to create a Flash vs HTML5 post either – I’ve seen real-world tests where HTML5 outperforms Flash, and vice-versa. Both have benefits and disadvantages
  • Similarly, I’m making no judgements or comments on the quality or efficiency of the demos shown below, they were all done by very talented people and the demos themselves are prime examples of what’s possible with HTML5, CSS3 and Javascript
  • For all these tests the methodology was the same – open Safari 5, then run the demo and take a screenshot during the demo
  • Finally, apologies for the lack of a LightBox view for the grabs, must fix my WordPress

Starting off, here’s Apple’s home page – it’s running a simple CSS transition in the Store div, but only taking 0.4%CPU -

Next, one of Apple’s own demos of another simple CSS3 transition, I didn’t quite catch the transition happening in the screenshot but the CPU is at 15.5% just after the transition completes;

Next, Apple’s VR demo – as I scroll the content, CPU averages about 20%;

Over to Andrew Hoyer’s blog now, with his demo of CSS3 animation, maxing out at an impressive 4% CPU;

Moving onto the <canvas> tag now. This draws directly into the browser using Javascript.

Firstly, the Voronoi demo from www.chromeexperiments.com. This demo takes up nearly 100% of my quad-core CPU;

Another <canvas> demo,  Deep Sea Stress – I added 5 ‘creatures’ here and ran the demo, 97.8% CPU;

A simple <canvas> gradient effect, 53.3% CPU;

A <canvas> particle effect, 27% CPU;

Ball pool by Mr. Doob, 27.7% CPU;

Finally, I tried a HTML5 game, Infiltration at Dusk, to see what the real-world effect was on CPU. It ran at around  41.5% CPU;

Conclusions

I like to see and experience rich content on the web. If I didn’t, I’d go read a book and save battery power that way. I expect that content to take CPU and memory and in return provide me with better experiences, and I understand this has implications for battery life. Here’s Apple’s own information on battery life for iPhone:

iPhone 4 can get 10 hours of video playback, or 40 hours of audio playback on a full charge at original capacity. In addition, iPhone 4 features up to 300 hours of standby time.


Why? Because decoding audio requires more CPU than standby, and video uses more CPU than audio. Even with dedicated decoding chips the device has to work much harder to display the rich media over static content.

A large part of the problem is that the Flash Platform is over-used by advertisers and this has a significant effect on CPU load. I hate Flash adverts as much as anyone, not least because they have given Flash a good degree of its bad reputation on the web. Multiple instances within a page, multiple tabs and windows compound the problem further. The reality is that if Flash is replaced by another technology like HTML5/CSS3/JS, the tests above demonstrate that any rich-media technology will require more processing power to render that content, versus plain text. It has the same potential for abuse too; it’s equally possible to write bad Javascript as it is to write bad Actionscript. There are so many factors involved that measuring the actual effects accurately will be difficult – some of these demos ran much faster in Safari than they did in Chrome, and vice-versa. Similarly I realise that Flash runs less optimally on OSX than Windows, but that debate is out of scope here.

Finally, a note about mobile devices – all of the above tests were carried out a desktop machine with a fast processor and 8GB of memory. On a mobile device the demos would be subject to much higher constraints/limitations. Some of the demos ran at well over 50% of CPU, and would either not run at all on a mobile device, or be very taxing on CPU and battery life. That’s the trade-off though – again I can accept that I can’t expect to run all rich content on my smart-phone, but I should be able to run some of it while knowing I will be using more battery when I do.

It’s the same as the difference between standby and talk-time to me.

Update 1: On the subject of HTML5 adverts, here’s a link to a HTML5 ad page, deliberately non-optimised and typical of the kinds of Flash advert used everywhere. On my machine this page took over 90% CPU. Thanks to John Auger for the link. See how your system performs.

Update 2: I tried the excellent HTML5/Flash Pong demo, and here are the results. As I stated before it’s not my intention to try and compare the two technologies, but as some people seem to think that guesswork is a good enough reason to assume HTML5 is less taxing on CPU, real-world results might be more credible. The developer says there’s a known performance issue on Safari, so I’ve posted the results for Chrome and Safari, but in my testing Flash outperforms <canvas> for CPU load here by a factor of around 4. Chrome actually seemed to be slightly worse than Safari in my tests.

Safari:

Chrome:

September 9, 2008

Silverlight to support H.264, AAC

Filed under: Flash — Steve @ 9:13 PM

Follow-up to <video> and Flash from [Ux]

Microsoft has announced that Silverlight will be supporting AAC audio and H.264 video. With regard to my previous entry about the video tag, this will make it even harder for Theora and can only consolidate H.264 as the current codec of choice. The announcement comes as no surprise to industry experts, but this move proves that MS is aggressively going after online video in a big way, but I’m not sure anyone really wins – based on browser/OS install base Silverlight may become more of a threat than the video tag would have been.

I’ve also been thinking a little more about Chrome – given that YouTube is part of Google it’s likely that when Chrome supports the video tag, the embedded codec(s) Google chooses could have a huge impact. Considering that H.264 is already being used on YouTube, perhaps that would be the default choice?

September 8, 2008

<video> and Flash

Filed under: Flash,H264 — Steve @ 3:53 PM

The availability of a Firefox 3.1 alpha (and Safari 3.1) with video tag support has prompted a few people to proclaim that Flash will soon be replaced as the primary medium for playing video on the web. I think predictions of Flash’s demise are both premature and inaccurate, and also think there’s a possibility that the introduction of the new tag could cause more problems than it was intended to solve. Why? – codecs…

Firefox (and Opera) will support one embedded codec in the first instance, Theora. Theora is completely open-source but based on an older-generation codec, On2 VP3. Flash currently uses VP6 and H.264, Quicktime Player supports H.264 and Silverlight adds support for VC-1, one of the newest kids on the video block. All are newer, higher-quality formats than Theora (typical comparison here). That’s not to say Theora is a bad codec, far from it, but in the world of video codecs it is at least a generation old, maybe more, and as such doesn’t represent the current best of breed in terms of video quality/performance.

Secondly, in order to take over from Flash, Theora needs to be on all or enough browsers to work as a standard. The HTML5 specification doesn’t advocate or recommend any codecs, and as the specification notes, this is problematic:

It would be helpful for interoperability if all browsers could support the same codecs. However, there are no known codecs that satisfy all the current players: we need a codec that is known to not require per-unit or per-distributor licensing, that is compatible with the open source development model, that is of sufficient quality as to be usable, and that is not an additional submarine patent risk for large companies. This is an ongoing issue and this section will be updated once more information is available.

So, if one browser advocates Theora, will every other browser follow? After all it’s a fairly trivial (and zero-licencing cost) thing to include support for it. Firefox and Opera combined currently have around 25-30% share of the browser market; while 30% (and growing) is an excellent starting point to build from, IE’s and Flash’s dominance in terms of installed base will make it difficult forTheora to overtake the current popular formats unless they support it too.

We’re not starting with a blank sheet of paper here either; many video services and sites that use video have spent a considerable amount of time and investment in encoding to a specific platform, one that is currently supported by enough browsers/platforms to make it worthwhile. At best a move to Theora would have to be transitional, and likely to take several years, as people migrate content over to the new format. IE is the most common browser by a considerable margin, and Microsoft has its own favoured formats; WMV via Windows Media Player and/or VC-1 via Silverlight, plus the formats supported by Flash Player. It’s hard to see these not being with us for a while yet, unless IE starts supporting Theora at the expense of Windows Media formats (unlikely); if Theora isn’t included in IE, it simply isn’t going to get traction as quickly as it needs to to become the standard. Similarly unless Webkit (as the core engine of Chrome and Safari) or YouTube also add native support for it (AFAIU latest Safari builds that support the video tag currently support Quicktime’s supported formats, which doesn’t include Theora as standard), it’s hard to see it taking over. While users will be able to add new codecs manually, assuming they will do so in order to view a new video format could be risky. Then there’s streaming support via RTSP/RTMP, etc.

My point is we are going to have to live with multiple formats whether we like it or not, and that Theora is a technically a backward step; it will only take one browser vendor to ignore Theora and it will become void as a standard, yet HTML5 isn’t forcing or even recommending a standard to follow. At this point video on the web will be in danger of becoming a mess as people find they don’t have the required codec and have to install support for it. I wouldn’t go as far as to suggest the video tag is going to create chaos where relative calm currently exists, but it might. I can’t see a good technical reason why the current model of using Flash as a wrapper for video and codec support is broken, or why it needs to be replaced with what may become a mess of format support where we move from two or three dominant formats to five, six or maybe more competing ones, plus multiple downloads for users. In my experience proprietary formats tend to be better, that’s why they cost, and neither do I buy into the ‘everything has to be open-source’ argument, especially considering that the dominant audio format is MP3, which isn’t an open-source format at all (even AAC codecs require patent licence).

Most importantly though, while all this takes place a solution already exists; Flash Player. It supports H.264, VP6 and Sorenson, most people already have it and it works on all the browsers mentioned above, across Windows, Linux and Mac. “But Flash is a terrible resource hog!” I hear you say, but when you consider that Flash is decoding and rendering video, in software, it’s worth noting that this takes CPU cycles and is a processor/memory intensive task. Even if you could run fully-hardware accelerated video (which Flash is moving towards), video is still relatively intensive work for the average desktop/laptop. In the first instance, Theora decoding in Firefox is going to run in software, just like Flash, and presumably consume CPU cycles in a similar way.

With all this in mind it would seem to me that the only reason for wanting to take Flash off its video pedestal is that it is owned by a commercial entity, ignoring the fact that Flash Player has been a key enabler and driver of the dramatic increase in use of video on the web, without any of the ‘payback’ people seem to fear. Without advocacy from W3C, the reality is that browsers are going to be free to implement their own choice of favoured codecs, but those choices are likely to be driven by different criteria, not necessarily whether they are open-source. It could be about to get messy.

Alternatively, we could all start using Mike Chambers’ workaround for getting Flash to display video wherever the video tag is used, and everyone could just carry on. ;-)

July 1, 2008

Flash SEO, Adobe manoeuvres

Filed under: Adobe,Flash,Flex — Steve @ 9:02 PM

Kudos to Adobe, Google and Yahoo for creating the mechanism for Flash content to be indexed on search engines. With one or two reservations (like how to distinguish between application content and a site) I think this is another significant move towards maintaining the ubiquity of Flash. It seems as though Adobe is steadily, but impressively quickly, removing piece by piece the most-cited drawbacks of Flash. Some of the most significant announcements (in no particular order):

  • H.264 video support
  • Open-sourced Flex SDK, BlazeDS
  • Opened access to Flash Player APIs
  • 3D support (thanks to Papervision, Away3D etc.)

Obviously it doesn’t take a genius to work out that, in part, Adobe has had to make some of these moves – HTML5, advances in Javascript and browser technologies promise or already offer many features that up until now Flash had to itself; the canvas and video tags in HTML5, DHTML effects, processing.js, SproutCore, the list goes on – it was inevitable that some of Flash’s functionality would be adopted and integrated into other technologies, but I think that those people who still maintain that they ‘hate Flash’ should still agree that it has been a primary driver of change on the web. This is particularly true in three areas; animation, user experience and video. Flash may responsible in their eyes for evil banner ads and inaccessible sites/UIs, but those things are created by people, and a technology that provides a degree of creative freedom is open to abuse – there are plenty of bad DHTML sites around and more than a fair share of annoying DIV-based floating ads out there already, but there is also a great number of fantastic and successful experiments in user interface design, animation and application experience. Flash has enabled developers to do things on the web they could not have easily done by other means, and those people that attribute bad practice to Flash are both ignorant and in for shock if they believe the same things won’t happen when other technologies catch up. Up until now it’s been far too easy to blame Flash, when Flash isn’t the problem. Use it where it works, not where it doesn’t (like whole sites).

In addition to the technology itself, Adobe has provided the means to develop and deploy it effectively, with the Flex SDK and FlexBuilder. Personally I have no objection to proprietary technologies when they a) work, b) don’t break anything and c) positively drive change and allow people to do things that standards-based technologies often take much longer to enable (and often not quite as well). Flash and Flex won’t be the standard, they will peacefully co-exist with other technologies (along with man, and fish); a single unified standard just isn’t possible in a competitive world, nor is it always desirable. Someone has to innovate, and attacking Flash (or Apple come to to think of it) for being proprietary is like attacking Ferrari for making a better sportscar (and charging for it). If it enables you to go faster, better, and (similar to Java and JS) is on 90-something% of desktops, who can blame Adobe for adding features and functionality that will maintain or increase edge and adoption? And at the same time if it is making key components of its platforms open, regardless of motivation, it’s A Good Thing*. So long as the standards do catch up, it’s fine.

There is I think, one thing left to do at the moment, the final hurdle as I see it – accessibility. It’s kind of in there, but if Adobe could make Flash and Flex as accessible as a typical web page, or at least as easy to make accessible as a web page, it would remove the one last stick with which it gets beaten. In fact and to bring this full circle, the same mechanism by which search indexing now works may also prove the key to unlocking accessibility, so maybe that’s already possible?

*None of these arguments apply to Microsoft, especially the Ferrari analogy. Silverlight is neither better or faster.

May 15, 2008

Flash Player 10 (‘Astro’) pre–release on Adobe Labs

Filed under: Adobe,Flash — Steve @ 8:55 AM

Writing about web page http://labs.adobe.com/technologies/flashplayer10/

A pre-release of Flash Player 10 is now out on Adobe Labs. Astro has a number of new features; P2P support, Speex voice codec (no more having to use ASAO!), dynamic streaming support and RTMFP, 3D, improved hardware acceleration and more sophisticated text control being the ones I’m most looking forward to.

The full feature list can be found here

May 14, 2008

Flash H264: Moving the MOOV Atom

Filed under: Flash,H264 — Steve @ 11:21 AM

I just had to encode a couple of videos from AVI to H.264 format for playing via a Flash-based video player and noticed that the player was having to download the whole file before playing it. I checked out this Adobe Tech article on H.264 encoding and here’s why:

One important thing about playing an H.264 video file as progressive download is that the moov atom needs to be located at the beginning of the file, or else the entire file will have to be downloaded before it begins playing. The moov atom is a part of the file that holds index information for the whole file.

I was using SUPER to encode to H.264, and by default the moov atom gets placed at the file end. Fortunately, Renaun Erickson at Adobe has kindly created a small AIR application (a port of a C++ application) that will take a standard H.264 file and relocate the moov atom to the front of the file, and then progressive download works properly.

Edit- thanks to Marshall’s comment, if you’re using SUPER there’s an option to select ‘Streamable H.264’ – do that and the moov atom will be put at the beginning of the file. :-)

May 9, 2008

AS Development skills sharing at Warwick

Filed under: AIR,Development,Flash,Flex — Steve @ 11:18 PM

Chris was talking today about intra-team/department skills sharing, which I think is a great idea. With regard to Flash and Flex development at Warwick we’ve recently embarked on something along these lines, with the formation of a University Flash and Flex developers group consisting of people actively using (or just interested in) Actionscript development. It started with a forum that was initially populated with ITS people (plus one or two others), but since starting to deliver some Flash courses for ITS Training I wanted to maintain contact with what attendees did with the skills they’d acquired after the course, and the forum was opened up to allow more people to contribute. We held the first ‘skills session’ a couple of weeks ago, in the Teaching Grid. It was purposely an informal gathering, consisting of developers from ITS and departments, plus lots of coffee and biscuits. The session was an opportunity for everyone to show what they’d been working on and share expertise and a mix of ongoing Flash, Flex and AIR projects were demonstrated, plus discussion on development approaches and potential applications in the future.

Overall I thought it went well (we ran out of time in the end) and although next time some structure to the session might help us cover ground more efficiently, the informality and range of projects kept it interesting, and it was especially good to share ideas with others while receiving questions and feedback. As a result of the session, I’ve been asked to present a more formal hour-long session on AIR to the CIS Team next week, which I think I’ll prepare some slides for.

Anyway, my point is that personally I found it quite fun/rewarding to find out what other people were doing or planning to do with the technology, and to see and discuss approaches/methodologies and alternatives; we should definitely do more of this kind of thing in future. Time will tell whether we maintain the Flash/Flex/AIR group sessions but I hope we can.

Warwick staff interested in Flash/Flex/AIR/AS3 development can join the developer group forum here. For students, Warwick Game Design have expertise with Flash for game development.

May 1, 2008

Adobe Open Screen Project

Filed under: Flash — Steve @ 8:34 AM

Writing about web page http://www.adobe.com/openscreenproject/faq/index.html

Adobe announced the Open Screen Project today. The project will allow access to key Flash specifications (AMF, SWF, FLV etc.) and remove licencing restrictions on Flash and AIR for devices. The stated aims are:

-Removing restrictions on use of the SWF and FLV/F4V specifications
- Publishing the device porting layer APIs for Adobe Flash Player
- Publishing the Adobe Flash® Cast™ protocol and the AMF protocol for robust data services
- Removing licensing fees – making next major releases of Adobe Flash Player and Adobe AIR for devices free

Fantastic news – actually something that may have been lost in the noise here is that it seems like AIR is definitely coming to devices. Allowing access to the SWF specifications is a major step for Adobe. Kevin Lynch of Adobe describes the rationale behind OSP in this video.

April 21, 2008

Flash 9 webcam support in Linux

Filed under: Flash,Webcam — Steve @ 3:23 PM

This is a bit annoying – we’d planned to use a number of EEE PCs with our Flash-based media streaming applications for a new kiosk-based project, and after some spectacularly unsuccessful tests today with an EEE PC and a range of webcams (including the built-in one) it seems that the problem is with Flash itself – it only supports VideoForLinux (V4L) and not the current V4L Level 2. V4L was removed from the Linux kernel way back in 2006 so I was surprised to find that V4L L2 isn’t there in Flash Player 9.

Apart from the EEE PC, this also discounts an number of other potential devices like the N800/810 and we’ll have to look at Windows UMPCs instead (which is likely to severely impact the budget), or alternatively install Windows on the EEEs (which is a hassle).

I agree with Aral – there’s an explosion of ultra-mobile, Linux-based devices out there now with webcams that could be used to enable video and audio-based communications and blogging, but at the moment we can’t lever them in Flash, and that’s a shame.

April 2, 2008

360degree video

Filed under: Flash — Steve @ 8:47 PM

Writing about web page http://www.cornflex.org/?p=1

Wish I could embed the full example here, but I’ve just been looking at these awesome examples by Quentin Lengelé of combining footage shot with a 360degree camera, a 3DSMax ASE sphere model and Papervision 3D to create a fully 3-dimensional video experience (the original footage is by Immersive Media, who now also offer the same kind of thing commercially in a Flash-based player, lots of other demos on the link). Wish I could come up with this kind of stuff…

Papervision Video Screen
Older Posts »