Adobe AIR provides a very attractive abstraction layer for video playback on mobile devices, but a certain level of knowledge is required to take full advantage. Before we start, it is important to understand the difference between Adobe AIR and Flash Player on mobile devices. Flash Player on mobile runs in the browser, and allows to display Flash web content that otherwise wouldn’t be available. The challenge of this approach is iOS does not have Flash support, meaning for reach, mobile sites are not authored in Flash, but rather HTML. Adobe recognized this, and made appropriate adjustments late last year.
Adobe AIR in contrast is based on the application approach, and can combine Flash, HTML and native functionality – as result, it is possible to take advantage of the advanced video features that are widely deployed with Flash on the desktop, including adaptive streaming, content protection, ad monetization and beyond.
To reaffirm the commitment to Adobe AIR, Danny Winokur, Vice President & General Manager, Interactive Development at Adobe made the following the statement.
Our future work with Flash on mobile devices will be focused on enabling Flash developers to package native apps with Adobe AIR for all the major app stores.
The only platform that has not full support for the Adobe video stack is AIR for iOS due to iOS limitations, but it contains very legit workarounds to enable equivalent, fully App Store compliant, functionality within the same workflow, resulting in the same end user experience. Let’s look at the media capabilities of the different mobile platforms.
Protocols and codecs
Fully hardware accelerated H.264 video playback is supported on all AIR target mobile platforms including iOS. The supported protocols are slightly different on iOS though.
Other codecs are supported as well, but not HW accelerated.
Content protection is supported, but again with slight differences on iOS.
The figures above show that video features are mostly equivalent to the desktop, with minor differences on iOS. For a more in-depth overview how to use a unified workflow to include iOS video delivery, read the mobile strategy post, and the article comparing HDS with HLS.
In addition, here is a short video of HLS H.264 live streaming running in AIR for iOS, demonstrating how well it integrates.
Performance
The performance of AIR for Mobile video delivery depends on the target platform, but iOS and the Blackberry Playbook have very limited fragmentation, and the AIR video performance is equivalent to native. Thus the overall Apple encoding recommendations also apply to AIR for iOS, and the Playbook’s very well performing H.264 hardware decoder can consume most H.264 encoded content.
Android on the other side, with hundreds of different devices, is a lot more challenging, but also benefits most from video delivery with Adobe AIR. But before I dive into the specifics, let’s look at a crucial performance feature for mobile video delivery.
StageVideo
StageVideo, in contrast to the older video object, GPU accelerates 100% of the video pipeline, meaning best video quality, performance and battery life.
On Android the video object supports hardware decoding, but part of the video pipeline might use CPU. This is sufficient for smart phones, but not a good technique for HD quality video on tablets.
As you can see above, StageVideo has different characteristics on each platform, but the overall guidance is to detect StageVideo and use if available. The Open Source Media Framework automatically enables this functionality, but there is also a manual detection approach.
(Update: 09/08/12) While Android 2.2+ and 3.x require the use of the StageVideo API to take advantage of StageVideo, AIR on Android 4.x devices uses StageVideo automatic mode. This means all content will automatically use StageVideo, even if you use the regular video object, and not specifically the StageVideo API.
Android video fragmentation
Android as a video delivery platform is more challenging than a platform like iOS with very limited fragmentation. Android runs on hundreds of different devices, with different hardware, and GPU layers underneath it.
Adobe AIR provides an abstract layer on top of a complex and fragmented foundation, with a lot of continuous efforts invested in supporting different chipsets and drivers. As pointed out in a previous blog post, even the supported streaming protocols differ based on the Android version. Once built on top of AIR, the application will adopt to new devices with minimal efforts due to continuous AIR updates. That said, the Android team is working on decreasing the fragmentation. But to reach all devices today, a native approach without Adobe AIR in comparison is time consuming and often requires third party support to enhance e.g. the native streaming capabilities.
With such a diversity comes the need for a video encoding performance baseline to reach all devices with the sames encoding, which Adobe provides in detailed form, and with my short guide from 2010. In addition tablets with StageVideo and Android 3.1+ support 720p video playback, and the Kindle Fire offers high quality playback even on Android 2.3.
Note: In case you experience performance issues with 720p playback on Android 3.1+ devices with StageVideo enabled, ensure to download the latest AIR runtime from the Android Market.
If device specific video optimization is desired, it is possible to fairly accurately detect the device based on the method the AIR application Device Info uses, and deliver device optimized profiles.
Monetization
As part of supporting the Flash Video ecosystem with Adobe AIR come all the monetization options, including video ads with Auditude, measurement with Omniture, and everything else that is part of the ecosystem.
To better understand how to convert your Flash video web experience into an Adobe AIR application, read the mobile strategy article.
Conclusion
Obviously there is more complexity to video delivery and the overall application approach, but this demonstrates the advantages of using a runtime with unified video features to reduce costs and increase scale. For successful examples, it’s worth checking out WatchESPN, Snagfilms or Psych Vision 2.
And as always, feedback and questions are welcome – this article is the result of reader comments from the recent survey.
Update 01/16/12: Added Source Code – 720p Video on iPad and Android Tablets with Adobe AIR.
Update 01/27/12: If you use the captive runtime approach, please update Flash Builder/Flash to the latest AIR version for 720p Android playback (AIR 3.1.0.5560 or higher)
Great summary article.
Do you have a list of Android tablets that Adobe might have tested and found H.264 (with StageVideo) playback performance to be acceptable ?
All Android 3.1+ tablets have AIR StageVideo support, meaning native video performance. The performance is dependent on the HW decoder and the encoding settings, but mobile optimized 720p video should work on all those devices (see encoding guidelines).
Tablets with Android 2.2/2.3 might or might not have StageVideo support depending on the chipset – if they do, it's dependent on the HW decoder, but likely 720p. If not, the video object can also perform well depending on the chipset (e.g. Kindle Fire or the original 7-inch Galaxy Tab with firmware upgrade can handle high quality video well), but less popular, older tablets, might not. It's case by case.
Thanks for the info. In my tests Samsung Galaxy Tab 10.1 does not handle video playback well. It stutters playing video even lower than 720p. Toshiba Thrive is better, still stutters occasionally. The Kindle Fire seems to crash even trying to play back H264.
This is my point, I see a lot of claims that streaming in AIR works fine on so and so tablet, but when we try to make one, I dont see results that match the claims. So a sample application with a link to a public 720p stream, and a list of sample devices where you tested them to be acceptable would really help. We can then grab the app..test it on the devices listed and then work to optimize our content to work at least as well as the reference app.
@superabe Thanks for the feedback, I will post a sample application. In your case, something doesn’t seem to be right.
I would confirm you set wmode to direct, and use StageVideo correctly. The encoding settings are also very important. AIR uses the underlying HW decoding, meaning if you set a too high level or profile, the decoder might not support it, and AIR will fallback to SW (resulting in poor performance given that it is HD).
I would be interested in your video asset, maybe you could share it? (if privately, there is a contact form on the right)
@superabe Here is an example – http://www.overdigital.com/2012/01/14/720p-video-on-ipad-and-android-tablets-with-adobe-air/
@Jens Loeffler
Excellent. Thank you.
Can’t wait to try this out next week. About the video assets, I cant share it, due to copyright issues, but if your sample above does not resolve the issue for me, I will try find some creative commons license based video demonstrates the issues and send it over.
Really refreshing to see prompt responses from an Adobe evangelist. Usually the process of resolving a video streaming issue across the AIR, FMS, OSMF, forums is quite frustrating. Appreciate it.
@superabe The Galaxy Tab 10.1 has the same hardware as the Xoom, it should behave the same. Let me know how it works.
@Jens Loeffler
So I tried out your stream above on the Galaxy Tab 10.1 and playback is choppy (nowhere near the quality you get on the desktop). This is especially noticeable on camera panning movements.
@superabe The hardware is identical, this is a bit mysterious. What is your Android version? I tested it again, I do see some drops initially, but then it’s perfectly smooth (not a dropped frame, even during movements). It’s possible this related to the fact it’s progressive download, and the device is trying to save it to cache initially while playing it. You could try switching to RTMP, it should eliminate all caching overhead (same issue with PD on desktop depending on how fast your HD and internet connection is, e.g. slow HD, high bitrate video, and very fast Internet connection could cause this).
@Jens Loeffler I am testing on a Galaxy Tab 10.1 with Android 3.2. I feel its not caching related because I see this on subsequent playback as well. Do you have an email id I can send the apk over to. I wonder if its something to with the way I am packaging the app.
Are you testing on a Xoom ?
@superabe Sure, you can send me the link to the APK via the contact form on the right, happy to test it on the Xoom. What AIR version are you packaging with?
@Jens Loeffler I am packaging it with AIR 3.1 as a captive runtime. I am not able to put out a public link to the APK just yet. If you don’t mind can you send me your email id to [superabe at gmail dot com] and I can send you the APK directly.
Thanks.
@superabe Please double check you use renderMode direct in the application.xml, otherwise the app can’t access StageVideo and falls back to video mode – which, on this device, can’t handle HD content.
@superabe The solution has been added to the article, the Flash Builder AIR SDK has issues with 720p video playback content on Android, upgrade to the latest.
Update 01/27/12: If you use the captive runtime approach, please update Flash Builder/Flash to the latest AIR version for 720p Android playback (AIR 3.1.0.5560 or higher)
Nice article.
Its like you read my mind! You appear to know so much about this, like you wrote the book in it or something. I think that you could do with some pics to drive the message home a little bit, but other than that, this is great blog. A great read. I will certainly be back.
I swear I can’t get Kindle Fire to play any video. I’m not bothering with StageVideo on Android because of so many devices that DON’T support it–plus, my project only needs 320×240 so software rendering is not a problem. However, I have a very detailed a simple case that demonstrates the problems posted here
http://forums.adobe.com/message/4320700
In summary: 3.2 doesn’t crash but the videos end early, 3.1 crashes on Kindle. Please look into this and advise. Thanks!
@phillipkerman Hi Philipp, that’s indeed very odd. The forum is the right place. Can you also try to file a bug with a simple test case? http://bugs.adobe.com. Did you try with other videos / other protocols? jefftapper Might have some tricks for you too, he worked on the Snagfilms application.
@Jens Loeffler @phillipkerman jefftapper Thanks. What I heard internally from Adobe was that it was a known issue for which one day there may be a fix. However, I don’t know if that’s to say crashing on Kindle in 3.1 or never reaching the end of the video on 3.2.
At this point I just need a workaround… and I think for kindle (and these videos which are all pretty short) it will mean an old-school .swf video in the timeline. ugh. My test case is very trim. Basically, play a video and it doesn’t reach the end. I’ll keep trying permutations. I’ll log a bug.
@Jens Loeffler jefftapper also, it looks like another bug (identical but only mentions one device) was “withdrawn”–not sure what that means: 3132375
https://bugbase.adobe.com/index.cfm?event=bug&id=3132375
@phillipkerman jefftapper Definitely file a new bug, and if you can include the video assets / a short test case, it will help. I will check internally.
@phillipkerman jefftapper We can reproduce it, and are looking into it. It would recommend some creative workaround though given the release timing.
@Jens Loeffler jefftapper which one (or both)? That is, in 3.1 it crashes on kindle. on 3.2 no video ever reaches the end (on any device).
@phillipkerman jefftapper We can’t really update 3.1, but you should be able to deploy a newer version via captive runtime.
Is the source code available for the app shown in the video that demonstrates how well HLS H.264 live streaming running in AIR for iOS integrates? Thx
@tleungavoca There is source code in the article referenced at the end. I didn’t make the source of the moving video view available, but it’s very easy to build – it’s a transparent MovieClip with drag functionality, and the StageVideo view syncs to the MovieClip.
I’m trying to call an OSMF movie clip with VideoDisplay control for iOS device, inside a list’s itemrenderer component, its playing in the component very gracefully, but why it looks like it didn’t added in the itemrenderer stage rather in the ‘view’ stage that holds the list component (?) because changing the itemrenderer’s position (when scrolling) not changes the videoDisplay’s position! I’m now wondering how to actually added over the renderer’s stage!
I forgot to mention, this issue happens ‘only’ when running in device.
@santanu4ver You have to move and sync the OSMF StageVideo rectangle manually, since StageVideo is not part of the display list. It works on desktop, since it likely doesn’t use StageVideo.
@Jens Loeffler Thank you so much for your previous post on StageVideo (http://www.overdigital.com/2012/01/14/720p-video-on-ipad-and-android-tablets-with-adobe-air/) and now the idea of how to change/move position of the stageVideo object. Now I can able to track my itemRenderer’s position and therefore I can change the StageVideo’s position too! Thank you! 🙂
@Jens Loeffler I must have to say, these actionscript codes are thousand times easy than code these in XCode 🙂 m/
@santanu4ver Great to hear 😉
i’m currently implementing StageVideo on Ipad 2, usign Flash Builder Premium SDK 4.6.0, it works some time propertly, and some times video doesn’t appear, i use wmode ”direct” on file configuration. params compiler : -swf-version=13 -target-player=11.0
’cause all aplication run on full AS3 proyect, i’m usign the Flesh builder proyect for Ios mobile device specially ipad2, so.. the main problem currently is that my video doesn’t appear on my app, but some times runs perfectly, does any one know how could i fix that problem, i guest the problem is caused per the stage implementation on class StageVideo ( class that i implement to display video). cause i was trying to put in a UIComponent to render it at specific plase, but unfurtunately it doesn’t seems to works, always is place as child of the main application.
so i would like to get an idea of how could i fix the problem of the sometimes stagevideo works and somethimes not.
the other things is to ask abound how could i add the video to an specific ui component, i usually apply a skin to main application to support this option as i read in other post, it works perfectly for the first run but latter it doesn’t appear agaen
@verza123
I’m using FB4 w/ 4.6 as well. Haven’t used any parameter like wmode-direct.
Make sure you have – <renderMode>gpu</renderMode> in your *.xml
-swf-version=13 -target-player=11.0 is also needed.
When your video doesn’t appear, is there sound? – Remember StageVideo only sits on the stage. You cannot append SV to anything. It must be the 1st child on the stage. In terms of skinning, you need to skin from the top down. You cannot put it in a UIComponent, it will not show.
@xcLusfx thanks for you reply, i’m actually having video sometimes, some times just listen the sound and sometimes i have video, the funny think is that if i install the application for first time.
i was using renderMode to direct in app descriptor.xml
my video is only 400x700px and must appear at right middle of the screen y was thinking to create an specific module to load this in the right place, but i suppouse stageVideo will sits in the Application Layer right??, is there any wink of index or a way to move my video to right middle?? i’m verza123 agaen
@verza123 @xcLusfx
There’s no point in writing a module to reposition your sv. Use .x, .y to reposition your sv on the stage, and structure code from above (the video). renderMode-direct doesn’t work for me, only gpu does – i think there’s bugs here.
When I compile down to the ipad, “fast” mode works for me. “normal” mode compile creates another bug. go figure.
You’re using 4.6 – which means you’re using Air 3.1 I’m current using Air 3.2, not sure if that makes a difference.
I’m also writing all this code in pure ActionScript3 – As3 Mobile Project. I”m not using any Flex components.
When you have sound – your video is playing. it is just underneath something, perhaps a component or a sprite. Turn Alpha to 0.5, and i’m pretty sure you’ll see the video THROUGH your displayObject.
@xcLusfx @verza123 yeah you’re right, i update my sdk4.6 with the new air3.3, and it works every time, i’m trying to run with flex project component but seems not working so, i’ll have to deploy everythink with full as3. by the way, with this new version you don’t require anyparams at compiler.
other request, does you know if there is a way to test StageVideo on PC, it always says it doesn’t work
@verza123 @xcLusfx You can try it in a ‘web’ project. but you might be wasting your time. It sort of a pain to code stageVideo.
@xcLusfx @verza123 It should work as a web project if you set “renderMode” to direct in the html template, but I don’t believe it works in the desktop mobile preview of Flash Builder. StageVideo also works in Flex, but you have to manually move the StageVideo view area and sync e.g. for scrolling (can’t add as child to Flex UI components), and set the background alpha to 0.
@Jens Loeffler
“I don’t believe it(stagevideo) works in the desktop mobile preview of Flash Builder.” – that is correct.
@xcLusfx @Jens Loeffler hi there, i need one more help, my app is working perfectly right now, but i got some question… i need to change the video size based on mobile orientation: i’m currrently usign: viewPort() methid, but it seems to change only the canvas or container of the video, so my video appear cut at top and bottom, is there any method that allow to change the video size, and container at the same time, or separet?
@verza123 @xcLusfx @Jens Do you mean you need to rotate the StageVideo? Did you try to change the orientation of the application itself?
@Jens Loeffler @xcLusfx @Jens hi jens thanks for your reply, ofcourse, let me try to explain it, in portrait orientation, the video size is: 250×427, when it goes to landscape it change to: 300×512; i have a listener to resize event, and it works fine, but i guess i’m missing somethink cause my video doesn’t fit the container, in landscape is much more bigger that container so it appear to be cut at top and bottom. so my point is that property “viewPort” only change the container or frame of the video not the video size, that’s why i would llike to know if there is a posibility to change viideo size?
one solution could be change source of the video when orientation change, but it will made to reload the entire video agaen. so i suppous there is another method to solve this. i found this method :
stream.videoStreamSettings.height but it’s read only
@verza123 @xcLusfx @Jens The issue is that you can’t scale the video? If you look at the video embedded in the article, it worked in the test app. What property did you change when trying to scale it?
@Jens Loeffler @xcLusfx @Jens the .viewPort, but i guess i found my bug, just let me try it to see how this work
Hi Jens,
Thanks for the awesome post. I am facing a strange issue with video on Andorid 4.x devices and this thread has given some direction to me. In my case, even after I remove the video, dispose the netstream, do everything to get it to garbage collect, a snapshot of the video still shows up behind the stage. Now, I realize, after reading your post that it would actually be StageVideo. But, I still don’t have any solution, what do I do to remove that snapshot or instance of Video. I thought the issue may be with the spark skins but no, it replicates when I simply create instance of a video and rotate device in quick succession.
Any help here would be greatly appreciated as I am completely stuck at this point and have important releases coming up.
Thanks in advance,
Pulkit
@pulkitgupta80 That is indeed odd. I would try it with a very simple player example, e.g. this one – http://www.overdigital.com/2012/01/14/720p-video-on-ipad-and-android-tablets-with-adobe-air/
If it happens there too, it might be a bug you want to file – http://bugs.adobe.com/flashplayer/
Hi Jen,
Somewhat related question to video on AIR. Any idea what progress is being made on the OSMF library? It looks like work on it has stopped since v 2.0. Is this still actively being developed by Adobe and are there plans to keep it openly available ?
Thx
Hi – OSMF is an open source project, and it will remain in that form. There is a new player that specifically better supports advertisement, but it’s not open source – http://blogs.adobe.com/digitalmedia/2012/11/introducing-the-primetime-media-player-a-new-way-to-distribute-monetize-and-analyze-online-video-content/
@Jens Loeffler
Thx for the reply. Is OSMF being actively worked on? Are there plans for a next version ?
Hi Jens,
Having played around with the StageVideo API a bit now, I have one quick question for you. Is there a way to set the viewport of the classic video object, that AIR has changed into a StageVideo? If yes, I can set it to a 0 width, height rectangle.
Thanks,
Pulkit
Hi Jens,
Having played around with the StageVideo API a bit now, I have one quick question for you. Is there a way to set the viewport of the classic video object, that AIR has changed into a StageVideo? If yes, I can set it to a 0 width, height rectangle.
I tried setting viewport of stage.stageVideos[0] but no avail.
Thanks,
Pulkit
@pulkitgupta80 Did you try to access the video object? The API for the upconverted video object to stagevideo on ICS should still be the actual video object.
@Jens Loeffler Thanks for prompt reply. Yes, the API is video only and that is why we don’t get the viewport property. Can I try any other thing?
@pulkitgupta80 try to set the width on the actual video object, not the view port. that should translate to stagevideo
@Jens LoefflerNothing worked, I had tried a very simple flex app where I added a video object on a UIComponent and attached a NetStream to it. Same issue was present. I tried setting width and height… although videoWidth and videoHeight go to 0, the video still displays in background…
I even tried drawing a 5000 x 5000 graphic on stage to cover it for time being but that was not able to prevent it from showing up.
Finally, I changed the encoding of the video from H264 to H263 and all got resolved. Seems, AIR uses normal video object for the H263 ones. For now, it seems I’d do with no hardware acceleration.
@Jens Loeffler
Nothing worked, I had tried a very simple flex app where I added a video object on a UIComponent and attached a NetStream to it. Same issue was present. I tried setting width and height… although videoWidth and videoHeight go to 0, the video still displays in background…
I even tried drawing a 5000 x 5000 graphic on stage to cover it for time being but that was not able to prevent it from showing up.
Finally, I changed the encoding of the video from H264 to H263 and all got resolved. Seems, AIR uses normal video object for the H263 ones. For now, it seems I’d do with no hardware acceleration.
I have logged a bug at https://bugbase.adobe.com/index.cfm?event=bug&id=3487503
But, have a poor record at adobe bug base, they have deferred all bugs I’ve reported till date.
@pulkitgupta80 Thanks for filing the bug – wondering if jefftapper has some thoughts.. did you try without Flex, only AS3?
@Jens Loeffler @pulkitgupta80 So is the problem removing a stageVideo object, or resizing it?
@JeffTapper @Jens Loeffler Removing it, actually. I tried the sample code in the other post by Jens. Even with the StageText API, could not find a way to dispose the video, you can just set the viewport to 0 width and 0 height rectangle.
However, in case where Video object is converted by AIR itself, there is no way to even resize the video to hide it.
@pulkitgupta80 @Jens Loeffler Can you post your sample code so i can see what you are doing? We have lots of air apps where we switch between StageVideo and other non video content.
@pulkitgupta80@Jens Loeffler Ok, just checked on what we normally do here.
stage.stageVideos[0].attachNetStream(null);
Should stop playback of the video. You can’t actually ‘remove’ a StageVideo object. It always exists and Flash figures out when to build/destroy them. All you can do is to tell it to stop playing a video, which should have the same effect as removing it.
@JeffTapper I recall to have tried it, the video stops but a reflection is there, so to remove it, we need to set the viewport.
However, the problem is not with the stageVideo API. It is with the use case, where AIR uses StageVideo by itself for a normal video object. In this case the viewport API is not available, hence no way to hide the video. Setting Viewport of stage.stageVideos[0] in this case has no effect in this case. This happens only on Android 4.x. I think I would be able to explain better with a sample code.
@pulkitgupta80 please do post some code, so we can be on the same page.
@JeffTapper @pulkitgupta80 Did you try to detect the StageVideo API and use it instead? That should give you control.
@phillipkerman @Jens Loeffler jefftapper Which version of the Kindle Fire are you running? Can you try the snagfilms app on their and see if that is working for you? http://www.amazon.com/s/ref=nb_sb_ss_i_0_5?url=search-alias%3Dmobile-apps&field-keywords=snagfilms&sprefix=snagf%2Cmobile-apps%2C174
@JeffTapper @phillipkerman @Jens Loeffler jefftapper Installed the latest Snag Films app on my gen1 Kindle Fire and it hangs every time on the match.com add–with 20 seconds remaining. Some dude in front of shelves of shoes.
@phillipkerman @Jens Loeffler Very odd, we have been testing on a gen1 kindle fire, and its not behaving like that, but of course its a different ad now. I’ll dig in further
Hello, I am not a developer; but have a vague understanding what is being discussed here due to experience providing interpretive services to the deaf, mainly H263 packets.
The bottom line for me is will adobe AIR work on the samsung tablet 2?
Thanks in advance.
Dan
@TrackerDan Hi dan – this is for H.264 video playback, which allows HW playback. H.263, or rather the Flash variant, Sorenson Spark, is supported too, but not HW accelerated (=won’t work for HD, leading to higher battery impact)
Hello I am using the Nook HD+ and when I am watching a video on a website it tells me I am not allowed to watch unless I have flashplayer version 9.0 or higher. I also have adobe air but it doesn’t work. What can I do?
@Andres Adobe AIR is only for application development – for websites it would be Flash Player, which is not available for Android anymore. You could try to side-load it, but not sure if that works on the Nook – http://www.youtube.com/watch?v=iLjLbm_nBU8
Hello,
What about audio live streaming (no video) ? Does it work on ios device with adobe air ?
Should work if it’s MP3 RTMP streaming based (not AAC), or AAC with HLS. Here is an article for Android, you can properly use similar code on iOS with the right codecs. http://www.overdigital.com/2012/01/18/tune-in-to-live-radio-with-adobe-air/
Obviously, Adobe AIR can no longer play live H.264 streams on Android over RTMP. There’re issues like the one described here: https://forums.adobe.com/thread/1571055 … until Adobe fix those weird bugs
I’ll forward this to the AIR team, thanks for bringing up