I worked for many years with the online video community, with an increasing focus on specific problems high-end users might have. But sometimes it’s also important to go back to the basics and focus on the fundamentals. This is my personal guide for everyone who wants to get started with online video, or is looking for a quick reference.
1) Platforms
What are the platforms you need to target? The answer to this question will impact the scope and costs of an online video roll-out. Must have platforms are desktops, iOS and Android based on the average traffic volume. Additional target platforms are highly depending on your business model. As example, gaming console are very attractive, but the ability to launch a featured application is limited to major brands.
HTML5 video is an easy option to get to the mobile web quickly for an instant in-browser presence. But it comes with some limitations, true scale is only possible if you limit your player experience to most basic video playback features – no DRM, no adaptive streaming, no live.
If you require advanced premium video features, it becomes exponentially more complicated to scale, and in most cases it’s required to fall back to applications.
2) Video codecs
With pretty much every target platform supporting H.264 hardware decoding – a must have feature for battery efficient playback – the choice of a common video codec is pretty straightforward these days.
Alternative codecs are WebM, Ogg Vorbis or the H.264 successor HEVC/H.265. WebM and Ogg Vorbis are primarily used for certain HTML5 browsers. HEVC/H.265 will eventually replace H.264, but doesn’t have a lot of traction across devices yet.
Meanwhile ON2 VP6 (aka Sorenson Spark, .flv) was the first higher quality codec introduced in Flash. It is sometimes still used to avoid the MPEG-LA H.264 license, but it is lacking the sophistication of H.264 in terms of configurability and quality.
This leads to an important point when using H.264 – commercial content requires a specific MPEG-LA license. This didn’t limit, as originally predicted, the widespread use of H.264, but rather is the reason for alternatives like WebM, which goal is to offer a “free” alternative for the open web.
Due to the complex set of encoding options for H.264, finding the right encoding profiles is quite complicated and requires experience. This is why encoding solutions often limit the user configurable options to a minimum. Lisa Larson-Kelley provides interesting insight in encoding best practices.
In addition Maxim Levkov’s encoding cookbook is an excellent resource.
3) Video Containers
Video containers are different from video codecs. A video codec defines the algorithm how a video is encoded and decoded to reduce file size, while a video container contains the encoded video, but also includes additional information.
A container or wrapper format is a metafile format whose specification describes how different data elements and metadata coexist in a computer file. [via Wikipedia]
As example, a HLS video stream is encoded with H.264/AAC, which is the video/audio codec, the container format is MPEG-2 transport stream (.ts), and the m3u8 files are the playlists. A mp4 file used for progressive download is a video container, containing H.264 encoded video.
4) Adaptive Streaming Formats
Adaptive or dynamic streaming is the concept of adapting the quality and bit rates of a stream based on changing network conditions. This is important because standard TCP based content delivery only has limited throughput, and the bandwidth capacity between video servers and clients is unpredictable. With HD video quality delivered over the Internet, adaptive streaming becomes increasingly important.
In contract to simply offering different bit rates to the end user to choose from, adaptive streaming formats allow the automatic change of bit rates based on players analyzing the available bandwidth, and in some cases the available performance, of the playback computer/device.
The main formats are the following:
HLS – HTTP Live Streaming (Apple)
HLS was introduced by Apple to enable adaptive streaming to iOS mobile devices. The spec then was submitted to the IETF, known as the “Pantos” HLS specs. To be able to use a common format, the openness lead to widespread adoption on other mobile devices. On desktop, HLS is less dominant since Flash Player had no native support until recently. Even though the HLS specs are publicly available, the direction of HLS is still controlled by Apple, which lead to emerging, independently controlled, standards like MPEG-Dash.
Due to the IETF submission, it’s also occasionally referenced as HTML5 video, which is not correct due to its “IETF informational” state. HLS is only available via the HTML5 video tag on Apple’s safari browser on iOS and desktop, to a limited degree in the Android Chrome browser, and not available on other browsers.
HLS is widely adopted due its simplicity, but has some disadvantages compared to more modern fragmented mp4 formats (fMP4).
HDS – HTTP Dynamic Streaming (Adobe)
HDS was developed and optimized by Adobe as the HTTP streaming format to the Adobe Flash Player and Adobe AIR. Due to the continuing popularity of Flash Video, HDS is a widely used format for desktop video delivery. It has almost no relevance for mobile delivery, with the only exception being Adobe AIR for Android, Blackberry or Digital Home, where HDS video delivery is possible. It is based on fMP4.
Smooth Streaming (Microsoft)
Smooth Streaming is Microsoft’s fMP4 based video format primarily targeting Microsoft platforms such as Silverlight and Xbox.
MPEG-Dash (MPEG)
MPEG-Dash is an initiative to standardize HTTP streaming to reduce the format fragmentation. It is available in some HTML5 browsers, and the most likely candidate to become a cross browser streaming standard. In a related effort, DRM is introduced via HTML5 Premium Video extensions. Similar to the HTML5 codec support situation, DRM faces fragmentation challenges with different browsers pursuing their own proprietary strategies, while Apple remains committed to HLS.
RTMP Dynamic Streaming (Adobe)
RTMP Dynamic Streaming is the adaptive streaming version of RTMP, a non HTTP-based streaming standard targeting the Adobe Flash/AIR platforms. Even though slowly getting replaced by HDS, it is still widely popular due to its ease of deployment and low live latency.
With all these format options, it is important to deploy a strategy that reduces the overall amount of streaming formats.
4) Additional Services
Besides enabling high quality playback, additional services layered around the video playback experience are relevant, such as QoS monitoring, analytics, social media integration, advertisement, closed captioning, searchable video metadata, and much more. This is beyond the scope of this overview, but is very relevant to promote, measure and increase the end user experience.
Summary
Creating your own online video workflow remains complex in 2013, with constantly increasing device fragmentation, and competing philosophies. Thankfully there are many tools available to reduce the fragmentation and address some of the challenges.
This guide hopefully helped to explain some of the fundamentals that are often assumed as given in the online video industry.
