HLS Streaming — How to Measure HTTP Live Stream Performance

By Andre Cheung, Director of Global Alliances, EdgeCast (a previous version of this post was published on Andre’s blog).

A recent Devoncroft’s 2013 Big Broadcast Survey (BBS) shows multi-platform content delivery is the most important trend considered by the broadcasting industry. It is not surprising news considering the time we spend viewing videos on non-TV devices, especially smartphones and tablets.

HTTP Live Streaming

Video streaming on iOS devices is supported by HLS (HTTP Live Streaming). HLS democratizes video streaming: everyone can now easily stream bitrate adaptive videos to iOS devices by using a low cost HTTP codec.

The challenges of streaming to Android

In comparison, it is much more challenging to stream videos to other mobile platforms. Here is a good reference article from EdgeCast client LongTail: The Pain of Live Streaming on Android

Now that we can easily stream videos to iOS devices, how do we measure the video performance? How do we validate that HLS videos delivered from a CDN are much better than those delivered from the customer origin? Which bitrate adaptive video stream on my demo page — HLS Performance Measurement — provides a better experience?

HLS Performance Measurement

Rather than subjective measurement by eye, PocketProbe Free from Bridge Technologies provides an objective test of the HLS video experience:


I did some PocketProbe tests around 12:30pm on June 26, 2013 (Hong Kong time). Left is the test result of the stream directly from the customer origin (CO); on the right is the result from the CDN:


The results

1. Frequency

The CDN stream was delivered more frequently at higher speed than the CO stream.

2. Speed

The CDN stream was retrieved much faster than the CO stream.

3. Efficiency

The CDN stream was delivered constantly at the highest bitrate, while the CO stream switched to lower bit-rates during the delivery.

Just by looking at the pictures side-by-side, we can see that the CDN stream resolution and quality is better. Hence, in order to provide users with the best possible viewing experience across devices, streaming files should always be offloaded to the edge.