As we saw in our last media series post, building a robust HTML5 player and media engine is no easy task, with scores of hidden complexities and possible pitfalls. Luckily, there are a variety of open-source media engines that can be used freely and customized to fit your specific use case.
In this post, we provide an overview of the most important criteria to take into account when choosing an HTML5 media engine that suits your video workflow, customization needs and feature requirements.
Depending on the resources you have available to allocate to the media player, you may either want to use a player’s media engine as-is or really dig in to the internals. We will cover both scenarios in this post. First, we will discuss the general criteria to take into account when choosing an open-source HTML5 player. In the second half of the post, we will delve into more developer-centric features and will talk about what to look for if simply interacting with the media engine API or if you actually want to fork and tweak the video player.
1. General featureS
a. Support & Compatibility: devices / formats / specific features
Device, format and codec dependencies
The first set of criteria to look at is what formats / packaging / codecs the video player’s media engine can handle. These depend greatly on the viewer’s platform, so you will most likely not be able to address your entire audience with a single format and media engine. Today DASH and HLS are by far the most logical choices to address all user platforms. Depending on your legacy workflow, it might be easier to use the first or the second: DASH is extremely similar to Smooth Streaming in terms of specifications, and HLS is widely used on mobile.
Luckily, there are great open-source media engines that support these formats.
Robustness: does your HTML5 PLAYER’S media engine support your special features?
You may be tempted to rush headlong into a media engine, but don’t forget to actually check if your streams work with it. Even if modern streaming formats are standardized, there are still dozens of different ways to create a manifest, and encoders are not all 100% standard compliant. Media engines often suffer from this fragmentation, as they must support edge cases that come from suboptimal encoding/packaging settings, not from the player directly.
A media engine becomes more robust with time and popularity, as more and more developers try their streams, encounter issues, investigate and find workarounds, etc.
Most media engines have online demo pages with a debugging setup, so if you already have a production stream with your encoder, start by trying it out with the player to make sure everything works, and post an issue on the project’s bug-tracker if it does not.
Custom features: DRMs/AES encryption, DVR, subtitles, etc.
If the stream works, list the specific business features you absolutely must have: DRM support, DVR, subtitles, etc. The different media engines cover a broad range of features depending on their focus. Check which features the media engine supports and, if not, whether they are on the roadmap.
As we will see, you can always add features yourself if you have the time and resources; in this case, you should also check if the media engine is easily extensible (we cover this in the second part of the post). The rarer the feature, the more likely it is that you will have to build it yourself.
b. Engine Performance and Quality
Looking beyond the most basic features, performance parameters are decisive in the choice of a media engine. These include startup time & adaptive bitrate algorithm performance, library size, and error handling & retry mechanisms.
Startup time, i.e. the time the player takes to load the first video frame, is paramount. This time can be broken down into:
scripts download time + lib initialization time + download & play of the first fragment
The size of the media engine is important, but not as important as the mechanism used to load the first fragment. For a faster startup time, it is often best to begin with a lower quality and then switch to a higher quality as of the second segment. This leads us to our next consideration: ABR algorithms.
Adaptive bitrate algorithms are crucial for the user experience: a good ABR algorithm reduces startup time, optimizes average bitrate and minimizes re-buffering events. ABR algorithms are (not surprisingly) difficult to get right.
First, the media engine needs to accurately estimate different parameters such as the user’s available bandwidth and CPU usage. These are not directly available, so the engine must approximate them, and some do better than others. Secondly, the media engine need to apply intelligent rules to these estimations. Significant research has been done on ABR, as it is a complex multi-criteria optimization issue for which compromises must be made. Have a look at what ABR algorithms are used by the media engine you are considering, and see if they are consistent with your use case. Certain algorithms work better for low latency streams, while others prefer minimizing buffering over maximizing bitrate, etc.
Error and retry strategies
This is less obvious at first sight. A good media engine doesn’t stop because it fails to obtain a segment, especially in live streaming. It should have smart retry mechanisms to handle network instabilities. This is not easy to see without digging into the media engine code, so we will make sure to talk about it in our benchmark in the next post.
c. “Street Cred”: Community, Issues, Stars…
Even without being a coder, you can quickly see if an open-source project is active, well-maintained, and has a big community around it. The easiest way is to look at the Github project metrics. A good project has:
- A lot of stars, forks and watches
- Frequent commits (> 1 per week)
- A lot of recent issues, and a lot of closed issues
- A lot of merged pull requests
- A lot of broadcasters using it in production
Make sure the media engine you choose is alive and well, and has a wide community that can help you. We’ll also include this information in our upcoming benchmark.
2. Developer-centric features
a. Light interaction with the media engine and API
If you have limited resources and a relatively common streaming use case, you probably don’t want to modify the internals of the media engine but rather focus your efforts on your UX and core business features.
How does the API look and what can you do with it?
In this case, simply make sure the media engine API is simple, understandable and stable. You should also check that it provides everything you need to incorporate special features such as ads or multi-player instance support. Some media engines provide easy extensibility features, like the possibility to replace the ABR switching rules or the downloader module (this is how we add client-acceleration capabilities to the various media engines!). You should also check which setup parameters are customizable: buffer size, live delay, etc. Lastly, it is also important for the API to expose certain playback and traffic events in case you would like to build a QoS analytics system or connect a 3rd-party QoS provider.
Is it easy to debug?
It is important to know where issues come from when interacting with a player’s media engine. A good media engine should have debugging tools that are simple and easy to use, as well as informative debug logs that can be switched on and off.
b. In-depth tweaking
Code architecture & design
A good media engine needs to have both a robust core design and allow external video developers to easily add new features. This is a difficult balance to strike, and it is important not to over-complexify a media engine to make everything extendable. Make sure the parts you would want to change (loader, ABR rules, subtitles) are changeable, and that the media engine architecture is understandable to you.
Code quality and tools
Testing and coverage
Media engines are not easy to test, especially in terms of end-to-end testing; it is difficult to mock a video player and make sure the frames work, even with tools like Selenium. Still, check out if there are unit tests, and how much code is really covered by the tests. As media engines are complex software, the chance of introducing regression bugs in each new version is high, especially when handling a significant number of new features. Finally, as we mentioned in the first section, the more testing streams there are, and the more use cases a media engine has at its disposal to test new versions, the more robust it will be.
As we touched on in section one, having people maintaining the project as well as a wide and active community is probably the most important factor when choosing an open-source media engine. Make sure the media engine has a full-time team working and maintaining it, and see how fast pull requests and new issues are treated. A player’s media engine will probably not have all the features you need. However, if the community is active and those maintaining it quick to integrate the most requested components, you will likely see your priority features added into the core project quickly. This will allow you to avoid forking the player on your side, a huge relief in terms of maintenance.
As we have seen, your own unique strategy, use case, audience, feature requirements and development resources should largely guide your choice of a video player media engine. When examining the different options out there, also make sure to take the time to analyze the quality of the engine in terms of performance, community support and active maintenance. And because we know you’re probably pressed for time, we’ll even do that last part for you! Our next post will offer a benchmark comparing the different HTML5 open-source media engines and the features, formats, etc. they support (including dash.js, shakaplayer, rx-player, hls.js and videojs-contrib-hls). Stay tuned!