This article is part of a series of articles I have written covering WPF; improving its performance, its quirks and associated workarounds, as well as porting it to WUA Applications:
- Improving WPF Rendering Performance (Part 1)
- Improving WPF Rendering performance (Part 2)
- Improving WPF Rendering performance (Part 3)
- Troubleshooting Video Playback in WPF
- Writing HLSL Pixel Shaders for WPF
- Porting WPF applications to Windows Universal Apps
For a long time Microsoft has offered an OCX or COM componentised version of the Windows Media player for inclusion into C# applications. With the introduction of WPF a few years back Microsoft required enhanced video support in end user applications. What we ended up with was the MediaElement control, which I think most programmers would say was a half finished attempt.
Most of the problems with MediaElement begins with what it doesn’t expose, such as MediaStateChange events and the ability to determine buffering levels and media issues. For example, if the audio can’t play but the video can, it will still open and no error will be exposed to the application. Other issues include when a file fails to load, it is often difficult to determine the issue (DirectShow filter construction, corrupt media etc.)
Instead the MediaElement is where the magic occurs. Microsoft has replaced the built in DirectShow rendering path used by Windows Media Player and replaced it with the Enhanced Video Renderer (EVR). This allows the composition, mixing and rendering of up to 16 streams simultaneously and is rendered via the Windows Display Manager (WDM) for possible hardware acceleration via DXVA.
This allows the possibility of future improvements and separation to WMP, however when video issues are encountered, this differentiation can cause debugging problems a pain.
A common assumption is that if a video plays in Windows Media Player, it will also play in a WPF application, and usually at around the same CPU load and visual appearance, but this isn’t always the case.
When determining which codecs to use, WMP and EVR have their own (separate) DirectShow Filter Graph constructors and negotiation, this causes problems when the EVR negotiates differently to WMP (e.g. codec packs altering WMP registry settings) To find out these issues, it is best to use GraphEdit which is part of Microsofts DirectX SDK.
In Graphedit you can create your own DirectShow graphs to test rendering, by opening a video file via the menu GraphEdit will use the same logic as WMP. If you replace the ‘Video Renderer’ filter with the ‘Enhanced Video Renderer’ from the DirectShow Filter list you can test how a video plays using the EVR renderer, and thus how it plays with WPF.
If the graph doesn’t throw an error and can be constructed, it should mean that there is no issues causing WPF to technically not be able to play the file, it may however mean that the order of presidence that codecs give themselves is out of order.
It should be noted that in Windows 7 Microsoft included its own set of codecs, which have ultimate presidence, meaning if they believe they can play the file (but can’t) WMP will not try other 3rd party codecs. Some codecs disable this, but if you feel compelled, there is an open source utility called Win7DSFilterTweaker to disable this functionality and give presidence to other codecs; such as ffdshow or LAV Splitter.
The quality of decoding and performance via WPF is different to WMP, this is due to another quality of WPF. Variable framerate rendering. When frames are rendered in WPF, it can internally adjust the framerate to improve performance/decrease CPU load. This is great for most applications but causes problems for video. Sometimes (although has improved in newer versions of WPF) video could become out of sync, or have visual tearing where the video FPW did not match up with the rendering framerate. Roman Hnatiuk has a great article expressing his annoyance at this issue.
This issue can’t be solved easily, and a better user solution may require a lot of pinvoking into custom EVR presenters.