wine users - Aug 2015 - RFC: New dependency on FFmpeg?

I have a working implementation of the new xaudio2 API in my tree.
Microsoft is treating it as a replacement for dsound[1], and as a
result, this new API is used by lots of recent games; see Bug
26808[2] for some examples.

Most games that use the xaudio2 API use a version of Microsoft's WMA
codec. Wine doesn't currently have the ability to decode this audio
for playback. In order for the xaudio2 implementation to be useful for
most games, we'll need some way to convert it. I chose to use the
FFmpeg library to decode WMA audio.

This means for most games, Wine's built-in xaudio2 will require that
the 32-bit FFmpeg library be available on the system to successfully
play audio.

The new code only uses the audio decoding features of FFmpeg,
specifically linking only against libavcodec and libavutil. The newest
APIs that we use were introduced in 2013. Most of the rest date from
2011 and a few from 2002. FFmpeg's API isn't stable, but I don't
expect much maintenance burden from the small piece that we're using.

An additional complication is the libav fork. The APIs that we're
using are currently unchanged between FFmpeg and libav, so libav
should function as a drop-in replacement. Further, distros seem to be
migrating back to FFmpeg, if they ever changed. So again, I don't
expect this to be a barrier.

Wine packagers and developers, do you have any thoughts or objections
to depending on FFmpeg/libav's libavutils and libavcodec for xaudio2
support?

[1]
https://msdn.microsoft.com/en-us/library/windows/desktop/ee415813%28v=vs.85%29.aspx

[2] https://bugs.winehq.org/show_bug.cgi?id=26808

Thanks,
Andrew

On 20/08/15 12:47, Andrew Eikum wrote:
> An additional complication is the libav fork. The APIs that we're
> using are currently unchanged between FFmpeg and libav, so libav
> should function as a drop-in replacement. Further, distros seem to be
> migrating back to FFmpeg, if they ever changed. So again, I don't
> expect this to be a barrier.
> 
> Wine packagers and developers, do you have any thoughts or objections
> to depending on FFmpeg/libav's libavutils and libavcodec for xaudio2
> support?
As unfortunate as this is, most applications that uses those libraries
are packaging their own copy of ffmpeg for a few reasons:

- ffmpeg APIs break (on point releases) too much
- Depending on the distro's copy of ffmpeg is very unreliable because of
said point releases
- libav (another project)/ffmpeg incompatibilities (I know you mentioned
this but numerous amount of people are probably still using libav by
default)

So basically I think you would have to propose inclusion of a version of
ffmpeg into Wine's source to get the green-light (and update as
necessary). Otherwise there will be a lot of new bug reports of crashes
landing in a probably stripped copy of the system's ffmpeg or libav,
with users not knowing how to give good information for debugging.

Just my 2 cents.

Andrew

On 08/21/2015 12:19 AM, Andrew Udvare wrote:
> So basically I think you would have to propose inclusion of a version of
> ffmpeg into Wine's source to get the green-light (and update as
> necessary). Otherwise there will be a lot of new bug reports of crashes
> landing in a probably stripped copy of the system's ffmpeg or libav,
> with users not knowing how to give good information for debugging.
> 
ffmpeg/libav sometimes has security issues. Including a vulnerable
version with Wine would make it impossible for the user to update if
Wine were stuck on an old ffmpeg version.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL:
<http://www.winehq.org/pipermail/wine-users/attachments/20150821/ad436474/attachment.sig>

FFmpeg doesn't have a particularly stable API or ABI. Dynamically 
linking to it isn't that great of an idea because the .so number version 
increments fairly often requiring at least a relink. The audio decode 
routine is up to avcodec_decode_audio4 by now, and who knows when 
avcodec_decode_audio5 will happen. The deprecated functions do 
eventually get removed and distros can have them compiled out.

It's a really good lib if you can require specific versions, and either 
static-link it or ship with the version you build with, but I imagine 
the constantly-breaking ABI would be more of a headache for a codebase 
like Wine unless you directly copy over the code (but then, why not just 
take the decoder routines ffmpeg has and wrap them into whatever Windows 
audio decoder API?).

Another issue is that ffmpeg requires C99 to build, whereas Wine doesn't.

On Fri, Aug 21, 2015 at 01:32:35AM -0700, Chris Robinson
wrote:
> FFmpeg doesn't have a particularly stable API or ABI. Dynamically
> linking to it isn't that great of an idea because the .so number
> version increments fairly often requiring at least a relink. The
> audio decode routine is up to avcodec_decode_audio4 by now, and who
> knows when avcodec_decode_audio5 will happen. The deprecated
> functions do eventually get removed and distros can have them
> compiled out.
> 
> It's a really good lib if you can require specific versions, and
> either static-link it or ship with the version you build with, but I
> imagine the constantly-breaking ABI would be more of a headache for
> a codebase like Wine unless you directly copy over the code (but
> then, why not just take the decoder routines ffmpeg has and wrap
> them into whatever Windows audio decoder API?).
> 
Hmm, okay, thanks. We really only need the WMA decoder. That probably
wouldn't be too hard to pull into Wine, but I was hoping to avoid it
for all the obvious reasons. If it's our best option, though, then
maybe that's what we should do.

Andrew

Hello,

Most Linux distributions using libav have voted to move to ffmpeg in the
future, Debian (and thus Ubuntu) being one of the most prominent ones.

Even with ffmpeg API changes, remaining source-compatible should not be a
problem for wine. But I guess the problem is not source-compatibility for
wine but binary-compatibility for CrossOver. In that case, I'd suggest
dlopen'ing and checking for the symbol you want. Maybe even have different
code paths if avcodec_decode_audio5 is available (that means a later
version of ffmpeg is available on that system).

Or you could forget about everything and just take libwmapro from RockBox:

https://github.com/mguentner/rockbox/tree/master/apps/codecs/libwmapro

They took ffmpeg, isolated the WMA code and made it a standalone library.
They even further optimized the code. It's taken from an old version of
ffmpeg (2010) but it works :-)



On Thu, Aug 20, 2015 at 9:47 PM, Andrew Eikum <aeikum at codeweavers.com>
wrote:
> I have a working implementation of the new xaudio2 API in my tree.
> Microsoft is treating it as a replacement for dsound[1], and as a
> result, this new API is used by lots of recent games; see Bug
> 26808[2] for some examples.
>
> Most games that use the xaudio2 API use a version of Microsoft's WMA
> codec. Wine doesn't currently have the ability to decode this audio
> for playback. In order for the xaudio2 implementation to be useful for
> most games, we'll need some way to convert it. I chose to use the
> FFmpeg library to decode WMA audio.
>
> This means for most games, Wine's built-in xaudio2 will require that
> the 32-bit FFmpeg library be available on the system to successfully
> play audio.
>
> The new code only uses the audio decoding features of FFmpeg,
> specifically linking only against libavcodec and libavutil. The newest
> APIs that we use were introduced in 2013. Most of the rest date from
> 2011 and a few from 2002. FFmpeg's API isn't stable, but I
don't
> expect much maintenance burden from the small piece that we're using.
>
> An additional complication is the libav fork. The APIs that we're
> using are currently unchanged between FFmpeg and libav, so libav
> should function as a drop-in replacement. Further, distros seem to be
> migrating back to FFmpeg, if they ever changed. So again, I don't
> expect this to be a barrier.
>
> Wine packagers and developers, do you have any thoughts or objections
> to depending on FFmpeg/libav's libavutils and libavcodec for xaudio2
> support?
>
> [1]
>
https://msdn.microsoft.com/en-us/library/windows/desktop/ee415813%28v=vs.85%29.aspx
>
> [2] https://bugs.winehq.org/show_bug.cgi?id=26808
>
> Thanks,
> Andrew
>
>

-- 
Pau Garcia i Quiles
http://www.elpauer.org
(Due to my workload, I may need 10 days to answer)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.winehq.org/pipermail/wine-users/attachments/20150823/59f0ea15/attachment.html>

On 08/22/2015 04:27 PM, Pau Garcia i Quiles wrote:
> But I guess the problem is not source-compatibility for
> wine but binary-compatibility for CrossOver. In that case, I'd suggest
> dlopen'ing and checking for the symbol you want.
There's also issues with structure changes which won't be caught that 
way. libwmapro sounds like the sanest solution, short of doing the same 
thing with Wine.