Hi all, With some downtime it's time for some CMake fixes. Most critically is the SSE fixes to avoid crashes that is described in 154 and 132 in github. Patch 5 should address this and also adding APPROX-FLOAT option. Hopefully this can give some gains for those of us running on Windows servers.j I went through the pull request and picked out a few that will ease up integration for consumers of Opus. Those patches are also added here with the original authors as commiters. I will go through the rest of the CMake related pull requests as well later this week. It would be nice to setup a CI-Pipeline with CMake builds whoever owns that on feel free to reach out to me and I can do the work needed. //Marcus -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xiph.org/pipermail/opus/attachments/20191218/73476b76/attachment-0001.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: 0004-CMake-use-PACKAGE_VERSION-for-the-pkg-config-file.patch Type: application/octet-stream Size: 1110 bytes Desc: 0004-CMake-use-PACKAGE_VERSION-for-the-pkg-config-file.patch URL: <http://lists.xiph.org/pipermail/opus/attachments/20191218/73476b76/attachment-0005.obj> -------------- next part -------------- A non-text attachment was scrubbed... Name: 0005-CMake-SSE-fixes-only-compile-SSE-source-with-SSE-fla.patch Type: application/octet-stream Size: 10213 bytes Desc: 0005-CMake-SSE-fixes-only-compile-SSE-source-with-SSE-fla.patch URL: <http://lists.xiph.org/pipermail/opus/attachments/20191218/73476b76/attachment-0006.obj> -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-CMake-add-option-to-set-BUILD_SHARED_LIBS-variable.patch Type: application/octet-stream Size: 1712 bytes Desc: 0001-CMake-add-option-to-set-BUILD_SHARED_LIBS-variable.patch URL: <http://lists.xiph.org/pipermail/opus/attachments/20191218/73476b76/attachment-0007.obj> -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-CMake-Add-shared-library-to-features.patch Type: application/octet-stream Size: 1224 bytes Desc: 0002-CMake-Add-shared-library-to-features.patch URL: <http://lists.xiph.org/pipermail/opus/attachments/20191218/73476b76/attachment-0008.obj> -------------- next part -------------- A non-text attachment was scrubbed... Name: 0003-CMake-Add-alias-Opus-opus-for-opus-library.-Useful-f.patch Type: application/octet-stream Size: 743 bytes Desc: 0003-CMake-Add-alias-Opus-opus-for-opus-library.-Useful-f.patch URL: <http://lists.xiph.org/pipermail/opus/attachments/20191218/73476b76/attachment-0009.obj>
Hi, This is great! I've literally run into the SSE/AVX issue over the past two days and was about to start putting together a PR myself. When using libopus compiled with the CMake toolchain on the Google Android emulator, `-mavx` causes SIGILL because it's vectorising the variadic function parameters and the emulator doesn't support AVX. I'd really appreciate if these make their way into master quickly because this has been causing me a bit of a headache. Thanks! On Wed, 18 Dec 2019 at 00:00, Marcus Asteborg <xnorpx at outlook.com> wrote:> Hi all, > > With some downtime it's time for some CMake fixes. > > Most critically is the SSE fixes to avoid crashes that is described in 154 > and 132 in github. Patch 5 should address this and also adding APPROX-FLOAT > option. > Hopefully this can give some gains for those of us running on Windows > servers.j > > I went through the pull request and picked out a few that will ease up > integration for consumers of Opus. Those patches are also added here with > the original authors as commiters. > > I will go through the rest of the CMake related pull requests as well > later this week. > > It would be nice to setup a CI-Pipeline with CMake builds whoever owns > that on feel free to reach out to me and I can do the work needed. > > //Marcus > _______________________________________________ > opus mailing list > opus at xiph.org > http://lists.xiph.org/mailman/listinfo/opus >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xiph.org/pipermail/opus/attachments/20200108/53737e2c/attachment.html>
Did the patch 5 split / AVX fix work get merged at all, I didn't see any more mails about it? Thanks! On Tue, 14 Jan 2020 at 21:34, Marcus Asteborg <xnorpx at outlook.com> wrote:> Awesome thanks! Good comments. > > Please apply patch 1-4 and I prepare another iteration on patch 5 for you > too look at. > > //Marcus > ------------------------------ > *From:* Jean-Marc Valin <jmvalin at jmvalin.ca> > *Sent:* Tuesday, January 14, 2020 08:24 > *To:* Marcus Asteborg <xnorpx at outlook.com> > *Cc:* Simon Jackson <simon.jackson at sonocent.com> > *Subject:* Re: [opus] CMake patches > > Hi Marcus, > > So I looked at the patches and have no issue with patches 1-4. For > patches 5 I have a few comments/questions. > > - The patch should probably be split in 3 (alloca, fast-math, and > SSE/AVX) to be clearer and easier to bisect if something goes wrong. > - About the alloca part, I'm not sure I understand what's happening but > it looks like alloca gets enabled only with MSVC and not gcc? That's > fine if gcc gets VAR_ARRAYS instead, but otherwise it's suboptimal and > may lead to problems. > - About FLOAT_APPROX, this is not the same as -ffast-math. FLOAT_APPROX > means that the code will approximate functions like exp() and log() by > directly playing with the bits *assuming* that the float data is in IEEE > 754 format. That's the case for x86 and ARM, but there are odd > architectures that implement non-754 float. Enabling FLOAT_APPROX on > those would break Opus badly. So it would be best to only enable > FLOAT_APPROX on archs that are known to have 754 float. > - About SSE/AVX, I read the post about why MAY_HAVE could break with > MSVC. Note that the Opus codebase uses static for every function defined > in header files, so that part shouldn't be a problem. OTOH, the math.h > problem may still be there. If you have a way to solve the math.h > problem, cpu detection on MSVC might be an option though. > > Cheers, > > Jean-Marc > > > On 1/9/20 12:48 AM, Marcus Asteborg wrote: > > Ping @Jean-Marc Valin <mailto:jmvalin at jmvalin.ca <jmvalin at jmvalin.ca>> ! > > ------------------------------------------------------------------------ > > *From:* Simon Jackson <simon.jackson at sonocent.com> > > *Sent:* Wednesday, January 8, 2020 09:54 > > *To:* Marcus Asteborg <xnorpx at outlook.com> > > *Cc:* opus at xiph.org <opus at xiph.org>; Jean-Marc Valin <jmvalin at jmvalin.ca > > > > *Subject:* Re: [opus] CMake patches > > Hi, > > > > This is great! I've literally run into the SSE/AVX issue over the past > > two days and was about to start putting together a PR myself. When using > > libopus compiled with the CMake toolchain on the Google Android > > emulator, `-mavx` causes SIGILL because it's vectorising the variadic > > function parameters and the emulator doesn't support AVX. I'd really > > appreciate if these make their way into master quickly because this has > > been causing me a bit of a headache. > > > > Thanks! > > > > On Wed, 18 Dec 2019 at 00:00, Marcus Asteborg <xnorpx at outlook.com > > <mailto:xnorpx at outlook.com <xnorpx at outlook.com>>> wrote: > > > > Hi all, > > > > With some downtime it's time for some CMake fixes. > > > > Most critically is the SSE fixes to avoid crashes that is described > > in 154 and 132 in github. Patch 5 should address this and also > > adding APPROX-FLOAT option. > > Hopefully this can give some gains for those of us running on > > Windows servers.j > > > > I went through the pull request and picked out a few that will ease > > up integration for consumers of Opus. Those patches are also added > > here with the original authors as commiters. > > > > I will go through the rest of the CMake related pull requests as > > well later this week. > > > > It would be nice to setup a CI-Pipeline with CMake builds whoever > > owns that on feel free to reach out to me and I can do the work > needed. > > > > //Marcus > > _______________________________________________ > > opus mailing list > > opus at xiph.org <mailto:opus at xiph.org <opus at xiph.org>> > > > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.xiph.org%2Fmailman%2Flistinfo%2Fopus&data=02%7C01%7C%7Cc15671b24d8e491dc19508d7990e4df7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637146159020540767&sdata=GdeehvmtVJ2EIjN9ZbfFBhVVfQHR4jaOoQoqeHhhZAY%3D&reserved=0 > > < > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.xiph.org%2Fmailman%2Flistinfo%2Fopus&data=02%7C01%7C%7Cc15671b24d8e491dc19508d7990e4df7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637146159020540767&sdata=GdeehvmtVJ2EIjN9ZbfFBhVVfQHR4jaOoQoqeHhhZAY%3D&reserved=0 > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xiph.org/pipermail/opus/attachments/20200407/59057f4e/attachment.html>
Not yet, Haven't got time to finalize it.
But hopefully next month or so.
Here is devbranch: https://github.com/xnorpx/opus/tree/dev/cmakefixes2
________________________________
From: Simon Jackson <simon.jackson at sonocent.com>
Sent: Tuesday, April 7, 2020 03:47
To: Marcus Asteborg <xnorpx at outlook.com>; opus at xiph.org <opus at
xiph.org>
Cc: Jean-Marc Valin <jmvalin at jmvalin.ca>
Subject: Re: [opus] CMake patches
Did the patch 5 split / AVX fix work get merged at all, I didn't see any
more mails about it?
Thanks!
On Tue, 14 Jan 2020 at 21:34, Marcus Asteborg <xnorpx at
outlook.com<mailto:xnorpx at outlook.com>> wrote:
Awesome thanks! Good comments.
Please apply patch 1-4 and I prepare another iteration on patch 5 for you too
look at.
//Marcus
________________________________
From: Jean-Marc Valin <jmvalin at jmvalin.ca<mailto:jmvalin at
jmvalin.ca>>
Sent: Tuesday, January 14, 2020 08:24
To: Marcus Asteborg <xnorpx at outlook.com<mailto:xnorpx at
outlook.com>>
Cc: Simon Jackson <simon.jackson at sonocent.com<mailto:simon.jackson at
sonocent.com>>
Subject: Re: [opus] CMake patches
Hi Marcus,
So I looked at the patches and have no issue with patches 1-4. For
patches 5 I have a few comments/questions.
- The patch should probably be split in 3 (alloca, fast-math, and
SSE/AVX) to be clearer and easier to bisect if something goes wrong.
- About the alloca part, I'm not sure I understand what's happening but
it looks like alloca gets enabled only with MSVC and not gcc? That's
fine if gcc gets VAR_ARRAYS instead, but otherwise it's suboptimal and
may lead to problems.
- About FLOAT_APPROX, this is not the same as -ffast-math. FLOAT_APPROX
means that the code will approximate functions like exp() and log() by
directly playing with the bits *assuming* that the float data is in IEEE
754 format. That's the case for x86 and ARM, but there are odd
architectures that implement non-754 float. Enabling FLOAT_APPROX on
those would break Opus badly. So it would be best to only enable
FLOAT_APPROX on archs that are known to have 754 float.
- About SSE/AVX, I read the post about why MAY_HAVE could break with
MSVC. Note that the Opus codebase uses static for every function defined
in header files, so that part shouldn't be a problem. OTOH, the math.h
problem may still be there. If you have a way to solve the math.h
problem, cpu detection on MSVC might be an option though.
Cheers,
Jean-Marc
On 1/9/20 12:48 AM, Marcus Asteborg wrote:> Ping @Jean-Marc Valin <mailto:jmvalin at jmvalin.ca> !
> ------------------------------------------------------------------------
> *From:* Simon Jackson <simon.jackson at
sonocent.com<mailto:simon.jackson at sonocent.com>>
> *Sent:* Wednesday, January 8, 2020 09:54
> *To:* Marcus Asteborg <xnorpx at outlook.com<mailto:xnorpx at
outlook.com>>
> *Cc:* opus at xiph.org<mailto:opus at xiph.org> <opus at
xiph.org<mailto:opus at xiph.org>>; Jean-Marc Valin <jmvalin at
jmvalin.ca<mailto:jmvalin at jmvalin.ca>>
> *Subject:* Re: [opus] CMake patches
> Hi,
>
> This is great! I've literally run into the SSE/AVX issue over the past
> two days and was about to start putting together a PR myself. When using
> libopus compiled with the CMake toolchain on the Google Android
> emulator, `-mavx` causes SIGILL because it's vectorising the variadic
> function parameters and the emulator doesn't support AVX. I'd
really
> appreciate if these make their way into master quickly because this has
> been causing me a bit of a headache.
>
> Thanks!
>
> On Wed, 18 Dec 2019 at 00:00, Marcus Asteborg <xnorpx at
outlook.com<mailto:xnorpx at outlook.com>
> <mailto:xnorpx at outlook.com>> wrote:
>
> Hi all,
>
> With some downtime it's time for some CMake fixes.
>
> Most critically is the SSE fixes to avoid crashes that is described
> in 154 and 132 in github. Patch 5 should address this and also
> adding APPROX-FLOAT option.
> Hopefully this can give some gains for those of us running on
> Windows servers.j
>
> I went through the pull request and picked out a few that will ease
> up integration for consumers of Opus. Those patches are also added
> here with the original authors as commiters.
>
> I will go through the rest of the CMake related pull requests as
> well later this week.
>
> It would be nice to setup a CI-Pipeline with CMake builds whoever
> owns that on feel free to reach out to me and I can do the work needed.
>
> //Marcus
> _______________________________________________
> opus mailing list
> opus at xiph.org<mailto:opus at xiph.org> <mailto:opus at
xiph.org>
>
https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.xiph.org%2Fmailman%2Flistinfo%2Fopus&data=02%7C01%7C%7Cc15671b24d8e491dc19508d7990e4df7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637146159020540767&sdata=GdeehvmtVJ2EIjN9ZbfFBhVVfQHR4jaOoQoqeHhhZAY%3D&reserved=0<https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.xiph.org%2Fmailman%2Flistinfo%2Fopus&data=02%7C01%7C%7C0060f1b1f8034b2d3ca408d7dae113d8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637218532552010001&sdata=C0YBw%2FN06EXY12Ok0aAEgfjm8HnS1Z43ChSvPYeAtfw%3D&reserved=0>
>
<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.xiph.org%2Fmailman%2Flistinfo%2Fopus&data=02%7C01%7C%7Cc15671b24d8e491dc19508d7990e4df7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637146159020540767&sdata=GdeehvmtVJ2EIjN9ZbfFBhVVfQHR4jaOoQoqeHhhZAY%3D&reserved=0<https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.xiph.org%2Fmailman%2Flistinfo%2Fopus&data=02%7C01%7C%7C0060f1b1f8034b2d3ca408d7dae113d8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637218532552019994&sdata=yVlKt2LPjxaxjB%2FlmSGn6NWWkuPLmC%2F80kK7J29ROX0%3D&reserved=0>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.xiph.org/pipermail/opus/attachments/20200407/d4968675/attachment.html>