llvm dev - Feb 2018 - [RFC] [lld] Replace use of 'concurrency::parallel_for_each' with standard library

Hello all,

Last year I submitted a bug regarding the use of 'parallel_for_each' and
Windows Concurrency Library. We have found that when compiling in Visual
Studio 2015 and 2017 with /MTd, the use of
'concurrency::parallel_for_each'
can cause unhandled exceptions when the program is exiting.

https://bugs.llvm.org/show_bug.cgi?id=34976

After having some difficulty clarifying this issue with Microsoft I
submitted the issue again this week and have found that they are aware of
the problem. However, I've yet to be given any information on a timescale
for a fix.

https://developercommunity.visualstudio.com/content/problem/198027/use-of-concurrencyparallel-for-each-with-the-multi.html

For our team this bug has caused problems when testing LLD. We suggest the
removal of this MSVC specific functionality in "llvm::parallel" until
a fix
is made or alternative found. This could be easily swapped to use the
existing standard library implementation in the meantime.

Owen Reynolds
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180215/55b2b3e9/attachment.html>

Thank you so much for tracking down such a tricky issue.

I agree that we should disable it. Should we just delete the ifdefs? We
could in theory disable it just for debug builds, but that could create
some hard to reproduce bugs.

Cheers,
Rafael

Owen Reynolds via llvm-dev <llvm-dev at lists.llvm.org> writes:
> Hello all,
>
> Last year I submitted a bug regarding the use of
'parallel_for_each' and
> Windows Concurrency Library. We have found that when compiling in Visual
> Studio 2015 and 2017 with /MTd, the use of
'concurrency::parallel_for_each'
> can cause unhandled exceptions when the program is exiting.
>
> https://bugs.llvm.org/show_bug.cgi?id=34976
>
> After having some difficulty clarifying this issue with Microsoft I
> submitted the issue again this week and have found that they are aware of
> the problem. However, I've yet to be given any information on a
timescale
> for a fix.
>
>
https://developercommunity.visualstudio.com/content/problem/198027/use-of-concurrencyparallel-for-each-with-the-multi.html
>
> For our team this bug has caused problems when testing LLD. We suggest the
> removal of this MSVC specific functionality in "llvm::parallel"
until a fix
> is made or alternative found. This could be easily swapped to use the
> existing standard library implementation in the meantime.
>
> Owen Reynolds
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

I personally haven't seen the stability issue on my Windows machine, and
looks like /MDd is passed to the command line for a debug build. I wonder
how you would get a build with the /MTd option. Did you explicitly pass the
option when you run CMake?

On Thu, Feb 15, 2018 at 7:33 AM, Owen Reynolds via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hello all,
>
> Last year I submitted a bug regarding the use of
'parallel_for_each' and
> Windows Concurrency Library. We have found that when compiling in Visual
> Studio 2015 and 2017 with /MTd, the use of
'concurrency::parallel_for_each'
> can cause unhandled exceptions when the program is exiting.
>
> https://bugs.llvm.org/show_bug.cgi?id=34976
>
> After having some difficulty clarifying this issue with Microsoft I
> submitted the issue again this week and have found that they are aware of
> the problem. However, I've yet to be given any information on a
timescale
> for a fix.
>
> https://developercommunity.visualstudio.com/content/problem/198027/use-of-
> concurrencyparallel-for-each-with-the-multi.html
>
> For our team this bug has caused problems when testing LLD. We suggest the
> removal of this MSVC specific functionality in "llvm::parallel"
until a fix
> is made or alternative found. This could be easily swapped to use the
> existing standard library implementation in the meantime.
>
> Owen Reynolds
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180215/7808e055/attachment.html>

Hi,

Unfortunately, it isn't quite as simple as switching to the existing STL
implementation as Owen suggested. The existing STL implementation is not
entirely Windows friendly and can cause various crashes on exit too. We
have some local patches to workaround these issues. I posted an
"enhanced"
Windows & Linux version of these local patches in https://reviews.llvm.org/
D40366 as a potential solution to a crash on exit problem seen on Linux
(see llvm-commits thread "std::thread::detach() rarely crashes in
Parallel.cpp").

So to disable the use of PPL on Windows would require some changes to the
existing STL implementation.

Cheers,
Andrew

On 15 February 2018 at 16:32, Rafael Avila de Espindola via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Thank you so much for tracking down such a tricky issue.
>
> I agree that we should disable it. Should we just delete the ifdefs? We
> could in theory disable it just for debug builds, but that could create
> some hard to reproduce bugs.
>
> Cheers,
> Rafael
>
> Owen Reynolds via llvm-dev <llvm-dev at lists.llvm.org> writes:
>
> > Hello all,
> >
> > Last year I submitted a bug regarding the use of
'parallel_for_each' and
> > Windows Concurrency Library. We have found that when compiling in
Visual
> > Studio 2015 and 2017 with /MTd, the use of
'concurrency::parallel_for_
> each'
> > can cause unhandled exceptions when the program is exiting.
> >
> > https://bugs.llvm.org/show_bug.cgi?id=34976
> >
> > After having some difficulty clarifying this issue with Microsoft I
> > submitted the issue again this week and have found that they are aware
of
> > the problem. However, I've yet to be given any information on a
timescale
> > for a fix.
> >
> > https://developercommunity.visualstudio.com/content/
> problem/198027/use-of-concurrencyparallel-for-each-with-the-multi.html
> >
> > For our team this bug has caused problems when testing LLD. We suggest
> the
> > removal of this MSVC specific functionality in
"llvm::parallel" until a
> fix
> > is made or alternative found. This could be easily swapped to use the
> > existing standard library implementation in the meantime.
> >
> > Owen Reynolds
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180215/5f756e84/attachment.html>