Nico Weber via llvm-dev
2015-Sep-04 17:27 UTC
[llvm-dev] Running tests on OS X 10.10 vs "Killed: 9"
Hi,
building 'check-all' on any of my machines running OS X 10.10 usually
fails
because a few tests fail due to some processes being killed by the kernel
(there's always "Killed: 9" somewhere in lit's error output).
Everything's
fine on 10.9.
How do folks deal with this? Don't use 10.10 for building llvm? Is there
some tweakable to tell the kernel "please don't kill my
processes"?
Here's an example from just now:
FAIL: LLVM :: MC/X86/shuffle-comments.s (18589 of 28139)
******************** TEST 'LLVM :: MC/X86/shuffle-comments.s' FAILED
********************
Script:
--
/Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/llvm-mc
/Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s
-triple=x86_64-unknown-unknown |
/Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/FileCheck
/Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s
--
Exit Code: 137
Command Output (stderr):
--
/Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm-bootstrap/test/MC/X86/Output/shuffle-comments.s.script:
line 1: 47457 Killed: 9
/Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/llvm-mc
/Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s
-triple=x86_64-unknown-unknown
47459 |
/Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/FileCheck
/Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s
Thanks,
Nico
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150904/74533011/attachment.html>
Sean Silva via llvm-dev
2015-Sep-04 19:37 UTC
[llvm-dev] [cfe-dev] Running tests on OS X 10.10 vs "Killed: 9"
I've run into this before; there is a relatively simple fix that Chris Bieneman shared. CC'ing him because I forget the magical incantation. It had something to do with setting `-m 1` on the taskgated command line in some init file IIRC. -- Sean Silva On Fri, Sep 4, 2015 at 10:27 AM, Nico Weber via cfe-dev < cfe-dev at lists.llvm.org> wrote:> Hi, > > building 'check-all' on any of my machines running OS X 10.10 usually > fails because a few tests fail due to some processes being killed by the > kernel (there's always "Killed: 9" somewhere in lit's error output). > Everything's fine on 10.9. > > How do folks deal with this? Don't use 10.10 for building llvm? Is there > some tweakable to tell the kernel "please don't kill my processes"? > > Here's an example from just now: > > FAIL: LLVM :: MC/X86/shuffle-comments.s (18589 of 28139) > ******************** TEST 'LLVM :: MC/X86/shuffle-comments.s' FAILED > ******************** > Script: > -- > /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/llvm-mc > /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s > -triple=x86_64-unknown-unknown | > /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/FileCheck > /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s > -- > Exit Code: 137 > > Command Output (stderr): > -- > /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm-bootstrap/test/MC/X86/Output/shuffle-comments.s.script: > line 1: 47457 Killed: 9 > /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/llvm-mc > /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s > -triple=x86_64-unknown-unknown > 47459 | > /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/FileCheck > /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s > > > Thanks, > Nico > > _______________________________________________ > cfe-dev mailing list > cfe-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150904/47a0c411/attachment.html>
Sean Silva via llvm-dev
2015-Sep-04 19:46 UTC
[llvm-dev] [cfe-dev] Running tests on OS X 10.10 vs "Killed: 9"
On Fri, Sep 4, 2015 at 10:27 AM, Nico Weber via cfe-dev < cfe-dev at lists.llvm.org> wrote:> Hi, > > building 'check-all' on any of my machines running OS X 10.10 usually > fails because a few tests fail due to some processes being killed by the > kernel (there's always "Killed: 9" somewhere in lit's error output). > Everything's fine on 10.9. > > How do folks deal with this? Don't use 10.10 for building llvm? Is there > some tweakable to tell the kernel "please don't kill my processes"? > > Here's an example from just now: > > FAIL: LLVM :: MC/X86/shuffle-comments.s (18589 of 28139) >Btw, for the sake of having this in writing somewhere: one other thing I've run into (I was generating a huge number of lit tests for an artificial reason; wasn't during normal development) is that once you go past ~32K tests (2^15) lit's current way it queues up processes will cause it to completely hang on Mac. Essentially the problem is that it has a for loop like: for t in tests: push_onto_threadsafe_queue(t) ... kick off worker threads that consume from the queue ... unfortunately, push_onto_threadsafe_queue ups an OS semaphore and on mac there is an upper limit of ~32K. Once you hit that the push_onto_threadsafe_queue operation will block and therefore lit will be completely blocked. Someday we will need to e.g. spawn off a feeder thread to do the push_onto_threadsafe_queue loop. Filipe, did you say that you had actually run into the 32K limit at some point during regular development? (what combination of LLVM repos were checked out and being tested simultaneously?) -- Sean Silva> ******************** TEST 'LLVM :: MC/X86/shuffle-comments.s' FAILED > ******************** > Script: > -- > /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/llvm-mc > /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s > -triple=x86_64-unknown-unknown | > /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/FileCheck > /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s > -- > Exit Code: 137 > > Command Output (stderr): > -- > /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm-bootstrap/test/MC/X86/Output/shuffle-comments.s.script: > line 1: 47457 Killed: 9 > /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/llvm-mc > /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s > -triple=x86_64-unknown-unknown > 47459 | > /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/FileCheck > /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s > > > Thanks, > Nico > > _______________________________________________ > cfe-dev mailing list > cfe-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150904/4365c42a/attachment.html>
Sean Silva via llvm-dev
2015-Sep-04 19:49 UTC
[llvm-dev] [cfe-dev] Running tests on OS X 10.10 vs "Killed: 9"
On Fri, Sep 4, 2015 at 12:46 PM, Sean Silva <chisophugis at gmail.com> wrote:> > > On Fri, Sep 4, 2015 at 10:27 AM, Nico Weber via cfe-dev < > cfe-dev at lists.llvm.org> wrote: > >> Hi, >> >> building 'check-all' on any of my machines running OS X 10.10 usually >> fails because a few tests fail due to some processes being killed by the >> kernel (there's always "Killed: 9" somewhere in lit's error output). >> Everything's fine on 10.9. >> >> How do folks deal with this? Don't use 10.10 for building llvm? Is there >> some tweakable to tell the kernel "please don't kill my processes"? >> >> Here's an example from just now: >> >> FAIL: LLVM :: MC/X86/shuffle-comments.s (18589 of 28139) >> > > Btw, for the sake of having this in writing somewhere: one other thing > I've run into (I was generating a huge number of lit tests for an > artificial reason; wasn't during normal development) is that once you go > past ~32K tests (2^15) lit's current way it queues up processes will cause > it to completely hang on Mac. Essentially the problem is that it has a for > loop like: > > for t in tests: > push_onto_threadsafe_queue(t) > ... kick off worker threads that consume from the queue ... >Maybe clearer: for t in tests: push_onto_threadsafe_queue(t) ... kick off worker threads that consume from the queue ... i.e. It pushes all tests onto the queue before it starts kicking off jobs. -- Sean Silva> > unfortunately, push_onto_threadsafe_queue ups an OS semaphore and on mac > there is an upper limit of ~32K. Once you hit that > the push_onto_threadsafe_queue operation will block and therefore lit will > be completely blocked. > > Someday we will need to e.g. spawn off a feeder thread to do the > push_onto_threadsafe_queue loop. > > Filipe, did you say that you had actually run into the 32K limit at some > point during regular development? (what combination of LLVM repos were > checked out and being tested simultaneously?) > > -- Sean Silva > > >> ******************** TEST 'LLVM :: MC/X86/shuffle-comments.s' FAILED >> ******************** >> Script: >> -- >> /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/llvm-mc >> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s >> -triple=x86_64-unknown-unknown | >> /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/FileCheck >> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s >> -- >> Exit Code: 137 >> >> Command Output (stderr): >> -- >> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm-bootstrap/test/MC/X86/Output/shuffle-comments.s.script: >> line 1: 47457 Killed: 9 >> /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/llvm-mc >> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s >> -triple=x86_64-unknown-unknown >> 47459 | >> /Users/thakis/src/chromefetch/src/third_party/llvm-bootstrap/./bin/FileCheck >> /Volumes/MacintoshHD2/src/chromefetch/src/third_party/llvm/test/MC/X86/shuffle-comments.s >> >> >> Thanks, >> Nico >> >> _______________________________________________ >> cfe-dev mailing list >> cfe-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev >> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150904/a83456ac/attachment.html>