Evan Cheng wrote:> On May 21, 2008, at 6:09 AM, Matthijs Kooijman wrote:
>
>
>> Hi,
>>
>> I've been toying around with the testsuite for a bit, and after
>> recompiling
>> llvm-gcc a bunch of times and fixing a nasty bashism bug, I actually
>> got it to
>> run and pass most tests.
>>
>> When running, I noticed a very long runtime on the lemon test. At
>> first I
>> suspected an infinite loop, but it turned out the test simply needed
>> around
>> 1000 seconds to run. Some investigation turned out that the lemon
>> program does
>> a lot of forking (it runs every input file 20 times, so that's
>> around 200
>> forks). However, since the forking happens quite early in the main
>> function,
>> it happens before most of the program is JITted. So, every one of
>> these forked
>> process is taking a few seconds to compile the other needed
>> functions, over
>> and over again.
>>
>> A solution for this problem seems to be to run the lemon test with the
>> -no-lazy option, so things get compiled before forking. In my case,
>> this
>> reduced runtime from 1000+ seconds to only 16 seconds.
>>
>> Intrestingly enough, I can't seem to find similar behaviour in the
>> test
>> results on llvm.org/nightlytest (though I only checked two machines
>> [since the
>> results take ages to load...] of which
>> only one actually ran the test). Am I the only one that is
>> experiencing this?
>>
>> Anyhow, the below patch fixes the problem for me, good to commit?
>>
>
> Ouch. I didn't notice this. I don't think passing this option is
the
> right fix though. Any test that forks a lot is simply not a good
> compiler benchmark. The right thing to do is to eliminate the forking.
> Is that cannot be easily done. We should remove it from our test suite.
>
The forking was introduced because the runtime was too small; the
program keeps global state, so there is no easy way to reset state.
I would prefer to remove the forking (and get short runtimes again),
rather than removing it completely from the testsuite.
What do you think?
Best regards.
--Edwin