llvm dev - Aug 2017 - AArch64 buildbots and PR33972

I'd like to mention that test does not allocate 30TB, it allocates 1TB, the
rest, ~20TB, is reserved (but not actually used) for ASan shadow memory, it
should not be a problem by itself.

The test on your bot failed because it tried to reserve 27TB of memory,
which is more than set by ulimit earlier in this test. I do not immediately
see why it wants to reserve that much shadow for AArch64 config, that's why
before fixing anything, I'd like to know what's going on. How and where
can
I access your bot configuration?



On Thu, Aug 24, 2017 at 2:45 AM, Renato Golin <renato.golin at linaro.org>
wrote:
> On 24 August 2017 at 10:38, Diana Picus <diana.picus at linaro.org>
wrote:
> > Hi all,
> >
> > It turns out we lost coverage of the release configuration on the
> > AArch64 buildbots for a while. This is because the machine that we
> > were running the clang-cmake-aarch64-full bot on became unstable
> > (random shutdowns after a day or two of building).
>
> I risk guess that the random shutdowns started around the time when
> the OOM tests were being introduced, so not likely the machine itself,
> but the tests.
>
> We had similar problems when running glibc memory tests, so I believe
> bringing the bot back up is the only sane way to make sure.
>
> Trying to allocate 30TB is not a good type of test on so many
> different hardware and operating systems we have as buildbots. I think
> we need to be a bit more practical and try to create environments
> where the memory is limited well below the actual hardware (different
> OSs have different ways to do that) and then try to allocate a small
> amount of memory. It would be better to have it working on at least
> one arch/OS than to go the full monty on all OSs and architectures.
>
> --renato
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170824/e0982ce3/attachment.html>

Ah, right, you mentioned that on the bug, sorry for missing it. Yes, this
test should be disabled on 48-bit VMA AArch64 for the same reason it is
disabled on s390 and then it is probably simpler to keep it disabled on
aarch64 altogether, r311674 does just that (thank you!).

On Thu, Aug 24, 2017 at 3:16 PM, Aleksey Shlyapnikov <alekseys at
google.com>
wrote:
> I'd like to mention that test does not allocate 30TB, it allocates 1TB,
> the rest, ~20TB, is reserved (but not actually used) for ASan shadow
> memory, it should not be a problem by itself.
>
> The test on your bot failed because it tried to reserve 27TB of memory,
> which is more than set by ulimit earlier in this test. I do not immediately
> see why it wants to reserve that much shadow for AArch64 config, that's
why
> before fixing anything, I'd like to know what's going on. How and
where can
> I access your bot configuration?
>
>
>
> On Thu, Aug 24, 2017 at 2:45 AM, Renato Golin <renato.golin at
linaro.org>
> wrote:
>
>> On 24 August 2017 at 10:38, Diana Picus <diana.picus at
linaro.org> wrote:
>> > Hi all,
>> >
>> > It turns out we lost coverage of the release configuration on the
>> > AArch64 buildbots for a while. This is because the machine that we
>> > were running the clang-cmake-aarch64-full bot on became unstable
>> > (random shutdowns after a day or two of building).
>>
>> I risk guess that the random shutdowns started around the time when
>> the OOM tests were being introduced, so not likely the machine itself,
>> but the tests.
>>
>> We had similar problems when running glibc memory tests, so I believe
>> bringing the bot back up is the only sane way to make sure.
>>
>> Trying to allocate 30TB is not a good type of test on so many
>> different hardware and operating systems we have as buildbots. I think
>> we need to be a bit more practical and try to create environments
>> where the memory is limited well below the actual hardware (different
>> OSs have different ways to do that) and then try to allocate a small
>> amount of memory. It would be better to have it working on at least
>> one arch/OS than to go the full monty on all OSs and architectures.
>>
>> --renato
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170824/d60f5e84/attachment.html>

Great, glad we got that sorted out. Thanks!

On 25 August 2017 at 00:52, Aleksey Shlyapnikov <alekseys at google.com>
wrote:
> Ah, right, you mentioned that on the bug, sorry for missing it. Yes, this
> test should be disabled on 48-bit VMA AArch64 for the same reason it is
> disabled on s390 and then it is probably simpler to keep it disabled on
> aarch64 altogether, r311674 does just that (thank you!).
>
> On Thu, Aug 24, 2017 at 3:16 PM, Aleksey Shlyapnikov <alekseys at
google.com>
> wrote:
>>
>> I'd like to mention that test does not allocate 30TB, it allocates
1TB,
>> the rest, ~20TB, is reserved (but not actually used) for ASan shadow
memory,
>> it should not be a problem by itself.
>>
>> The test on your bot failed because it tried to reserve 27TB of memory,
>> which is more than set by ulimit earlier in this test. I do not
immediately
>> see why it wants to reserve that much shadow for AArch64 config,
that's why
>> before fixing anything, I'd like to know what's going on. How
and where can
>> I access your bot configuration?
>>
>>
>>
>> On Thu, Aug 24, 2017 at 2:45 AM, Renato Golin <renato.golin at
linaro.org>
>> wrote:
>>>
>>> On 24 August 2017 at 10:38, Diana Picus <diana.picus at
linaro.org> wrote:
>>> > Hi all,
>>> >
>>> > It turns out we lost coverage of the release configuration on
the
>>> > AArch64 buildbots for a while. This is because the machine
that we
>>> > were running the clang-cmake-aarch64-full bot on became
unstable
>>> > (random shutdowns after a day or two of building).
>>>
>>> I risk guess that the random shutdowns started around the time when
>>> the OOM tests were being introduced, so not likely the machine
itself,
>>> but the tests.
>>>
>>> We had similar problems when running glibc memory tests, so I
believe
>>> bringing the bot back up is the only sane way to make sure.
>>>
>>> Trying to allocate 30TB is not a good type of test on so many
>>> different hardware and operating systems we have as buildbots. I
think
>>> we need to be a bit more practical and try to create environments
>>> where the memory is limited well below the actual hardware
(different
>>> OSs have different ways to do that) and then try to allocate a
small
>>> amount of memory. It would be better to have it working on at least
>>> one arch/OS than to go the full monty on all OSs and architectures.
>>>
>>> --renato
>>
>>
>