llvm dev - Apr 2020 - LLD default page size for arm32

Hello,

In the recent days we have been debugging a really thorny issue where
binaries build with clang and linked with lld was just "Killed" when
started on a specific armv7 device we ship on.

After quite a bit of head scratching it turns out that the kernel on this
device ships with a 32k default page size (getconf PAGESIZE) and lld uses
4k default page size.

We fixed this by passing -zmax-page-size=0x10000 to lld.

The default page size in GNU ld for arm is 64k so binaries linked with ld
just worked on this device.

I put the question in the discord lld channel and after a bit back and
forth I think it's better to discuss it here.

Is 4k the right default for lld on Arm32? Does anyone know if there is a
strong reason to not change this?

Since we have a workaround I am not in a hurry to change this - but if
nothing else I hope this email will help someone else finding the solution
to the problem I had.

Thanks,
Tobias
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200401/da259734/attachment.html>

Hello,
> Is 4k the right default for lld on Arm32? Does anyone know if there is a
strong reason to not change this?
>
At the time the Arm support for LLD was added, 4k page size was the overwhelming
default for the major platforms that wanted to use LLD (largely those migrating
from Gold which also uses 4k page size), including several that were quite size
conscious and had reduced the page-size to 4k from the 64k default on AArch64
due to size concerns.

So a reason, whether it counts as strong or not, is that moving to 64k page size
would mean a lot of users having to switch back to 4k pages.
> Since we have a workaround I am not in a hurry to change this - but if
nothing else I hope this email will help someone else finding the solution to
the problem I had.
Thanks for reporting it. I think it will have to be a community decision. If the
number of people with > 4k page size systems using LLD is very small and the
number using 4k and care about the size is high, then it is may be more
disruptive to change. However if there is a significant minority that do then it
is worth changing to 64k for compatibility.

At the moment, based on the platforms that have adopted LLD so far, I think most
have used 4k as this is the first problem report citing it. If the user base of
LLD grows, with more platforms using a greater than 4k page size then it makes
sense to aim for compatibility and raise the page size.

If you've got access would you be able to raise a PR with a title of
something like Consider increasing LLD Arm default pagesize? We can collect
opinions there.

Peter

________________________________________
From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Tobias
Hieta via llvm-dev <llvm-dev at lists.llvm.org>
Sent: 01 April 2020 20:33
To: llvm-dev
Subject: [llvm-dev] LLD default page size for arm32

Hello,

In the recent days we have been debugging a really thorny issue where binaries
build with clang and linked with lld was just "Killed" when started on
a specific armv7 device we ship on.

After quite a bit of head scratching it turns out that the kernel on this device
ships with a 32k default page size (getconf PAGESIZE) and lld uses 4k default
page size.

We fixed this by passing -zmax-page-size=0x10000 to lld.

The default page size in GNU ld for arm is 64k so binaries linked with ld just
worked on this device.

I put the question in the discord lld channel and after a bit back and forth I
think it's better to discuss it here.

Is 4k the right default for lld on Arm32? Does anyone know if there is a strong
reason to not change this?

Since we have a workaround I am not in a hurry to change this - but if nothing
else I hope this email will help someone else finding the solution to the
problem I had.

Thanks,
Tobias

Thanks for the reply Peter,

I have uploaded a diff to phab: https://reviews.llvm.org/D77330

-- Tobias

On Thu, Apr 2, 2020 at 10:18 AM Peter Smith <Peter.Smith at arm.com>
wrote:
>
> Hello,
>
> > Is 4k the right default for lld on Arm32? Does anyone know if there is
a strong reason to not change this?
> >
>
> At the time the Arm support for LLD was added, 4k page size was the
overwhelming default for the major platforms that wanted to use LLD (largely
those migrating from Gold which also uses 4k page size), including several that
were quite size conscious and had reduced the page-size to 4k from the 64k
default on AArch64 due to size concerns.
>
> So a reason, whether it counts as strong or not, is that moving to 64k page
size would mean a lot of users having to switch back to 4k pages.
>
> > Since we have a workaround I am not in a hurry to change this - but if
nothing else I hope this email will help someone else finding the solution to
the problem I had.
>
> Thanks for reporting it. I think it will have to be a community decision.
If the number of people with > 4k page size systems using LLD is very small
and the number using 4k and care about the size is high, then it is may be more
disruptive to change. However if there is a significant minority that do then it
is worth changing to 64k for compatibility.
>
> At the moment, based on the platforms that have adopted LLD so far, I think
most have used 4k as this is the first problem report citing it. If the user
base of LLD grows, with more platforms using a greater than 4k page size then it
makes sense to aim for compatibility and raise the page size.
>
> If you've got access would you be able to raise a PR with a title of
something like Consider increasing LLD Arm default pagesize? We can collect
opinions there.
>
> Peter
>
> ________________________________________
> From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of
Tobias Hieta via llvm-dev <llvm-dev at lists.llvm.org>
> Sent: 01 April 2020 20:33
> To: llvm-dev
> Subject: [llvm-dev] LLD default page size for arm32
>
> Hello,
>
> In the recent days we have been debugging a really thorny issue where
binaries build with clang and linked with lld was just "Killed" when
started on a specific armv7 device we ship on.
>
> After quite a bit of head scratching it turns out that the kernel on this
device ships with a 32k default page size (getconf PAGESIZE) and lld uses 4k
default page size.
>
> We fixed this by passing -zmax-page-size=0x10000 to lld.
>
> The default page size in GNU ld for arm is 64k so binaries linked with ld
just worked on this device.
>
> I put the question in the discord lld channel and after a bit back and
forth I think it's better to discuss it here.
>
> Is 4k the right default for lld on Arm32? Does anyone know if there is a
strong reason to not change this?
>
> Since we have a workaround I am not in a hurry to change this - but if
nothing else I hope this email will help someone else finding the solution to
the problem I had.
>
> Thanks,
> Tobias

On Thu, Apr 2, 2020 at 4:19 AM Peter Smith via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hello,
>
> > Is 4k the right default for lld on Arm32? Does anyone know if there is
a
> strong reason to not change this?
> >
>
> At the time the Arm support for LLD was added, 4k page size was the
> overwhelming default for the major platforms that wanted to use LLD
> (largely those migrating from Gold which also uses 4k page size), including
> several that were quite size conscious and had reduced the page-size to 4k
> from the 64k default on AArch64 due to size concerns.
>
> So a reason, whether it counts as strong or not, is that moving to 64k
> page size would mean a lot of users having to switch back to 4k pages.
>
I believe the issues with binary size stemmed from the lack of support for
common-page-size -- which is now implemented -- so this may not be an issue
anymore.

> > Since we have a workaround I am not in a hurry to change this - but if
> nothing else I hope this email will help someone else finding the solution
> to the problem I had.
>
> Thanks for reporting it. I think it will have to be a community decision.
> If the number of people with > 4k page size systems using LLD is very
small
> and the number using 4k and care about the size is high, then it is may be
> more disruptive to change. However if there is a significant minority that
> do then it is worth changing to 64k for compatibility.
>
> At the moment, based on the platforms that have adopted LLD so far, I
> think most have used 4k as this is the first problem report citing it. If
> the user base of LLD grows, with more platforms using a greater than 4k
> page size then it makes sense to aim for compatibility and raise the page
> size.
>
> If you've got access would you be able to raise a PR with a title of
> something like Consider increasing LLD Arm default pagesize? We can collect
> opinions there.
>
> Peter
>
> ________________________________________
> From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of
Tobias
> Hieta via llvm-dev <llvm-dev at lists.llvm.org>
> Sent: 01 April 2020 20:33
> To: llvm-dev
> Subject: [llvm-dev] LLD default page size for arm32
>
> Hello,
>
> In the recent days we have been debugging a really thorny issue where
> binaries build with clang and linked with lld was just "Killed"
when
> started on a specific armv7 device we ship on.
>
> After quite a bit of head scratching it turns out that the kernel on this
> device ships with a 32k default page size (getconf PAGESIZE) and lld uses
> 4k default page size.
>
> We fixed this by passing -zmax-page-size=0x10000 to lld.
>
> The default page size in GNU ld for arm is 64k so binaries linked with ld
> just worked on this device.
>
> I put the question in the discord lld channel and after a bit back and
> forth I think it's better to discuss it here.
>
> Is 4k the right default for lld on Arm32? Does anyone know if there is a
> strong reason to not change this?
>
> Since we have a workaround I am not in a hurry to change this - but if
> nothing else I hope this email will help someone else finding the solution
> to the problem I had.
>
> Thanks,
> Tobias
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200402/991795dd/attachment.html>