[llvm-dev] problems with objects larger than PTRDIFF_MAX

Alexander Cherepanov via llvm-dev
Sun, 29 May 2016 21:09:30 -0000

On 2016-05-29 21:16, David Majnemer wrote:
> On Sun, May 29, 2016 at 6:18 AM, Alexander Cherepanov <ch3root at
openwall.com>
> wrote:
>
>> On 2016-05-20 19:58, David Majnemer via llvm-dev wrote:
>>
>>> I've come across this issue before and came to the following
conclusion:
>>> - We are not obligated to support objects that large, C11 5.2.4.1/1
only
>>> requires that we support objects of size 65535!
>>>
>>
>> Right, the standard doesn't require it. But I guess you don't
imply that
>> it's fine for clang to silently miscompile any program that works
with
>> objects larger than 65535 bytes.
>
> I think that we should error when we can statically determine that an
> object is larger than PTRDIFF_MAX.

It seems we agree that the current situation is not good and something
has to be fixed in clang/llvm. Ok.

Why don't you think that clang/llvm should just be fixed to support
objects larger than PTRDIFF_MAX bytes?

>> - The expectation that PTRDIFF_MAX is more or less a proxy for the
largest
>>> object size is not uncommon.   For example, C++'s std::count
doesn't
>>> return
>>> a size_t but a iterator_traits<>::difference_type which is
going to be a
>>> ptrdiff_t for things like std::vector.
>>
>> Bug in C++? Because max_size() of std::vector<char> has the type
>> `size_type` and returns SIZE_MAX in practice (even on x86-64).
>
> Some folks in the C++ community believe that using unsigned types in those
> places was the wrong decision.

Result of max_size() cannot be changed without changing its type?

>> Let's see other examples (on x86-32):
>>
>> - glibc's malloc is happy to allocate more than PTRDIFF_MAX bytes
(and
>> this is important in practice so (at least some) glibc devs are
reluctant
>> to "fix" it);
>>
>
> glibc is not the only game in town.

Right, for example musl doesn't create objects larger than PTRDIFF_MAX
bytes. I think this is good that different libc implementations are free
to choose their own policies.

The problem is that this is only possible when compilers support large
objects. When compilers don't support large objects (like now), support
in libc is not enough -- you get your large objects but your accesses to
it will be miscompiled. Sometimes.

> Android prevents it: https://android-review.googlesource.com/#/c/170800/
> jemalloc prevents it:
>
https://github.com/jemalloc/jemalloc/commit/0c516a00c4cb28cff55ce0995f756b5aae074c9e

A good share of the reasons for this is exactly the fact that clang/llvm
don't support large objects. So this reasoning is circular:-)

> There are other implementations of malloc which also act similarly.

--
Alexander Cherepanov