llvm dev - May 2016 - problems with objects larger than PTRDIFF_MAX

It could be that 32-bit systems are disappearing so rapidly that nobody 
cares too much about this issue, but this blog post is still worth reading:

   http://trust-in-soft.com/objects-larger-than-ptrdiff_max-bytes/

John

I've come across this issue before and came to the following conclusion:
- We are not obligated to support objects that large, C11 5.2.4.1/1 only
requires that we support objects of size 65535!  Their guidance for maximum
object size is stated to be half of SIZE_MAX in C11 K.3.4/4 which is
typically equivalent to PTRDIFF_MAX.
- The expectation that PTRDIFF_MAX is more or less a proxy for the largest
object size is not uncommon.  For example, C++'s std::count doesn't
return
a size_t but a iterator_traits<>::difference_type which is going to be a
ptrdiff_t for things like std::vector.

On Fri, May 20, 2016 at 1:31 AM, John Regehr via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> It could be that 32-bit systems are disappearing so rapidly that nobody
> cares too much about this issue, but this blog post is still worth reading:
>
>   http://trust-in-soft.com/objects-larger-than-ptrdiff_max-bytes/
>
> John
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160520/dc228a9e/attachment.html>

I agree, LLVM is not obligated to support objects larger than 
PTRDIFF_MAX bytes.

A reasonable and friendly goal would be to document this constraint and 
perhaps also arrange for some compile-time warnings and/or runtime 
errors to be emitted when it is violated.

John


On 5/20/16 6:58 PM, David Majnemer via llvm-dev wrote:
> I've come across this issue before and came to the following
conclusion:
> - We are not obligated to support objects that large, C11 5.2.4.1/1
> <http://5.2.4.1/1> only requires that we support objects of size
65535!
> Their guidance for maximum object size is stated to be half of SIZE_MAX
> in C11 K.3.4/4 which is typically equivalent to PTRDIFF_MAX.
> - The expectation that PTRDIFF_MAX is more or less a proxy for the
> largest object size is not uncommon.  For example, C++'s std::count
> doesn't return a size_t but a iterator_traits<>::difference_type
which
> is going to be a ptrdiff_t for things like std::vector.
>
> On Fri, May 20, 2016 at 1:31 AM, John Regehr via llvm-dev
> <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>
>     It could be that 32-bit systems are disappearing so rapidly that
>     nobody cares too much about this issue, but this blog post is still
>     worth reading:
>
>       http://trust-in-soft.com/objects-larger-than-ptrdiff_max-bytes/
>
>     John
>     _______________________________________________
>     LLVM Developers mailing list
>     llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>

On 2016-05-20 19:58, David Majnemer via llvm-dev wrote:
> I've come across this issue before and came to the following
conclusion:
> - We are not obligated to support objects that large, C11 5.2.4.1/1 only
> requires that we support objects of size 65535!
Right, the standard doesn't require it. But I guess you don't imply that
it's fine for clang to silently miscompile any program that works with 
objects larger than 65535 bytes.
> Their guidance for maximum
> object size is stated to be half of SIZE_MAX in C11 K.3.4/4 which is
> typically equivalent to PTRDIFF_MAX.
Whose guidance? Annex K is kinda alien to the rest of the standard and 
its future is not clear. See, e.g., 
http://open-std.org/jtc1/sc22/wg14/www/docs/n1969.htm .
> - The expectation that PTRDIFF_MAX is more or less a proxy for the largest
> object size is not uncommon.   For example, C++'s std::count
doesn't return
> a size_t but a iterator_traits<>::difference_type which is going to
be a
> ptrdiff_t for things like std::vector.
Bug in C++? Because max_size() of std::vector<char> has the type 
`size_type` and returns SIZE_MAX in practice (even on x86-64).

Let's see other examples (on x86-32):

- glibc's malloc is happy to allocate more than PTRDIFF_MAX bytes (and 
this is important in practice so (at least some) glibc devs are 
reluctant to "fix" it);

- clang has a compile-time limit for sizes of objects and types which is 
SIZE_MAX. For example, it compiles "char a[-1ul]; int main() {}" but 
complains about "char a[-1ul + 1ull]; int main() {}";

- `new` for more than PTRDIFF_MAX works fine when compiled by clang++ 
(but throws std::bad_array_new_length when compiled by g++).
> On Fri, May 20, 2016 at 1:31 AM, John Regehr via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> It could be that 32-bit systems are disappearing so rapidly that nobody
>> cares too much about this issue,
Please note that there is a whole new 32-bit architecture -- x32.
>> but this blog post is still worth reading:
>>
>>    http://trust-in-soft.com/objects-larger-than-ptrdiff_max-bytes/
-- 
Alexander Cherepanov