llvm dev - May 2016 - A "Cross-Platform Runtime Library API" in LLVM IR

On 23 May 2016, at 10:13, Lorenzo Laneve <lore97drk at icloud.com>
wrote:
> 
> I know but maybe malloc() is a bit higher level than a hypothetical
__alloc(), which may be lower-level and then faster.
How?
> And malloc() is contained in the libc as Matthias said and so maybe a
program using malloc() even for a non-C language linking against the crt is
needed.
On many *NIX platforms, libc or some equivalent is the canonical interface to
the system.
> I'm just assuming that the functions we use in C such as malloc() are
quite high-level interfaces and that there might be something faster.
> 
> For example, does Swift use malloc() to allocate its objects?
A number of high-level language don’t use malloc, but not because it is too
high-level; quite the reverse.  You can have a more efficient malloc
implementation if you know something about allocation patterns.  For example, in
a class-based OO language you’re likely to have a small number of class sizes
and having a separate slab allocator for each of these can be faster.  If you
need garbage collection, then you’ll want to associate more metadata with
allocations.

malloc() is about as low-level an interface to memory management as is possible
while maintaining some abstraction: you give it a size (no type, for example)
and it gives you a pointer to some uninitialised memory.  The only lower-level
interfaces would be related to page table management (mmap and friends on *NIX,
more complex APIs on Mach and entirely delegated page table manipulation on
something like Xen or DUNE).

Again, what problem are you trying to solve?  What is your use case for such a
library?

David

This is the point,
High-level OO languages don't use malloc(), they use something else.
My idea was a C API with implementations of functions:
For example
Assuming I need to implement a function which allocates a new object.
The API provides a really basic allocator function I can use to implement the
complete function the high-level language needs.

void *__newobject(int class_size) {
 void *r = __alloc(class_size);
 // Add GC metadata here
 return r;
}

This function will be included in the runtime library. Obviously this is a
stupid example maybe a better implementation can be done.

Something like this can be done for I/O.
> On May 23, 2016, at 11:28 AM, David Chisnall <David.Chisnall at
cl.cam.ac.uk> wrote:
> 
>> On 23 May 2016, at 10:13, Lorenzo Laneve <lore97drk at
icloud.com> wrote:
>> 
>> I know but maybe malloc() is a bit higher level than a hypothetical
__alloc(), which may be lower-level and then faster.
> 
> How?
> 
>> And malloc() is contained in the libc as Matthias said and so maybe a
program using malloc() even for a non-C language linking against the crt is
needed.
> 
> On many *NIX platforms, libc or some equivalent is the canonical interface
to the system.
> 
>> I'm just assuming that the functions we use in C such as malloc()
are quite high-level interfaces and that there might be something faster.
>> 
>> For example, does Swift use malloc() to allocate its objects?
> 
> A number of high-level language don’t use malloc, but not because it is too
high-level; quite the reverse.  You can have a more efficient malloc
implementation if you know something about allocation patterns.  For example, in
a class-based OO language you’re likely to have a small number of class sizes
and having a separate slab allocator for each of these can be faster.  If you
need garbage collection, then you’ll want to associate more metadata with
allocations.
> 
> malloc() is about as low-level an interface to memory management as is
possible while maintaining some abstraction: you give it a size (no type, for
example) and it gives you a pointer to some uninitialised memory.  The only
lower-level interfaces would be related to page table management (mmap and
friends on *NIX, more complex APIs on Mach and entirely delegated page table
manipulation on something like Xen or DUNE).
> 
> Again, what problem are you trying to solve?  What is your use case for
such a library?
> 
> David
>

On 23 May 2016, at 12:00, Lorenzo Laneve <lore97drk at icloud.com>
wrote:
> 
> This is the point,
No, you are still failing to make a point.
> High-level OO languages don't use malloc(), they use something else.
That’s not entirely true.  They need to get the memory that they provide to
programs from somewhere.  They may provide their own pool allocators, but they
will often just call malloc() and then subdivide the returned chunk.  If they’re
doing garbage collection, they will ask for a large number of pages and use that
as a heap managed by the garbage collector.
> My idea was a C API with implementations of functions:
> For example
> Assuming I need to implement a function which allocates a new object.
> The API provides a really basic allocator function I can use to implement
the complete function the high-level language needs.
> 
> void *__newobject(int class_size) {
> void *r = __alloc(class_size);
> // Add GC metadata here
> return r;
> }
You have still completely failed to explain why __alloc() is better than
malloc().  If you wish to implement a language-specific allocator like this,
then why would it want to call __alloc() from your hypothetical library instead
of malloc(), provided by something like jemalloc(), which has over a decade of
careful optimisation behind it to ensure that it scales well on multithreaded
systems?
> This function will be included in the runtime library. Obviously this is a
stupid example maybe a better implementation can be done.
Again, why would I, as the author of a high-level language, want to depend on
your runtime library?  You have not given a compelling use case for someone
wanting to depend on your hypothetical runtime library instead of the target
platform’s libc.  If I write my language-specific code against libc, then other
people will implement the platform-specific parts for me.  I can rely on C
interfaces existing on pretty much any platform that I want to target, from
desktop and server operating systems down to real-time embedded operating
systems.  What does your proposal give me?

David

Your FIRST premise, that languages don't use `malloc` to allocate memory is
incorrect in the first place.

gnu libc++:
https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/libsupc%2B%2B/new_op.cc

libcxx (LLVM's C++ library):
https://github.com/llvm-mirror/libcxx/blob/master/src/new.cpp#L50

My Pascal compiler does essentially the same thing:
https://github.com/Leporacanthicus/lacsap/blob/master/runtime/alloc.c

--
Mats

On 23 May 2016 at 12:00, Lorenzo Laneve <lore97drk at icloud.com> wrote:
> This is the point,
> High-level OO languages don't use malloc(), they use something else.
> My idea was a C API with implementations of functions:
> For example
> Assuming I need to implement a function which allocates a new object.
> The API provides a really basic allocator function I can use to implement
> the complete function the high-level language needs.
>
> void *__newobject(int class_size) {
>  void *r = __alloc(class_size);
>  // Add GC metadata here
>  return r;
> }
>
> This function will be included in the runtime library. Obviously this is a
> stupid example maybe a better implementation can be done.
>
> Something like this can be done for I/O.
>
> > On May 23, 2016, at 11:28 AM, David Chisnall <
> David.Chisnall at cl.cam.ac.uk> wrote:
> >
> >> On 23 May 2016, at 10:13, Lorenzo Laneve <lore97drk at
icloud.com> wrote:
> >>
> >> I know but maybe malloc() is a bit higher level than a
hypothetical
> __alloc(), which may be lower-level and then faster.
> >
> > How?
> >
> >> And malloc() is contained in the libc as Matthias said and so
maybe a
> program using malloc() even for a non-C language linking against the crt is
> needed.
> >
> > On many *NIX platforms, libc or some equivalent is the canonical
> interface to the system.
> >
> >> I'm just assuming that the functions we use in C such as
malloc() are
> quite high-level interfaces and that there might be something faster.
> >>
> >> For example, does Swift use malloc() to allocate its objects?
> >
> > A number of high-level language don’t use malloc, but not because it
is
> too high-level; quite the reverse.  You can have a more efficient malloc
> implementation if you know something about allocation patterns.  For
> example, in a class-based OO language you’re likely to have a small number
> of class sizes and having a separate slab allocator for each of these can
> be faster.  If you need garbage collection, then you’ll want to associate
> more metadata with allocations.
> >
> > malloc() is about as low-level an interface to memory management as is
> possible while maintaining some abstraction: you give it a size (no type,
> for example) and it gives you a pointer to some uninitialised memory.  The
> only lower-level interfaces would be related to page table management (mmap
> and friends on *NIX, more complex APIs on Mach and entirely delegated page
> table manipulation on something like Xen or DUNE).
> >
> > Again, what problem are you trying to solve?  What is your use case
for
> such a library?
> >
> > David
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160523/02275b39/attachment.html>