llvm dev - Oct 2014 - [LLVMdev] RFC: Are we ready to completely move away from the optionality of a DataLayout?

I've just wasted a day chasing my tail because of subtleties introduced to
handle the optionality of the DataLayout. I would like to never do this
again. =]

We now have this attached to the Module with just a flimsy faked-up pass to
keep APIs consistent. So, is there any problem with beginning down the path
of:

1) Synthesizing a "default" boring DataLayout for all modules that
don't
specify one.
2) Changing the APIs to make it clear that this can never be missing and is
always available.
3) Start ripping out all of the complexity in the compiler dealing with
this.

If there isn't, I'm willing to do some of the leg work here.
-Chandler
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20141019/2aa1b3be/attachment.html>

On 19 Oct 2014, at 09:22, Chandler Carruth <chandlerc at gmail.com> wrote:
> I've just wasted a day chasing my tail because of subtleties introduced
to handle the optionality of the DataLayout. I would like to never do this
again. =]
> 
> We now have this attached to the Module with just a flimsy faked-up pass to
keep APIs consistent. So, is there any problem with beginning down the path of:
> 
> 1) Synthesizing a "default" boring DataLayout for all modules
that don't specify one.
> 2) Changing the APIs to make it clear that this can never be missing and is
always available.
> 3) Start ripping out all of the complexity in the compiler dealing with
this.
I've also recently had to chase down test case failures caused by assuming
that it was safe to dereference the DataLayout, so a hearty 'yes
please!' from me!

David

I would love to see a mandatory data layout. Thanks for working on this!

Sent from my iPhone
> On Oct 19, 2014, at 04:22, Chandler Carruth <chandlerc at gmail.com>
wrote:
> 
> I've just wasted a day chasing my tail because of subtleties introduced
to handle the optionality of the DataLayout. I would like to never do this
again. =]
> 
> We now have this attached to the Module with just a flimsy faked-up pass to
keep APIs consistent. So, is there any problem with beginning down the path of:
> 
> 1) Synthesizing a "default" boring DataLayout for all modules
that don't specify one.
> 2) Changing the APIs to make it clear that this can never be missing and is
always available.
> 3) Start ripping out all of the complexity in the compiler dealing with
this.
> 
> If there isn't, I'm willing to do some of the leg work here.
> -Chandler

----- Original Message -----
> From: "Chandler Carruth" <chandlerc at gmail.com>
> To: "LLVM Developers Mailing List" <llvmdev at
cs.uiuc.edu>, nicholas at mxc.ca, "Rafael Ávila de Espíndola"
> <rafael.espindola at gmail.com>
> Sent: Sunday, October 19, 2014 3:22:26 AM
> Subject: [LLVMdev] RFC: Are we ready to completely move away from the
optionality of a DataLayout?
> 
> I've just wasted a day chasing my tail because of subtleties
> introduced to handle the optionality of the DataLayout. I would like
> to never do this again. =]
> 
I agree; while I've heard of use cases, I don't personally feel that, at
this stage, it is worth the maintenance burden. Realistically, we just don't
have a good way to test the no-data-layout code paths (the great majority of our
testing coverage comes from frontends that always add DataLayout). I also vote
for making it mandatory.

 -Hal
> 
> We now have this attached to the Module with just a flimsy faked-up
> pass to keep APIs consistent. So, is there any problem with
> beginning down the path of:
> 
> 
> 1) Synthesizing a "default" boring DataLayout for all modules
that
> don't specify one.
> 2) Changing the APIs to make it clear that this can never be missing
> and is always available.
> 3) Start ripping out all of the complexity in the compiler dealing
> with this.
> 
> 
> If there isn't, I'm willing to do some of the leg work here.
> -Chandler
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

Sent from my iPhone
> On Oct 19, 2014, at 1:22 AM, Chandler Carruth <chandlerc at
gmail.com> wrote:
> 
> I've just wasted a day chasing my tail because of subtleties introduced
to handle the optionality of the DataLayout. I would like to never do this
again. =]
Agreed, it's a pain to do this.
> 
> We now have this attached to the Module with just a flimsy faked-up pass to
keep APIs consistent. So, is there any problem with beginning down the path of:
> 
> 1) Synthesizing a "default" boring DataLayout for all modules
that don't specify one.
> 2) Changing the APIs to make it clear that this can never be missing and is
always available.
> 3) Start ripping out all of the complexity in the compiler dealing with
this.
Sounds like a good plan.

One more thing I'd like us to consider after this is where the struct layout
map should live. Currently it's in DataLayout which feels right until you
think that DataLayout lives in the module but is caching based on pointers in
the context.

It makes me feel like DataLayout should live in the context, but then LTO is an
issue with linking modules with different layouts (is that even allowed?). I can
think of a bunch of ways it could fail with struct layouts of the same struct on
different DataLayouts.

Pete
> 
> If there isn't, I'm willing to do some of the leg work here.
> -Chandler
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

----- Original Message -----
> From: "Pete Cooper" <peter_cooper at apple.com>
> To: "Chandler Carruth" <chandlerc at gmail.com>
> Cc: "LLVM Developers Mailing List" <llvmdev at cs.uiuc.edu>
> Sent: Sunday, October 19, 2014 3:34:59 PM
> Subject: Re: [LLVMdev] RFC: Are we ready to completely move away from the
optionality of a DataLayout?
> 
> 
> 
> Sent from my iPhone
> 
> > On Oct 19, 2014, at 1:22 AM, Chandler Carruth <chandlerc at
gmail.com>
> > wrote:
> > 
> > I've just wasted a day chasing my tail because of subtleties
> > introduced to handle the optionality of the DataLayout. I would
> > like to never do this again. =]
> Agreed, it's a pain to do this.
> > 
> > We now have this attached to the Module with just a flimsy faked-up
> > pass to keep APIs consistent. So, is there any problem with
> > beginning down the path of:
> > 
> > 1) Synthesizing a "default" boring DataLayout for all
modules that
> > don't specify one.
> > 2) Changing the APIs to make it clear that this can never be
> > missing and is always available.
> > 3) Start ripping out all of the complexity in the compiler dealing
> > with this.
> Sounds like a good plan.
> 
> One more thing I'd like us to consider after this is where the struct
> layout map should live. Currently it's in DataLayout which feels
> right until you think that DataLayout lives in the module but is
> caching based on pointers in the context.
> 
> It makes me feel like DataLayout should live in the context, but then
> LTO is an issue with linking modules with different layouts (is that
> even allowed?
I think that, generally speaking, this does not make sense. You could imagine
linking together two modules where one data layout was a "subset" of
the other (one is missing details of the vector types, for example, in a module
that used no vector types), but even that seems tenuous.

 -Hal 
>). I can think of a bunch of ways it could fail with
> struct layouts of the same struct on different DataLayouts.
> 
> Pete
> > 
> > If there isn't, I'm willing to do some of the leg work here.
> > -Chandler
> > _______________________________________________
> > LLVM Developers mailing list
> > LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 
-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory

Hi,

I have a question:
> 1) Synthesizing a "default" boring DataLayout for all modules
that don't
> specify one.
What is a default boring value for endianness?

-- Sanjoy

On Sun, Oct 19, 2014 at 2:32 PM, Sanjoy Das <sanjoy at
playingwithpointers.com>
wrote:
> I have a question:
>
> > 1) Synthesizing a "default" boring DataLayout for all
modules that don't
> > specify one.
>
> What is a default boring value for endianness?
>
Little. Sorry, but LE won here.

I mean, we could make the default big-endian just to test the less common
scenario, but I think it would just result in bugs in people's test cases
rather than teasing out actual bugs in their code.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20141019/4d0eadf1/attachment.html>

Just as a heads up, I'm hearing widespread support and no concerns with
this. I'll probably start poking it forward, although it's not likely to
be
a top priority at any time. I'll at least try to update the documentation
where I can find it so that we stop fixing bugs with missing datalayout and
just delete that code path.

On Sun, Oct 19, 2014 at 1:22 AM, Chandler Carruth <chandlerc at gmail.com>
wrote:
> I've just wasted a day chasing my tail because of subtleties introduced
to
> handle the optionality of the DataLayout. I would like to never do this
> again. =]
>
> We now have this attached to the Module with just a flimsy faked-up pass
> to keep APIs consistent. So, is there any problem with beginning down the
> path of:
>
> 1) Synthesizing a "default" boring DataLayout for all modules
that don't
> specify one.
> 2) Changing the APIs to make it clear that this can never be missing and
> is always available.
> 3) Start ripping out all of the complexity in the compiler dealing with
> this.
>
> If there isn't, I'm willing to do some of the leg work here.
> -Chandler
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20141019/690d1460/attachment.html>

> Just as a heads up, I'm hearing widespread support 
After 12 hours on a Sunday?  You people need Real Lives. :-)
--paulr

P.S. This won't make multiarch stuff (arm/thumb, or the stuff 
Eric will be talking about at the dev meeting) more painful, 
right? If it's already module-level that seems unlikely but I
felt like I should ask.

> On 19.10.2014, at 10:22, Chandler Carruth <chandlerc at gmail.com>
wrote:
> 
> I've just wasted a day chasing my tail because of subtleties introduced
to handle the optionality of the DataLayout. I would like to never do this
again. =]
I'm wondering how large the engineering tradeoff actually is. I'm biased
towards making DataLayout mandatory but it does break legitimate use cases.
Target-independent bitcode is not in the best shape but this change would kill
it off entirely, so we better make sure the maintenance is causing enough pain
to rectify the change. I debugged missing 'DL != nullptr' checks a
couple of times, not the most pleasant task in the world but also not a big
hassle.

- Ben
> 
> We now have this attached to the Module with just a flimsy faked-up pass to
keep APIs consistent. So, is there any problem with beginning down the path of:
> 
> 1) Synthesizing a "default" boring DataLayout for all modules
that don't specify one.
> 2) Changing the APIs to make it clear that this can never be missing and is
always available.
> 3) Start ripping out all of the complexity in the compiler dealing with
this.
> 
> If there isn't, I'm willing to do some of the leg work here.
> -Chandler
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

On 20 Oct 2014, at 14:00, Benjamin Kramer <benny.kra at gmail.com> wrote:
> I'm biased towards making DataLayout mandatory but it does break
legitimate use cases. Target-independent bitcode is not in the best shape but
this change would kill it off entirely, so we better make sure the maintenance
is causing enough pain to rectify the change.
Target-independent bitcode exists in the form of things like SPIR and PNaCl. 
These all have a DataLayout.  The IR already implicitly depends on some of these
things (e.g. pointer size), making it explicit doesn't break things.
> I debugged missing 'DL != nullptr' checks a couple of times, not
the most pleasant task in the world but also not a big hassle.
In the case of one of the things that I have in our local branch, the !DL case
does the wrong thing (or, at least, probably does the wrong thing).  It's
easy to make sure that the !DL case does *something*, but it's hard to be
sure that that something is actually correct.

David

On Oct 19, 2014, at 1:22 AM, Chandler Carruth <chandlerc at gmail.com>
wrote:
> I've just wasted a day chasing my tail because of subtleties introduced
to handle the optionality of the DataLayout. I would like to never do this
again. =]
> 
> We now have this attached to the Module with just a flimsy faked-up pass to
keep APIs consistent. So, is there any problem with beginning down the path of:
> 
> 1) Synthesizing a "default" boring DataLayout for all modules
that don't specify one.
> 2) Changing the APIs to make it clear that this can never be missing and is
always available.
> 3) Start ripping out all of the complexity in the compiler dealing with
this.
+1 from me.  The theoretical blue-sky reasons for working with a module that has
no datalayout never happened and almost certainly never will.

-Chris

On Mon, Oct 20, 2014 at 9:51 AM, Chris Lattner <clattner at apple.com>
wrote:
>
> On Oct 19, 2014, at 1:22 AM, Chandler Carruth <chandlerc at
gmail.com> wrote:
>
> > I've just wasted a day chasing my tail because of subtleties
introduced
> to handle the optionality of the DataLayout. I would like to never do this
> again. =]
> >
> > We now have this attached to the Module with just a flimsy faked-up
pass
> to keep APIs consistent. So, is there any problem with beginning down the
> path of:
> >
> > 1) Synthesizing a "default" boring DataLayout for all
modules that don't
> specify one.
> > 2) Changing the APIs to make it clear that this can never be missing
and
> is always available.
> > 3) Start ripping out all of the complexity in the compiler dealing
with
> this.
>
> +1 from me.  The theoretical blue-sky reasons for working with a module
> that has no datalayout never happened and almost certainly never will.
>
>
Agreed. The DataLayout should move (back) to the TargetMachine and live
there (I'm doing that part right now). I don't particularly want to put
it
on the module because of (admittedly pie in the sky) plans of being able to
compile a module with two target machines at the same time.

-eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20141020/4bc3ba45/attachment.html>

Folks,

Is it safe to copy a function pointer contents from an ObjectImage ?
I made something like:

void (*fptr)() = nullptr;
llvm::object::symbol_iterator i = obj.begin_symbols();
if (!i->getType(type) && type ==
llvm::object::SymbolRef::ST_Function)
{
   llvm::StringRef name;
   uint64_t addr, size;
   if (!i->getName(name) && !i->getAddress(addr) &&
!i->getSize(size))
   {
      llvm::sys::MemoryBlock mem = llvm::sys::Memory::AllocateRWX(size, 
nullptr);
      memcpy(mem.base(), (void *)addr, size);
      fptr = reinterpret_cast<void(*)()>(mem.base());
   }
}
...

My first experiments shew that it works but I'd like to know it could 
have any side effects?
My goal is to delete a finalized module and just keep the copyed 
function (to decrease the memory use)

Best regards,

Calixte

-- 
Calixte Denizet
Software Development Engineer
-----------------------------------------------------------
Scilab Enterprises
143bis rue Yves Le Coz - 78000 Versailles, France
http://www.scilab-enterprises.com