llvm dev - Apr 2017 - Relocation design of different architecture

Thanks for the reply. I was just asking about in general whatever header
files are there in Targets/ for different architectures are not including
any function except this processRelocationRef() to be used in
RuntimeDyldELF.cpp or RuntimeDyldCOFF.cpp or RuntimeDyldMachO.cpp and i
think these files are the ones which are actually doing the relocation and
linking work. So what purpose do these header files inside Targets/
actually serve. Also they include exception handling in form of exception
frames, So can u guide on this issue ?

Thanks,
Siddharth

On Thu, Apr 20, 2017 at 4:02 PM, mats petersson <mats at
planetcatfish.com>
wrote:
> The x86_64 and i386 architectures have different actual relocation
> records. So if you build code for i386, you need one processRelocationRef()
> function (handling the relevant relocations in that model), and when
> producing code for x86_64, there are different relocation records. The two
> files contain the derived form of the class that processes the relocation
> records when dynamically loading JITed code in LLVM - mainly implementing
> the two different forms of symbol entries that refer to the relocations -
> i386 uses COFF::IMAGE_REL_I386_*, in x86_64 the relocation types are
> COFF::IMAGE_REL_AMD64_*.
>
> Conceptually, they do the same thing, it's the details of exactly how
and
> where the relocation ends up and how it's recorded by the linker that
> differs.
>
> Theoretically, one could probably construct a loadable file that
doesn't
> care what architecture it is for, but it would end up with a lot of
> redundant & overlapping functionality, and the code to handle every
> different architecture in one huge switch-statement would be rather complex
> (and long!). So splitting the functionality per architecture helps make the
> code clear.
>
> If you need further help to understand the code, you'll probably need
to
> ask a more concrete question, as it is probably not possible to describe
> all the relevant information on this subject in less than 200 pages, never
> mind a simple email-thread.
>
> --
> Mats
>
> On 20 April 2017 at 11:13, Siddharth Shankar Swain via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi,
>> Can anyone explain in lib/ExecutionEngine/RuntimeDyld/Targets/ the
>> header files included for different architectures like
>> RuntimeDyldCOFFX86_64.h or RuntimeDyldCOFFI386.h etc, what is the
>> connection of these files for relocation and linking as the linking and
>> relocation for diff architecture is done in RuntimeDyldELF.cpp,
>> RuntimeDyldCOFF.cpp  and it doesn't use any function from these
header file
>> except the processRelocationRef(). The header files in Targets/ also
>> handles exceptions, so what is the need for that in relocation and
linking
>> process ? Also please help with what this processRelocationRef()
actually
>> does ? . Please guide.
>>
>> sincerely,
>> Siddharth
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170420/1d4310d6/attachment.html>

Basic Object Oriented design uses a derived class to implement a
functionality of the generic case. It's the same basic principle as:

class Animal
{
    void virtual Say() = 0;
};

class Cat: public Animal
{
    void Say() override { cout << "Meow!" << endl; }
}

class Dog: public Animal
{
    void Say() override { cout << "Woof!" << endl; }
}

class Fish: public Animal
{
    void Say() override { cout << "Blub!" << endl; }
}

In this case, different types of COFF-architectures use different
relocation entries, and based on the architecture, a specific
implementation of the RelocationDyldCOFF class is created to perform the
relocation.

See http://llvm.org/docs/doxygen/html/classllvm_1_1RuntimeDyldCOFF.html for
a class diagram of how this is done.

The generic code in RuntimeDyld*.cpp only knows that relocations exists,
and that they need to be dealt with. Not HOW to actually perform the
relocation - just like "Animal" doesn't know what a cat or a dog
"says".
The processRelocationRef() is called here:
http://llvm.org/docs/doxygen/html/RuntimeDyld_8cpp_source.html#l00251

Again, it's not clear exactly what you are asking for, so I'm not sure
whether my explanation is helpful or not...

--
Mats


On 20 April 2017 at 12:05, Siddharth Shankar Swain <
h2015096 at pilani.bits-pilani.ac.in> wrote:
> Thanks for the reply. I was just asking about in general whatever header
> files are there in Targets/ for different architectures are not including
> any function except this processRelocationRef() to be used in
> RuntimeDyldELF.cpp or RuntimeDyldCOFF.cpp or RuntimeDyldMachO.cpp and i
> think these files are the ones which are actually doing the relocation and
> linking work. So what purpose do these header files inside Targets/
> actually serve. Also they include exception handling in form of exception
> frames, So can u guide on this issue ?
>
> Thanks,
> Siddharth
>
> On Thu, Apr 20, 2017 at 4:02 PM, mats petersson <mats at
planetcatfish.com>
> wrote:
>
>> The x86_64 and i386 architectures have different actual relocation
>> records. So if you build code for i386, you need one
processRelocationRef()
>> function (handling the relevant relocations in that model), and when
>> producing code for x86_64, there are different relocation records. The
two
>> files contain the derived form of the class that processes the
relocation
>> records when dynamically loading JITed code in LLVM - mainly
implementing
>> the two different forms of symbol entries that refer to the relocations
-
>> i386 uses COFF::IMAGE_REL_I386_*, in x86_64 the relocation types are
>> COFF::IMAGE_REL_AMD64_*.
>>
>> Conceptually, they do the same thing, it's the details of exactly
how and
>> where the relocation ends up and how it's recorded by the linker
that
>> differs.
>>
>> Theoretically, one could probably construct a loadable file that
doesn't
>> care what architecture it is for, but it would end up with a lot of
>> redundant & overlapping functionality, and the code to handle every
>> different architecture in one huge switch-statement would be rather
complex
>> (and long!). So splitting the functionality per architecture helps make
the
>> code clear.
>>
>> If you need further help to understand the code, you'll probably
need to
>> ask a more concrete question, as it is probably not possible to
describe
>> all the relevant information on this subject in less than 200 pages,
never
>> mind a simple email-thread.
>>
>> --
>> Mats
>>
>> On 20 April 2017 at 11:13, Siddharth Shankar Swain via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> Hi,
>>> Can anyone explain in lib/ExecutionEngine/RuntimeDyld/Targets/ the
>>> header files included for different architectures like
>>> RuntimeDyldCOFFX86_64.h or RuntimeDyldCOFFI386.h etc, what is the
>>> connection of these files for relocation and linking as the linking
and
>>> relocation for diff architecture is done in RuntimeDyldELF.cpp,
>>> RuntimeDyldCOFF.cpp  and it doesn't use any function from these
header file
>>> except the processRelocationRef(). The header files in Targets/
also
>>> handles exceptions, so what is the need for that in relocation and
linking
>>> process ? Also please help with what this processRelocationRef()
actually
>>> does ? . Please guide.
>>>
>>> sincerely,
>>> Siddharth
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170420/777c03be/attachment.html>

Note: Please include the mailing list when replying to discussions, as
someone else may well want to see the discussion, and may be better placed
to answer.

Like I've tried to explain, there is a generic piece of code that
understands how to load code in general (the class RuntimeDyld and related
bits), and then specific implementations that derive from a base class to
do the specific relocation and exception handling for that particular
hardware and file-format - for each supported processor architecture and
file-format, there needs to be a specific class that implements some
functions (processRelocationRef is one of those). Technically, it looks
like it's using a "pImpl" pattern, but the basic principle is the
same
either way - generic code handles the generic case, a derived class that
understands how to deal with the specifics is used to actually perform
relocations in that particular case.

Exception handling is also target-specific, so in x86-64 and i386, how
exception information is stored and used is different (I don't know the
exact details in this case as COFF is the file-format used on Windows, and
it's been at least 8 or 10 years since I did any programming at all on a
Windows machine, I know that i386 on Linux uses an exception table, and
x86-64 on linux essentially has debug information [DWARF tables]). The
exception information is used to determine how to unwind the stack and
destroy objects on the way back to the "catch" for that particular
exception. There is code required both to load the exception tables into
memory, and to interpret/use those tables - but I'm not overly familiar
with how that works for JIT'd code. [Actually, looking at the code for
x86-64, it looks like it's mainly SEH (Structured Exception Handling) that
is dealt with - the overall concept still applies, but SEH is a Windows
concept for handling exceptions, which includes hardware exceptions such as
integer division by zero and memory access exceptions - regular C++
exceptions are dealt with separately, and that is what uses what I
described for Linux earlier in this paragraph].

As to WHY different architectures use different relocations and exception
handling tables, that's an ABI design issue - a convention that is based on
the needs and requirements for each architecture, and a bunch of
compromises between simplicity (a very simple table is easy to construct),
space (simple table takes up more space than a more complex table
construction - like a zip file or a text file - the zip file is more
complicated to read, but takes up a lot less space) and code complexity
(save space in table, more complex code most likely). Either way, for a
given platform (OS, Processor, file format), there is a given ABI for
handling exceptions. The loader needs to load the table in the correct way
into the correct part of memory, and when an exception is thrown, the
table(s) need to be understood and acted upon to find the way back to the
relevant place where the exception is caught.

The fact that the classes are declared in different files is similar to my
simple animal example, where you'd have a animal.h for the base class, a
cat.h, dog.h and fish.h for the actual implementations. Obviously, the
specific implementations for the RuntimeDyld belongs in "Target"
because
they are dependent on the actual target (which is the combination of
fileformat, OS and processor architecture).

--
Mats

On 20 April 2017 at 14:04, Siddharth Shankar Swain <
h2015096 at pilani.bits-pilani.ac.in> wrote:
> So RuntimeDyldELF.cpp or RuntimeDyldCOFF.cpp or RuntimeDyldMachO.cpp  are
> doing relocation and linking for specific object file format and all
> architectures using that object file format. Am i correct? If that is so
> then these  .cpp files are not using any header file in Targets/ so what
> are these header files in Targets/ made for ? Another thing is that why
> these header files in Targets/ handling exception and that too using a
> different concept of exception frames and exception tables. Please guide
> Thanks,
> Siddharth
>
> On Thu, Apr 20, 2017 at 6:06 PM, mats petersson <mats at
planetcatfish.com>
> wrote:
>
>> Basic Object Oriented design uses a derived class to implement a
>> functionality of the generic case. It's the same basic principle
as:
>>
>> class Animal
>> {
>>     void virtual Say() = 0;
>> };
>>
>> class Cat: public Animal
>> {
>>     void Say() override { cout << "Meow!" <<
endl; }
>> }
>>
>> class Dog: public Animal
>> {
>>     void Say() override { cout << "Woof!" <<
endl; }
>> }
>>
>> class Fish: public Animal
>> {
>>     void Say() override { cout << "Blub!" <<
endl; }
>> }
>>
>> In this case, different types of COFF-architectures use different
>> relocation entries, and based on the architecture, a specific
>> implementation of the RelocationDyldCOFF class is created to perform
the
>> relocation.
>>
>> See http://llvm.org/docs/doxygen/html/classllvm_1_1RuntimeDyldCOFF.html
>> for a class diagram of how this is done.
>>
>> The generic code in RuntimeDyld*.cpp only knows that relocations
exists,
>> and that they need to be dealt with. Not HOW to actually perform the
>> relocation - just like "Animal" doesn't know what a cat
or a dog "says".
>> The processRelocationRef() is called here:
>> http://llvm.org/docs/doxygen/html/RuntimeDyld_8cpp_source.html#l00251
>>
>> Again, it's not clear exactly what you are asking for, so I'm
not sure
>> whether my explanation is helpful or not...
>>
>> --
>> Mats
>>
>>
>> On 20 April 2017 at 12:05, Siddharth Shankar Swain <
>> h2015096 at pilani.bits-pilani.ac.in> wrote:
>>
>>> Thanks for the reply. I was just asking about in general whatever
header
>>> files are there in Targets/ for different architectures are not
including
>>> any function except this processRelocationRef() to be used in
>>> RuntimeDyldELF.cpp or RuntimeDyldCOFF.cpp or RuntimeDyldMachO.cpp
and i
>>> think these files are the ones which are actually doing the
relocation and
>>> linking work. So what purpose do these header files inside Targets/
>>> actually serve. Also they include exception handling in form of
exception
>>> frames, So can u guide on this issue ?
>>>
>>> Thanks,
>>> Siddharth
>>>
>>> On Thu, Apr 20, 2017 at 4:02 PM, mats petersson <mats at
planetcatfish.com>
>>> wrote:
>>>
>>>> The x86_64 and i386 architectures have different actual
relocation
>>>> records. So if you build code for i386, you need one
processRelocationRef()
>>>> function (handling the relevant relocations in that model), and
when
>>>> producing code for x86_64, there are different relocation
records. The two
>>>> files contain the derived form of the class that processes the
relocation
>>>> records when dynamically loading JITed code in LLVM - mainly
implementing
>>>> the two different forms of symbol entries that refer to the
relocations -
>>>> i386 uses COFF::IMAGE_REL_I386_*, in x86_64 the relocation
types are
>>>> COFF::IMAGE_REL_AMD64_*.
>>>>
>>>> Conceptually, they do the same thing, it's the details of
exactly how
>>>> and where the relocation ends up and how it's recorded by
the linker that
>>>> differs.
>>>>
>>>> Theoretically, one could probably construct a loadable file
that
>>>> doesn't care what architecture it is for, but it would end
up with a lot of
>>>> redundant & overlapping functionality, and the code to
handle every
>>>> different architecture in one huge switch-statement would be
rather complex
>>>> (and long!). So splitting the functionality per architecture
helps make the
>>>> code clear.
>>>>
>>>> If you need further help to understand the code, you'll
probably need
>>>> to ask a more concrete question, as it is probably not possible
to describe
>>>> all the relevant information on this subject in less than 200
pages, never
>>>> mind a simple email-thread.
>>>>
>>>> --
>>>> Mats
>>>>
>>>> On 20 April 2017 at 11:13, Siddharth Shankar Swain via llvm-dev
<
>>>> llvm-dev at lists.llvm.org> wrote:
>>>>
>>>>> Hi,
>>>>> Can anyone explain in
lib/ExecutionEngine/RuntimeDyld/Targets/ the
>>>>> header files included for different architectures like
>>>>> RuntimeDyldCOFFX86_64.h or RuntimeDyldCOFFI386.h etc, what
is the
>>>>> connection of these files for relocation and linking as the
linking and
>>>>> relocation for diff architecture is done in
RuntimeDyldELF.cpp,
>>>>> RuntimeDyldCOFF.cpp  and it doesn't use any function
from these header file
>>>>> except the processRelocationRef(). The header files in
Targets/ also
>>>>> handles exceptions, so what is the need for that in
relocation and linking
>>>>> process ? Also please help with what this
processRelocationRef() actually
>>>>> does ? . Please guide.
>>>>>
>>>>> sincerely,
>>>>> Siddharth
>>>>>
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170420/991c963d/attachment.html>

(Again: Please always REPLY to all recipients, including llvm-dev, unless
there is VERY specific reasons not to)

The ELF support for relocation is all baked into a single, large function
for all different processor architectures. In my humble opinion, it would
make the code simpler and more readable to implement this code as multiple
derived classes based on architecture (there are several "if(Arch ==
...)"
or similar, then a large section of code for that architecture). But I've
not worked on this code personally, and this is just from a basic "look at
the code for a few minutes to understand it". It's probably one of
those
things that has evolved over time - originally only one or two processor
architectures where supported, then someone added one or two more, and
eventually you have a function that is ~600 lines of code and a file that
is over 1800 lines, compared to the COFF_x86_64 class that is just over 200
lines for the entire file. There are positive and negative things about
having large or small functions, but my personal choice would be a split -
that's not to say that such a split ends up "near the top" of the
priority
list of "things to do to make LLVM better" - presumably the code works
as
it is, so changing it MAY break things.

--
Mats

On 20 April 2017 at 15:14, Siddharth Shankar Swain <
h2015096 at pilani.bits-pilani.ac.in> wrote:
> Thanks for the reply. It was really helpful. So to be more specific there
> is a processRelocationRef() and resolveRelocation() in
> Targets/RuntimeDyld(objectfile format)(arch).h and also in
> RuntimeDyldELF.cpp . Whats the different between these to and for what diff
> purpose they are used ?
>
> Thanks,
> Siddharth
>
> On Thu, Apr 20, 2017 at 7:10 PM, mats petersson <mats at
planetcatfish.com>
> wrote:
>
>> Note: Please include the mailing list when replying to discussions, as
>> someone else may well want to see the discussion, and may be better
placed
>> to answer.
>>
>> Like I've tried to explain, there is a generic piece of code that
>> understands how to load code in general (the class RuntimeDyld and
related
>> bits), and then specific implementations that derive from a base class
to
>> do the specific relocation and exception handling for that particular
>> hardware and file-format - for each supported processor architecture
and
>> file-format, there needs to be a specific class that implements some
>> functions (processRelocationRef is one of those). Technically, it looks
>> like it's using a "pImpl" pattern, but the basic
principle is the same
>> either way - generic code handles the generic case, a derived class
that
>> understands how to deal with the specifics is used to actually perform
>> relocations in that particular case.
>>
>> Exception handling is also target-specific, so in x86-64 and i386, how
>> exception information is stored and used is different (I don't know
the
>> exact details in this case as COFF is the file-format used on Windows,
and
>> it's been at least 8 or 10 years since I did any programming at all
on a
>> Windows machine, I know that i386 on Linux uses an exception table, and
>> x86-64 on linux essentially has debug information [DWARF tables]). The
>> exception information is used to determine how to unwind the stack and
>> destroy objects on the way back to the "catch" for that
particular
>> exception. There is code required both to load the exception tables
into
>> memory, and to interpret/use those tables - but I'm not overly
familiar
>> with how that works for JIT'd code. [Actually, looking at the code
for
>> x86-64, it looks like it's mainly SEH (Structured Exception
Handling) that
>> is dealt with - the overall concept still applies, but SEH is a Windows
>> concept for handling exceptions, which includes hardware exceptions
such as
>> integer division by zero and memory access exceptions - regular C++
>> exceptions are dealt with separately, and that is what uses what I
>> described for Linux earlier in this paragraph].
>>
>> As to WHY different architectures use different relocations and
exception
>> handling tables, that's an ABI design issue - a convention that is
based on
>> the needs and requirements for each architecture, and a bunch of
>> compromises between simplicity (a very simple table is easy to
construct),
>> space (simple table takes up more space than a more complex table
>> construction - like a zip file or a text file - the zip file is more
>> complicated to read, but takes up a lot less space) and code complexity
>> (save space in table, more complex code most likely). Either way, for a
>> given platform (OS, Processor, file format), there is a given ABI for
>> handling exceptions. The loader needs to load the table in the correct
way
>> into the correct part of memory, and when an exception is thrown, the
>> table(s) need to be understood and acted upon to find the way back to
the
>> relevant place where the exception is caught.
>>
>> The fact that the classes are declared in different files is similar to
>> my simple animal example, where you'd have a animal.h for the base
class, a
>> cat.h, dog.h and fish.h for the actual implementations. Obviously, the
>> specific implementations for the RuntimeDyld belongs in
"Target" because
>> they are dependent on the actual target (which is the combination of
>> fileformat, OS and processor architecture).
>>
>> --
>> Mats
>>
>>
>> On 20 April 2017 at 14:04, Siddharth Shankar Swain <
>> h2015096 at pilani.bits-pilani.ac.in> wrote:
>>
>>> So RuntimeDyldELF.cpp or RuntimeDyldCOFF.cpp or
RuntimeDyldMachO.cpp
>>>  are doing relocation and linking for specific object file format
and all
>>> architectures using that object file format. Am i correct? If that
is so
>>> then these  .cpp files are not using any header file in Targets/ so
what
>>> are these header files in Targets/ made for ? Another thing is that
why
>>> these header files in Targets/ handling exception and that too
using a
>>> different concept of exception frames and exception tables. Please
guide
>>> Thanks,
>>> Siddharth
>>>
>>> On Thu, Apr 20, 2017 at 6:06 PM, mats petersson <mats at
planetcatfish.com>
>>> wrote:
>>>
>>>> Basic Object Oriented design uses a derived class to implement
a
>>>> functionality of the generic case. It's the same basic
principle as:
>>>>
>>>> class Animal
>>>> {
>>>>     void virtual Say() = 0;
>>>> };
>>>>
>>>> class Cat: public Animal
>>>> {
>>>>     void Say() override { cout << "Meow!"
<< endl; }
>>>> }
>>>>
>>>> class Dog: public Animal
>>>> {
>>>>     void Say() override { cout << "Woof!"
<< endl; }
>>>> }
>>>>
>>>> class Fish: public Animal
>>>> {
>>>>     void Say() override { cout << "Blub!"
<< endl; }
>>>> }
>>>>
>>>> In this case, different types of COFF-architectures use
different
>>>> relocation entries, and based on the architecture, a specific
>>>> implementation of the RelocationDyldCOFF class is created to
perform the
>>>> relocation.
>>>>
>>>> See
http://llvm.org/docs/doxygen/html/classllvm_1_1RuntimeDyldCOFF.html
>>>> for a class diagram of how this is done.
>>>>
>>>> The generic code in RuntimeDyld*.cpp only knows that
relocations
>>>> exists, and that they need to be dealt with. Not HOW to
actually perform
>>>> the relocation - just like "Animal" doesn't know
what a cat or a dog
>>>> "says". The processRelocationRef() is called here:
>>>>
http://llvm.org/docs/doxygen/html/RuntimeDyld_8cpp_source.html#l00251
>>>>
>>>> Again, it's not clear exactly what you are asking for, so
I'm not sure
>>>> whether my explanation is helpful or not...
>>>>
>>>> --
>>>> Mats
>>>>
>>>>
>>>> On 20 April 2017 at 12:05, Siddharth Shankar Swain <
>>>> h2015096 at pilani.bits-pilani.ac.in> wrote:
>>>>
>>>>> Thanks for the reply. I was just asking about in general
whatever
>>>>> header files are there in Targets/ for different
architectures are not
>>>>> including any function except this processRelocationRef()
to be used in
>>>>> RuntimeDyldELF.cpp or RuntimeDyldCOFF.cpp or
RuntimeDyldMachO.cpp and i
>>>>> think these files are the ones which are actually doing the
relocation and
>>>>> linking work. So what purpose do these header files inside
Targets/
>>>>> actually serve. Also they include exception handling in
form of exception
>>>>> frames, So can u guide on this issue ?
>>>>>
>>>>> Thanks,
>>>>> Siddharth
>>>>>
>>>>> On Thu, Apr 20, 2017 at 4:02 PM, mats petersson <
>>>>> mats at planetcatfish.com> wrote:
>>>>>
>>>>>> The x86_64 and i386 architectures have different actual
relocation
>>>>>> records. So if you build code for i386, you need one
processRelocationRef()
>>>>>> function (handling the relevant relocations in that
model), and when
>>>>>> producing code for x86_64, there are different
relocation records. The two
>>>>>> files contain the derived form of the class that
processes the relocation
>>>>>> records when dynamically loading JITed code in LLVM -
mainly implementing
>>>>>> the two different forms of symbol entries that refer to
the relocations -
>>>>>> i386 uses COFF::IMAGE_REL_I386_*, in x86_64 the
relocation types are
>>>>>> COFF::IMAGE_REL_AMD64_*.
>>>>>>
>>>>>> Conceptually, they do the same thing, it's the
details of exactly how
>>>>>> and where the relocation ends up and how it's
recorded by the linker that
>>>>>> differs.
>>>>>>
>>>>>> Theoretically, one could probably construct a loadable
file that
>>>>>> doesn't care what architecture it is for, but it
would end up with a lot of
>>>>>> redundant & overlapping functionality, and the code
to handle every
>>>>>> different architecture in one huge switch-statement
would be rather complex
>>>>>> (and long!). So splitting the functionality per
architecture helps make the
>>>>>> code clear.
>>>>>>
>>>>>> If you need further help to understand the code,
you'll probably need
>>>>>> to ask a more concrete question, as it is probably not
possible to describe
>>>>>> all the relevant information on this subject in less
than 200 pages, never
>>>>>> mind a simple email-thread.
>>>>>>
>>>>>> --
>>>>>> Mats
>>>>>>
>>>>>> On 20 April 2017 at 11:13, Siddharth Shankar Swain via
llvm-dev <
>>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>> Can anyone explain in
lib/ExecutionEngine/RuntimeDyld/Targets/ the
>>>>>>> header files included for different architectures
like
>>>>>>> RuntimeDyldCOFFX86_64.h or RuntimeDyldCOFFI386.h
etc, what is the
>>>>>>> connection of these files for relocation and
linking as the linking and
>>>>>>> relocation for diff architecture is done in
RuntimeDyldELF.cpp,
>>>>>>> RuntimeDyldCOFF.cpp  and it doesn't use any
function from these header file
>>>>>>> except the processRelocationRef(). The header files
in Targets/ also
>>>>>>> handles exceptions, so what is the need for that in
relocation and linking
>>>>>>> process ? Also please help with what this
processRelocationRef() actually
>>>>>>> does ? . Please guide.
>>>>>>>
>>>>>>> sincerely,
>>>>>>> Siddharth
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> LLVM Developers mailing list
>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170420/41f5efe5/attachment-0001.html>

Thanks for reply, it was really helpful. Can u just be more specific and
tell about processRelocationRef() and resolveRelocation() in
Targets/RuntimeDyld(objectfile format)(arch).h and also in
RuntimeDyldELF.cpp and how the same function is implemented in different
ways in both the files ?
Thanks,
Siddharth

On Thu, Apr 20, 2017 at 8:16 PM, mats petersson <mats at
planetcatfish.com>
wrote:
> (Again: Please always REPLY to all recipients, including llvm-dev, unless
> there is VERY specific reasons not to)
>
> The ELF support for relocation is all baked into a single, large function
> for all different processor architectures. In my humble opinion, it would
> make the code simpler and more readable to implement this code as multiple
> derived classes based on architecture (there are several "if(Arch ==
...)"
> or similar, then a large section of code for that architecture). But
I've
> not worked on this code personally, and this is just from a basic
"look at
> the code for a few minutes to understand it". It's probably one of
those
> things that has evolved over time - originally only one or two processor
> architectures where supported, then someone added one or two more, and
> eventually you have a function that is ~600 lines of code and a file that
> is over 1800 lines, compared to the COFF_x86_64 class that is just over 200
> lines for the entire file. There are positive and negative things about
> having large or small functions, but my personal choice would be a split -
> that's not to say that such a split ends up "near the top" of
the priority
> list of "things to do to make LLVM better" - presumably the code
works as
> it is, so changing it MAY break things.
>
> --
> Mats
>
> On 20 April 2017 at 15:14, Siddharth Shankar Swain <
> h2015096 at pilani.bits-pilani.ac.in> wrote:
>
>> Thanks for the reply. It was really helpful. So to be more specific
there
>> is a processRelocationRef() and resolveRelocation() in
>> Targets/RuntimeDyld(objectfile format)(arch).h and also in
>> RuntimeDyldELF.cpp . Whats the different between these to and for what
diff
>> purpose they are used ?
>>
>> Thanks,
>> Siddharth
>>
>> On Thu, Apr 20, 2017 at 7:10 PM, mats petersson <mats at
planetcatfish.com>
>> wrote:
>>
>>> Note: Please include the mailing list when replying to discussions,
as
>>> someone else may well want to see the discussion, and may be better
placed
>>> to answer.
>>>
>>> Like I've tried to explain, there is a generic piece of code
that
>>> understands how to load code in general (the class RuntimeDyld and
related
>>> bits), and then specific implementations that derive from a base
class to
>>> do the specific relocation and exception handling for that
particular
>>> hardware and file-format - for each supported processor
architecture and
>>> file-format, there needs to be a specific class that implements
some
>>> functions (processRelocationRef is one of those). Technically, it
looks
>>> like it's using a "pImpl" pattern, but the basic
principle is the same
>>> either way - generic code handles the generic case, a derived class
that
>>> understands how to deal with the specifics is used to actually
perform
>>> relocations in that particular case.
>>>
>>> Exception handling is also target-specific, so in x86-64 and i386,
how
>>> exception information is stored and used is different (I don't
know the
>>> exact details in this case as COFF is the file-format used on
Windows, and
>>> it's been at least 8 or 10 years since I did any programming at
all on a
>>> Windows machine, I know that i386 on Linux uses an exception table,
and
>>> x86-64 on linux essentially has debug information [DWARF tables]).
The
>>> exception information is used to determine how to unwind the stack
and
>>> destroy objects on the way back to the "catch" for that
particular
>>> exception. There is code required both to load the exception tables
into
>>> memory, and to interpret/use those tables - but I'm not overly
familiar
>>> with how that works for JIT'd code. [Actually, looking at the
code for
>>> x86-64, it looks like it's mainly SEH (Structured Exception
Handling) that
>>> is dealt with - the overall concept still applies, but SEH is a
Windows
>>> concept for handling exceptions, which includes hardware exceptions
such as
>>> integer division by zero and memory access exceptions - regular C++
>>> exceptions are dealt with separately, and that is what uses what I
>>> described for Linux earlier in this paragraph].
>>>
>>> As to WHY different architectures use different relocations and
>>> exception handling tables, that's an ABI design issue - a
convention that
>>> is based on the needs and requirements for each architecture, and a
bunch
>>> of compromises between simplicity (a very simple table is easy to
>>> construct), space (simple table takes up more space than a more
complex
>>> table construction - like a zip file or a text file - the zip file
is more
>>> complicated to read, but takes up a lot less space) and code
complexity
>>> (save space in table, more complex code most likely). Either way,
for a
>>> given platform (OS, Processor, file format), there is a given ABI
for
>>> handling exceptions. The loader needs to load the table in the
correct way
>>> into the correct part of memory, and when an exception is thrown,
the
>>> table(s) need to be understood and acted upon to find the way back
to the
>>> relevant place where the exception is caught.
>>>
>>> The fact that the classes are declared in different files is
similar to
>>> my simple animal example, where you'd have a animal.h for the
base class, a
>>> cat.h, dog.h and fish.h for the actual implementations. Obviously,
the
>>> specific implementations for the RuntimeDyld belongs in
"Target" because
>>> they are dependent on the actual target (which is the combination
of
>>> fileformat, OS and processor architecture).
>>>
>>> --
>>> Mats
>>>
>>>
>>> On 20 April 2017 at 14:04, Siddharth Shankar Swain <
>>> h2015096 at pilani.bits-pilani.ac.in> wrote:
>>>
>>>> So RuntimeDyldELF.cpp or RuntimeDyldCOFF.cpp or
RuntimeDyldMachO.cpp
>>>>  are doing relocation and linking for specific object file
format and all
>>>> architectures using that object file format. Am i correct? If
that is so
>>>> then these  .cpp files are not using any header file in
Targets/ so what
>>>> are these header files in Targets/ made for ? Another thing is
that why
>>>> these header files in Targets/ handling exception and that too
using a
>>>> different concept of exception frames and exception tables.
Please guide
>>>> Thanks,
>>>> Siddharth
>>>>
>>>> On Thu, Apr 20, 2017 at 6:06 PM, mats petersson <mats at
planetcatfish.com
>>>> > wrote:
>>>>
>>>>> Basic Object Oriented design uses a derived class to
implement a
>>>>> functionality of the generic case. It's the same basic
principle as:
>>>>>
>>>>> class Animal
>>>>> {
>>>>>     void virtual Say() = 0;
>>>>> };
>>>>>
>>>>> class Cat: public Animal
>>>>> {
>>>>>     void Say() override { cout << "Meow!"
<< endl; }
>>>>> }
>>>>>
>>>>> class Dog: public Animal
>>>>> {
>>>>>     void Say() override { cout << "Woof!"
<< endl; }
>>>>> }
>>>>>
>>>>> class Fish: public Animal
>>>>> {
>>>>>     void Say() override { cout << "Blub!"
<< endl; }
>>>>> }
>>>>>
>>>>> In this case, different types of COFF-architectures use
different
>>>>> relocation entries, and based on the architecture, a
specific
>>>>> implementation of the RelocationDyldCOFF class is created
to perform the
>>>>> relocation.
>>>>>
>>>>> See
http://llvm.org/docs/doxygen/html/classllvm_1_1RuntimeDyldCO
>>>>> FF.html for a class diagram of how this is done.
>>>>>
>>>>> The generic code in RuntimeDyld*.cpp only knows that
relocations
>>>>> exists, and that they need to be dealt with. Not HOW to
actually perform
>>>>> the relocation - just like "Animal" doesn't
know what a cat or a dog
>>>>> "says". The processRelocationRef() is called
here:
>>>>>
http://llvm.org/docs/doxygen/html/RuntimeDyld_8cpp_source.html#l00251
>>>>>
>>>>> Again, it's not clear exactly what you are asking for,
so I'm not sure
>>>>> whether my explanation is helpful or not...
>>>>>
>>>>> --
>>>>> Mats
>>>>>
>>>>>
>>>>> On 20 April 2017 at 12:05, Siddharth Shankar Swain <
>>>>> h2015096 at pilani.bits-pilani.ac.in> wrote:
>>>>>
>>>>>> Thanks for the reply. I was just asking about in
general whatever
>>>>>> header files are there in Targets/ for different
architectures are not
>>>>>> including any function except this
processRelocationRef() to be used in
>>>>>> RuntimeDyldELF.cpp or RuntimeDyldCOFF.cpp or
RuntimeDyldMachO.cpp and i
>>>>>> think these files are the ones which are actually doing
the relocation and
>>>>>> linking work. So what purpose do these header files
inside Targets/
>>>>>> actually serve. Also they include exception handling in
form of exception
>>>>>> frames, So can u guide on this issue ?
>>>>>>
>>>>>> Thanks,
>>>>>> Siddharth
>>>>>>
>>>>>> On Thu, Apr 20, 2017 at 4:02 PM, mats petersson <
>>>>>> mats at planetcatfish.com> wrote:
>>>>>>
>>>>>>> The x86_64 and i386 architectures have different
actual relocation
>>>>>>> records. So if you build code for i386, you need
one processRelocationRef()
>>>>>>> function (handling the relevant relocations in that
model), and when
>>>>>>> producing code for x86_64, there are different
relocation records. The two
>>>>>>> files contain the derived form of the class that
processes the relocation
>>>>>>> records when dynamically loading JITed code in LLVM
- mainly implementing
>>>>>>> the two different forms of symbol entries that
refer to the relocations -
>>>>>>> i386 uses COFF::IMAGE_REL_I386_*, in x86_64 the
relocation types are
>>>>>>> COFF::IMAGE_REL_AMD64_*.
>>>>>>>
>>>>>>> Conceptually, they do the same thing, it's the
details of exactly
>>>>>>> how and where the relocation ends up and how
it's recorded by the linker
>>>>>>> that differs.
>>>>>>>
>>>>>>> Theoretically, one could probably construct a
loadable file that
>>>>>>> doesn't care what architecture it is for, but
it would end up with a lot of
>>>>>>> redundant & overlapping functionality, and the
code to handle every
>>>>>>> different architecture in one huge switch-statement
would be rather complex
>>>>>>> (and long!). So splitting the functionality per
architecture helps make the
>>>>>>> code clear.
>>>>>>>
>>>>>>> If you need further help to understand the code,
you'll probably
>>>>>>> need to ask a more concrete question, as it is
probably not possible to
>>>>>>> describe all the relevant information on this
subject in less than 200
>>>>>>> pages, never mind a simple email-thread.
>>>>>>>
>>>>>>> --
>>>>>>> Mats
>>>>>>>
>>>>>>> On 20 April 2017 at 11:13, Siddharth Shankar Swain
via llvm-dev <
>>>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> Can anyone explain in
lib/ExecutionEngine/RuntimeDyld/Targets/ the
>>>>>>>> header files included for different
architectures like
>>>>>>>> RuntimeDyldCOFFX86_64.h or
RuntimeDyldCOFFI386.h etc, what is the
>>>>>>>> connection of these files for relocation and
linking as the linking and
>>>>>>>> relocation for diff architecture is done in
RuntimeDyldELF.cpp,
>>>>>>>> RuntimeDyldCOFF.cpp  and it doesn't use any
function from these header file
>>>>>>>> except the processRelocationRef(). The header
files in Targets/ also
>>>>>>>> handles exceptions, so what is the need for
that in relocation and linking
>>>>>>>> process ? Also please help with what this
processRelocationRef() actually
>>>>>>>> does ? . Please guide.
>>>>>>>>
>>>>>>>> sincerely,
>>>>>>>> Siddharth
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> LLVM Developers mailing list
>>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170421/1b74be68/attachment.html>