llvm dev - Mar 2009 - [LLVMdev] Issue with paper http://llvm.org/devmtg/2008-08/Geoffray_VMKitProject.pdf and presentation http://hal.inria.fr/docs/00/35/45/77/PDF/RR-6799.pdf

Hi Chris,


Just in case that this information does not get corrupted I want to
point to the issue:


In http://llvm.org/devmtg/2008-08/Geoffray_VMKitProject.pdf it is on
page 21: "VMKit, Mono, PNet", Array bounds checks.

In http://hal.inria.fr/docs/00/35/45/77/PDF/RR-6799.pdf it is on page
17: "We compare N3 with Mono [9] version 1.2.6, and PNet version 0.7.4
[2], which uses libjit as dynamic compiler...."
This statement is false. PNet version 0.7.4 has not been using LibJIT.
It has been using an unroller.

"The PNet performance show that libjit has very few optimizations and
does not compete against robust compilers."

In fact any amateur in Just-In-Time compilers, and Virtual Machine
engines, by just looking at this benchmark on the next page "Figure 7:
PNetMark results (higher is better)", can tell that PNet engine used
to take these measures has not been a Just-In-Time compiler. However,
the authors of the paper insist.

This benchmark definitely does not tell anything about LibJIT, as
LibJIT has not been used. In fact, we started development of LibJIT as
an answer to such benchmarks issues.

I can suggest the authors of these papers to use a real industrial VM
engine to compare. For instance, check Microsoft .NET or if you cannot
do this "for legal reason" try Java HotSpot. But not a 5 years old
version, although even that should give you more objective results
that you can understand what these benchmark mean in fact.


Thanks,
Kirill

2009/3/2 Chris Lattner <clattner at apple.com>:
>
> On Mar 2, 2009, at 8:25 AM, Kirill Kononenko wrote:
>
>> Hello Dear LLVM Developers and Users
>
> Hi Kirill,
>
> Please talk to Nicholas about this, thanks!
>
> -Chris
>
>>
>>
>> I would like to complain about scientific inaccuracy of the paper
>> http://hal.inria.fr/docs/00/35/45/77/PDF/RR-6799.pdf written by
>> Nicolas Geoffray, Gaël Thomas, Charles Clément, Bertil Folliot, Gilles
>> Muller and presentation done by Nicolas Geoffray on LLVM conference
>> supported by Google:
>> http://llvm.org/devmtg/2008-08/Geoffray_VMKitProject.pdf
>>
>> They provide comparison of their system that uses LLVM, with our
>> system Portable.NET that uses LibJIT for Just-In-Time compilation.
>> All measures in this paper have been done for Portable.NET 0.7.4,
>> which does not use LibJIT as its engine. However, the author provides
>> results of Portable.NET 0.7.4. as results of LibJIT. The Portable.NET
>> 0.7.4 uses a dynamic compiler, which has nothing in common with the
>> project and development of LibJIT. We are very upset with these papers
>> and that there are such "researchers" in your group.
Portable.NET
>> 0.7.4 was released 3 years ago. LibJIT have been developed during the
>> last 3 years. We require that you remove this paper and this
>> presentation and make official excuses not only in this maillist but
>> also on the next conferences when you talk about LLVM.
>>
>>
>> Thanks,
>> Kirill
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>

Dear all,

First, sorry for all that noise. I know you guys *just don't care*TM, 
but this requires some clarification. I won't say more than what's in 
this email.

The VMKit paper dates from early 2008. It compares VMKit with pnet 0.7.4 
and not the latest release (0.8.0) because we were not able to get any 
information to build it correctly at the time; the website that hosted 
libjit was broken, i.e. http://www.southern-storm.com.au. That said, 
here's the info that we have on the pnet website:
> January 15-01-2006: DotGNU Portable.NET 0.7.4 and libjit 0.0.6 released!
>
> New versions of Portable.NET and libjit are available. Far too many 
> new features and bug fixes to list them here, see the announcement 
> which includes a full list of new features and bug fixes. You can get 
> it from the download page.
>
> December 03-12-2005: Portable.NET and libjit progress
>
> Work on libjit (A JIT designed to be independent of any particular 
> virtual machine bytecode format or language) has resumed again and we 
> are gearing towards using it in Portable.NET soon. Recently, there 
> have also been donations which will be distributed amongst developers 
> who contribute code to get libjit finished and hook it into 
> Portable.NET's runtime engine.
Maybe it's our mistake if we misinterpreted the informations given on 
that webpage. From our understanding, libjit and pnet were merged in 
0.7.4. Now that we all know pnet and libjit are two separate, alive, 
projects, we will stop refering to pnet only. The VMKit paper got rid of 
libjit references. As for the LLVM developer meeting presentation, there 
is no mention of libjit.

End of the story.

Nicolas

Kirill Kononenko wrote:
> Hi Chris,
>
>
> Just in case that this information does not get corrupted I want to
> point to the issue:
>
>
> In http://llvm.org/devmtg/2008-08/Geoffray_VMKitProject.pdf it is on
> page 21: "VMKit, Mono, PNet", Array bounds checks.
>
> In http://hal.inria.fr/docs/00/35/45/77/PDF/RR-6799.pdf it is on page
> 17: "We compare N3 with Mono [9] version 1.2.6, and PNet version 0.7.4
> [2], which uses libjit as dynamic compiler...."
> This statement is false. PNet version 0.7.4 has not been using LibJIT.
> It has been using an unroller.
>
> "The PNet performance show that libjit has very few optimizations and
> does not compete against robust compilers."
>
> In fact any amateur in Just-In-Time compilers, and Virtual Machine
> engines, by just looking at this benchmark on the next page "Figure 7:
> PNetMark results (higher is better)", can tell that PNet engine used
> to take these measures has not been a Just-In-Time compiler. However,
> the authors of the paper insist.
>
> This benchmark definitely does not tell anything about LibJIT, as
> LibJIT has not been used. In fact, we started development of LibJIT as
> an answer to such benchmarks issues.
>
> I can suggest the authors of these papers to use a real industrial VM
> engine to compare. For instance, check Microsoft .NET or if you cannot
> do this "for legal reason" try Java HotSpot. But not a 5 years
old
> version, although even that should give you more objective results
> that you can understand what these benchmark mean in fact.
>
>
> Thanks,
> Kirill
>
> 2009/3/2 Chris Lattner <clattner at apple.com>:
>   
>> On Mar 2, 2009, at 8:25 AM, Kirill Kononenko wrote:
>>
>>     
>>> Hello Dear LLVM Developers and Users
>>>       
>> Hi Kirill,
>>
>> Please talk to Nicholas about this, thanks!
>>
>> -Chris
>>
>>     
>>> I would like to complain about scientific inaccuracy of the paper
>>> http://hal.inria.fr/docs/00/35/45/77/PDF/RR-6799.pdf written by
>>> Nicolas Geoffray, Gaël Thomas, Charles Clément, Bertil Folliot,
Gilles
>>> Muller and presentation done by Nicolas Geoffray on LLVM conference
>>> supported by Google:
>>> http://llvm.org/devmtg/2008-08/Geoffray_VMKitProject.pdf
>>>
>>> They provide comparison of their system that uses LLVM, with our
>>> system Portable.NET that uses LibJIT for Just-In-Time compilation.
>>> All measures in this paper have been done for Portable.NET 0.7.4,
>>> which does not use LibJIT as its engine. However, the author
provides
>>> results of Portable.NET 0.7.4. as results of LibJIT. The
Portable.NET
>>> 0.7.4 uses a dynamic compiler, which has nothing in common with the
>>> project and development of LibJIT. We are very upset with these
papers
>>> and that there are such "researchers" in your group.
Portable.NET
>>> 0.7.4 was released 3 years ago. LibJIT have been developed during
the
>>> last 3 years. We require that you remove this paper and this
>>> presentation and make official excuses not only in this maillist
but
>>> also on the next conferences when you talk about LLVM.
>>>
>>>
>>> Thanks,
>>> Kirill
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>       
>>     
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

Dear all,

This new version of the paper at:

http://hal.archives-ouvertes.fr/docs/00/36/53/37/PDF/RR-6799.pdf

get ride of libJIT. Now my question is the how scientific is this
research if instead of using libJIT it simply decided to remove this
information all together? Also how object is Related Work in this
context?


Thanks,
Kirill

2009/3/3 Nicolas Geoffray <nicolas.geoffray at
lip6.fr>:
> Dear all,
>
> First, sorry for all that noise. I know you guys *just don't care*TM,
but
> this requires some clarification. I won't say more than what's in
this
> email.
>
> The VMKit paper dates from early 2008. It compares VMKit with pnet 0.7.4
and
> not the latest release (0.8.0) because we were not able to get any
> information to build it correctly at the time; the website that hosted
> libjit was broken, i.e. http://www.southern-storm.com.au. That said,
here's
> the info that we have on the pnet website:
>
>> January 15-01-2006: DotGNU Portable.NET 0.7.4 and libjit 0.0.6
released!
>>
>> New versions of Portable.NET and libjit are available. Far too many new
>> features and bug fixes to list them here, see the announcement which
>> includes a full list of new features and bug fixes. You can get it from
the
>> download page.
>>
>> December 03-12-2005: Portable.NET and libjit progress
>>
>> Work on libjit (A JIT designed to be independent of any particular
virtual
>> machine bytecode format or language) has resumed again and we are
gearing
>> towards using it in Portable.NET soon. Recently, there have also been
>> donations which will be distributed amongst developers who contribute
code
>> to get libjit finished and hook it into Portable.NET's runtime
engine.
>
> Maybe it's our mistake if we misinterpreted the informations given on
that
> webpage. From our understanding, libjit and pnet were merged in 0.7.4. Now
> that we all know pnet and libjit are two separate, alive, projects, we will
> stop refering to pnet only. The VMKit paper got rid of libjit references.
As
> for the LLVM developer meeting presentation, there is no mention of libjit.
>
> End of the story.
>
> Nicolas
>
> Kirill Kononenko wrote:
>>
>> Hi Chris,
>>
>>
>> Just in case that this information does not get corrupted I want to
>> point to the issue:
>>
>>
>> In http://llvm.org/devmtg/2008-08/Geoffray_VMKitProject.pdf it is on
>> page 21: "VMKit, Mono, PNet", Array bounds checks.
>>
>> In http://hal.inria.fr/docs/00/35/45/77/PDF/RR-6799.pdf it is on page
>> 17: "We compare N3 with Mono [9] version 1.2.6, and PNet version
0.7.4
>> [2], which uses libjit as dynamic compiler...."
>> This statement is false. PNet version 0.7.4 has not been using LibJIT.
>> It has been using an unroller.
>>
>> "The PNet performance show that libjit has very few optimizations
and
>> does not compete against robust compilers."
>>
>> In fact any amateur in Just-In-Time compilers, and Virtual Machine
>> engines, by just looking at this benchmark on the next page
"Figure 7:
>> PNetMark results (higher is better)", can tell that PNet engine
used
>> to take these measures has not been a Just-In-Time compiler. However,
>> the authors of the paper insist.
>>
>> This benchmark definitely does not tell anything about LibJIT, as
>> LibJIT has not been used. In fact, we started development of LibJIT as
>> an answer to such benchmarks issues.
>>
>> I can suggest the authors of these papers to use a real industrial VM
>> engine to compare. For instance, check Microsoft .NET or if you cannot
>> do this "for legal reason" try Java HotSpot. But not a 5
years old
>> version, although even that should give you more objective results
>> that you can understand what these benchmark mean in fact.
>>
>>
>> Thanks,
>> Kirill
>>
>> 2009/3/2 Chris Lattner <clattner at apple.com>:
>>
>>>
>>> On Mar 2, 2009, at 8:25 AM, Kirill Kononenko wrote:
>>>
>>>
>>>>
>>>> Hello Dear LLVM Developers and Users
>>>>
>>>
>>> Hi Kirill,
>>>
>>> Please talk to Nicholas about this, thanks!
>>>
>>> -Chris
>>>
>>>
>>>>
>>>> I would like to complain about scientific inaccuracy of the
paper
>>>> http://hal.inria.fr/docs/00/35/45/77/PDF/RR-6799.pdf written by
>>>> Nicolas Geoffray, Gaël Thomas, Charles Clément, Bertil Folliot,
Gilles
>>>> Muller and presentation done by Nicolas Geoffray on LLVM
conference
>>>> supported by Google:
>>>> http://llvm.org/devmtg/2008-08/Geoffray_VMKitProject.pdf
>>>>
>>>> They provide comparison of their system that uses LLVM, with
our
>>>> system Portable.NET that uses LibJIT for Just-In-Time
compilation.
>>>> All measures in this paper have been done for Portable.NET
0.7.4,
>>>> which does not use LibJIT as its engine. However, the author
provides
>>>> results of Portable.NET 0.7.4. as results of LibJIT. The
Portable.NET
>>>> 0.7.4 uses a dynamic compiler, which has nothing in common with
the
>>>> project and development of LibJIT. We are very upset with these
papers
>>>> and that there are such "researchers" in your group.
Portable.NET
>>>> 0.7.4 was released 3 years ago. LibJIT have been developed
during the
>>>> last 3 years. We require that you remove this paper and this
>>>> presentation and make official excuses not only in this
maillist but
>>>> also on the next conferences when you talk about LLVM.
>>>>
>>>>
>>>> Thanks,
>>>> Kirill
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>
>>>
>>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>
>


-- 
http://code.google.com/p/libjit-linear-scan-register-allocator/

Dear all,


This new version of the paper at:

http://hal.archives-ouvertes.fr/docs/00/36/53/37/PDF/RR-6799.pdf

got ride of libJIT. Now my question is the how scientific is this
research and research paper if instead of using libJIT and
Portable.NET JIT:
authors of "VMKit: a Substrate for Virtual Machines by Nicolas
Geoffray — Gaël Thomas— Charles Clément —Bertil Folliot — Gilles
Muller" simply decided to decided remove research information all
together? Also how objective is Related Work part of this research
paper in this context?


Thanks,
Kirill

2009/3/3 Nicolas Geoffray <nicolas.geoffray at
lip6.fr>:
> Dear all,
>
> First, sorry for all that noise. I know you guys *just don't care*TM,
but
> this requires some clarification. I won't say more than what's in
this
> email.
>
> The VMKit paper dates from early 2008. It compares VMKit with pnet 0.7.4
and
> not the latest release (0.8.0) because we were not able to get any
> information to build it correctly at the time; the website that hosted
> libjit was broken, i.e. http://www.southern-storm.com.au. That said,
here's
> the info that we have on the pnet website:
>
>> January 15-01-2006: DotGNU Portable.NET 0.7.4 and libjit 0.0.6
released!
>>
>> New versions of Portable.NET and libjit are available. Far too many new
>> features and bug fixes to list them here, see the announcement which
>> includes a full list of new features and bug fixes. You can get it from
the
>> download page.
>>
>> December 03-12-2005: Portable.NET and libjit progress
>>
>> Work on libjit (A JIT designed to be independent of any particular
virtual
>> machine bytecode format or language) has resumed again and we are
gearing
>> towards using it in Portable.NET soon. Recently, there have also been
>> donations which will be distributed amongst developers who contribute
code
>> to get libjit finished and hook it into Portable.NET's runtime
engine.
>
> Maybe it's our mistake if we misinterpreted the informations given on
that
> webpage. From our understanding, libjit and pnet were merged in 0.7.4. Now
> that we all know pnet and libjit are two separate, alive, projects, we will
> stop refering to pnet only. The VMKit paper got rid of libjit references.
As
> for the LLVM developer meeting presentation, there is no mention of libjit.
>
> End of the story.
>
> Nicolas
>
> Kirill Kononenko wrote:
>>
>> Hi Chris,
>>
>>
>> Just in case that this information does not get corrupted I want to
>> point to the issue:
>>
>>
>> In http://llvm.org/devmtg/2008-08/Geoffray_VMKitProject.pdf it is on
>> page 21: "VMKit, Mono, PNet", Array bounds checks.
>>
>> In http://hal.inria.fr/docs/00/35/45/77/PDF/RR-6799.pdf it is on page
>> 17: "We compare N3 with Mono [9] version 1.2.6, and PNet version
0.7.4
>> [2], which uses libjit as dynamic compiler...."
>> This statement is false. PNet version 0.7.4 has not been using LibJIT.
>> It has been using an unroller.
>>
>> "The PNet performance show that libjit has very few optimizations
and
>> does not compete against robust compilers."
>>
>> In fact any amateur in Just-In-Time compilers, and Virtual Machine
>> engines, by just looking at this benchmark on the next page
"Figure 7:
>> PNetMark results (higher is better)", can tell that PNet engine
used
>> to take these measures has not been a Just-In-Time compiler. However,
>> the authors of the paper insist.
>>
>> This benchmark definitely does not tell anything about LibJIT, as
>> LibJIT has not been used. In fact, we started development of LibJIT as
>> an answer to such benchmarks issues.
>>
>> I can suggest the authors of these papers to use a real industrial VM
>> engine to compare. For instance, check Microsoft .NET or if you cannot
>> do this "for legal reason" try Java HotSpot. But not a 5
years old
>> version, although even that should give you more objective results
>> that you can understand what these benchmark mean in fact.
>>
>>
>> Thanks,
>> Kirill
>>
>> 2009/3/2 Chris Lattner <clattner at apple.com>:
>>
>>>
>>> On Mar 2, 2009, at 8:25 AM, Kirill Kononenko wrote:
>>>
>>>
>>>>
>>>> Hello Dear LLVM Developers and Users
>>>>
>>>
>>> Hi Kirill,
>>>
>>> Please talk to Nicholas about this, thanks!
>>>
>>> -Chris
>>>
>>>
>>>>
>>>> I would like to complain about scientific inaccuracy of the
paper
>>>> http://hal.inria.fr/docs/00/35/45/77/PDF/RR-6799.pdf written by
>>>> Nicolas Geoffray, Gaël Thomas, Charles Clément, Bertil Folliot,
Gilles
>>>> Muller and presentation done by Nicolas Geoffray on LLVM
conference
>>>> supported by Google:
>>>> http://llvm.org/devmtg/2008-08/Geoffray_VMKitProject.pdf
>>>>
>>>> They provide comparison of their system that uses LLVM, with
our
>>>> system Portable.NET that uses LibJIT for Just-In-Time
compilation.
>>>> All measures in this paper have been done for Portable.NET
0.7.4,
>>>> which does not use LibJIT as its engine. However, the author
provides
>>>> results of Portable.NET 0.7.4. as results of LibJIT. The
Portable.NET
>>>> 0.7.4 uses a dynamic compiler, which has nothing in common with
the
>>>> project and development of LibJIT. We are very upset with these
papers
>>>> and that there are such "researchers" in your group.
Portable.NET
>>>> 0.7.4 was released 3 years ago. LibJIT have been developed
during the
>>>> last 3 years. We require that you remove this paper and this
>>>> presentation and make official excuses not only in this
maillist but
>>>> also on the next conferences when you talk about LLVM.
>>>>
>>>>
>>>> Thanks,
>>>> Kirill
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>
>>>
>>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>
>


-- 
http://code.google.com/p/libjit-linear-scan-register-allocator/