llvm dev - Sep 2013 - [LLVMdev] VMKit state of the union, android support, and .net/CLI

I looked into the archives as far back as 2009 and searched around for more
information about vmkit, but I still have some questions.

First of all, what is that status of VMKit? Is there any active
development? A roadmap? Is it in maintenance mode?

Secondly, can VMKit generate binaries that can be used on android/jni?

Finally, I understand that the .net/CLI support is no longer being
maintained. Were there technical reasons for this, or just a lack of
interest? Is the code still within the repository or was it removed at some
point? If it was removed, what is the last tag/version of the code contains
the .net/CLI implementation?

The reason I ask is that I took over a project to bring Mono to android for
open source development. You can see it here:
https://github.com/JeroMiya/androidmono

But development hit a wall due to licensing, because Xamarin, the company
that develops mono, sells commercial licenses for Mono to use in place of
the LGPL on embedded platforms like Android/iOS, and claims that the LGPL
is not compatible with the Play store. While I disagree (as does the FSF as
I recall), I'm not a lawyer and can't justify hiring one for a hobbyist
project. In either case, it's clear Xamarin has become hostile to open
source development with their product, so it's not something I want to
test. This means apps built using my project could only be distributed
directly (via sideloading) or via F-Droid or other open source friendly
third party app stores, even if they're open source apps.

So, I've been shopping around for alternatives to the LGPL portions of the
Mono runtime which can more easily be licensed for open or closed source
development on mobile or restricted app store deployments. The only
credible alternatives I've found so far are the .Net MicroFramework, which
is licensed under Apache but is meant for low level hardware, and VMKit,
which sounds promising but the .Net implementation has been deprecated.

If someone is more familiar with the VMKit codebase, could you comment on
what kind of work would be needed to update the existing CLI implementation
of VMKit to the latest source? Also, what features still need to be
implemented, and what barriers have been identified? I'm not expecting
anything production ready by any means, and I'm willing to do the work as
long as there's at least a chance at success.

Thanks for your time!
Regards,
Jeremy Bell
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130913/ea8ecb66/attachment.html>

Hi Jeremy,

2013/9/13 Jeremy Bell <bell.jeremy at gmail.com>:
> I looked into the archives as far back as 2009 and searched around for more
> information about vmkit, but I still have some questions.
>
> First of all, what is that status of VMKit? Is there any active
development?
> A roadmap? Is it in maintenance mode?
The development is inactive since two month because the main engineer
of VMKit is now working for a big company (cool for him, but that's
why I was so slow to answer your mail:)). For the moment, we are in
maintenance mode, but a new engineer will probably start working on
the project in two or three weeks. And we have a roadmap, we are
currently implementing a new VM for the Scilab language and we are
developing a clang pass to help the development of new VMs. Basically,
the clang pass is able to generate the glue between the C code and the
GC, as it was presented in ISMM few years ago by another team (Jon
Rafkind, Adam Wick, John Regehr, Matthew Flatt: Precise garbage
collection for C. ISMM 2009).

>
> Secondly, can VMKit generate binaries that can be used on android/jni?
As android relies on Linux, it's probably easy to generate binaries
compliant with android (VMKit runs on Linux and MacOS). The problem
that can arise is related to the processor: we have only tested VMKit
on x86_32, x86_64 and powerpc architectures, not on arm
architectures... Porting VMKit on arm should not be so complicated as
llvm has already a backend for this architecture. It's probably a
matter of defining the size of the word for this architecture.

By JNI, you mean the Java Native Interface? On one way, as VMKit
generates native code with LLVM, you can link this code with a JNI
library and use the generated code in any Java virtual machine.
However, using VMKit through JNI requires the implementation of a
small wrapper because VMKit defines its own threads and a Java virtual
machine also. It probably means that you should have an intermediate
JNI threads that forwards the JNI calls to the VMKit threads. On the
other way, J3, the Java virtual machine built upon VMKit, is a full
Java virtual machine and fully implements JNI itself.
>
> Finally, I understand that the .net/CLI support is no longer being
> maintained. Were there technical reasons for this, or just a lack of
> interest? Is the code still within the repository or was it removed at some
> point? If it was removed, what is the last tag/version of the code contains
> the .net/CLI implementation?
We haven't maintained the CLI since three years because we didn't have
the time for that (we are a small team:)). You should find the code in
the repository. If you need it I can try to find the last revision.
Anyway, even if you are able to find this code, you will have to use a
very old version of llvm to compile it, and the vmkit version will be
very old with a lot of bugs... So, you will have to port the code of
N3 (the .Net virtual machine) on the latest VMKit release, which can
take few weeks because everything has changed in VMKit :) (N3 was
relying on the Boehm GC while we are currently using MMTk).
>
> The reason I ask is that I took over a project to bring Mono to android for
> open source development. You can see it here:
> https://github.com/JeroMiya/androidmono
Great project!
>
> But development hit a wall due to licensing, because Xamarin, the company
> that develops mono, sells commercial licenses for Mono to use in place of
> the LGPL on embedded platforms like Android/iOS, and claims that the LGPL
is
> not compatible with the Play store. While I disagree (as does the FSF as I
> recall), I'm not a lawyer and can't justify hiring one for a
hobbyist
> project. In either case, it's clear Xamarin has become hostile to open
> source development with their product, so it's not something I want to
test.
> This means apps built using my project could only be distributed directly
> (via sideloading) or via F-Droid or other open source friendly third party
> app stores, even if they're open source apps.
>
> So, I've been shopping around for alternatives to the LGPL portions of
the
> Mono runtime which can more easily be licensed for open or closed source
> development on mobile or restricted app store deployments. The only
credible
> alternatives I've found so far are the .Net MicroFramework, which is
> licensed under Apache but is meant for low level hardware, and VMKit, which
> sounds promising but the .Net implementation has been deprecated.
>
> If someone is more familiar with the VMKit codebase, could you comment on
> what kind of work would be needed to update the existing CLI implementation
> of VMKit to the latest source?
As I said, it should not be so difficult. For someone familiar with
VMKit, it will take one month. And understanding the code of VMKit
takes around one month more.
> Also, what features still need to be
> implemented, and what barriers have been identified?
We haven't identified barriers. The N3 runtime was minimal and we were
able to execute several scientific benchmarks. Having a full .Net
virtual machine with all the bytecodes is probably only a matter of
time.
> I'm not expecting
> anything production ready by any means, and I'm willing to do the work
as
> long as there's at least a chance at success.
If you start this work, we can try to help you to understand the code
(we can not help for implementation itself as, for the moment, we
don't have a lot of manpower). Having an android version of VMKit
could be fun, and having the .Net virtual machine resurrected could be
pretty cool :)

Gaël

PS: if you start playing with VMKit, we have a bug that we are unable
to solve yet with the last version of openjdk. We are only able to
compile VMKit with the version 7u23 of openjdk.
>
> Thanks for your time!
> Regards,
> Jeremy Bell
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>


-- 
-------------------------------------------------------------------
Gaël Thomas, Associate Professor, UPMC
http://pagesperso-systeme.lip6.fr/Gael.Thomas/
-------------------------------------------------------------------

Thanks for responding! It's good to know there weren't any particular
technical barriers that would prevent a working N3 implementation.

I have some followup questions:
Did N3 support reflection and/or generics? How, if at all, did N3 support
structs? What about ref and out parameters? Have you considered using
Mono.Cecil for parsing out the .net assemblies, metadata, and bytecode?

Also, Mono has support for LLVM already, though the generated LLVM code
links with the mono runtime. It might be possible to adapt this LLVM
compiler backend to link with VMKit instead of mono, or else provide an
adapter library that follows the interface of the mono VM, but forwards
calls into VMKit, so the generated code isn't bound by the LGPL of the mono
runtime. There are several known issues, as described here (e.g. varargs
doesn't work):

http://www.mono-project.com/Mono:Runtime:Documentation:LLVM

Either way, this might provide a head-start in terms of implementing the
full range of CIL opcodes.


On Sun, Sep 15, 2013 at 3:33 AM, Gaël Thomas <gael.thomas at lip6.fr>
wrote:
> Hi Jeremy,
>
> 2013/9/13 Jeremy Bell <bell.jeremy at gmail.com>:
> > I looked into the archives as far back as 2009 and searched around for
> more
> > information about vmkit, but I still have some questions.
> >
> > First of all, what is that status of VMKit? Is there any active
> development?
> > A roadmap? Is it in maintenance mode?
>
> The development is inactive since two month because the main engineer
> of VMKit is now working for a big company (cool for him, but that's
> why I was so slow to answer your mail:)). For the moment, we are in
> maintenance mode, but a new engineer will probably start working on
> the project in two or three weeks. And we have a roadmap, we are
> currently implementing a new VM for the Scilab language and we are
> developing a clang pass to help the development of new VMs. Basically,
> the clang pass is able to generate the glue between the C code and the
> GC, as it was presented in ISMM few years ago by another team (Jon
> Rafkind, Adam Wick, John Regehr, Matthew Flatt: Precise garbage
> collection for C. ISMM 2009).
>
>
> >
> > Secondly, can VMKit generate binaries that can be used on android/jni?
>
> As android relies on Linux, it's probably easy to generate binaries
> compliant with android (VMKit runs on Linux and MacOS). The problem
> that can arise is related to the processor: we have only tested VMKit
> on x86_32, x86_64 and powerpc architectures, not on arm
> architectures... Porting VMKit on arm should not be so complicated as
> llvm has already a backend for this architecture. It's probably a
> matter of defining the size of the word for this architecture.
>
> By JNI, you mean the Java Native Interface? On one way, as VMKit
> generates native code with LLVM, you can link this code with a JNI
> library and use the generated code in any Java virtual machine.
> However, using VMKit through JNI requires the implementation of a
> small wrapper because VMKit defines its own threads and a Java virtual
> machine also. It probably means that you should have an intermediate
> JNI threads that forwards the JNI calls to the VMKit threads. On the
> other way, J3, the Java virtual machine built upon VMKit, is a full
> Java virtual machine and fully implements JNI itself.
>
> >
> > Finally, I understand that the .net/CLI support is no longer being
> > maintained. Were there technical reasons for this, or just a lack of
> > interest? Is the code still within the repository or was it removed at
> some
> > point? If it was removed, what is the last tag/version of the code
> contains
> > the .net/CLI implementation?
>
> We haven't maintained the CLI since three years because we didn't
have
> the time for that (we are a small team:)). You should find the code in
> the repository. If you need it I can try to find the last revision.
> Anyway, even if you are able to find this code, you will have to use a
> very old version of llvm to compile it, and the vmkit version will be
> very old with a lot of bugs... So, you will have to port the code of
> N3 (the .Net virtual machine) on the latest VMKit release, which can
> take few weeks because everything has changed in VMKit :) (N3 was
> relying on the Boehm GC while we are currently using MMTk).
>
> >
> > The reason I ask is that I took over a project to bring Mono to
android
> for
> > open source development. You can see it here:
> > https://github.com/JeroMiya/androidmono
>
> Great project!
>
> >
> > But development hit a wall due to licensing, because Xamarin, the
company
> > that develops mono, sells commercial licenses for Mono to use in place
of
> > the LGPL on embedded platforms like Android/iOS, and claims that the
> LGPL is
> > not compatible with the Play store. While I disagree (as does the FSF
as
> I
> > recall), I'm not a lawyer and can't justify hiring one for a
hobbyist
> > project. In either case, it's clear Xamarin has become hostile to
open
> > source development with their product, so it's not something I
want to
> test.
> > This means apps built using my project could only be distributed
directly
> > (via sideloading) or via F-Droid or other open source friendly third
> party
> > app stores, even if they're open source apps.
> >
> > So, I've been shopping around for alternatives to the LGPL
portions of
> the
> > Mono runtime which can more easily be licensed for open or closed
source
> > development on mobile or restricted app store deployments. The only
> credible
> > alternatives I've found so far are the .Net MicroFramework, which
is
> > licensed under Apache but is meant for low level hardware, and VMKit,
> which
> > sounds promising but the .Net implementation has been deprecated.
> >
> > If someone is more familiar with the VMKit codebase, could you comment
on
> > what kind of work would be needed to update the existing CLI
> implementation
> > of VMKit to the latest source?
>
> As I said, it should not be so difficult. For someone familiar with
> VMKit, it will take one month. And understanding the code of VMKit
> takes around one month more.
>
> > Also, what features still need to be
> > implemented, and what barriers have been identified?
>
> We haven't identified barriers. The N3 runtime was minimal and we were
> able to execute several scientific benchmarks. Having a full .Net
> virtual machine with all the bytecodes is probably only a matter of
> time.
>
> > I'm not expecting
> > anything production ready by any means, and I'm willing to do the
work as
> > long as there's at least a chance at success.
>
> If you start this work, we can try to help you to understand the code
> (we can not help for implementation itself as, for the moment, we
> don't have a lot of manpower). Having an android version of VMKit
> could be fun, and having the .Net virtual machine resurrected could be
> pretty cool :)
>
> Gaël
>
> PS: if you start playing with VMKit, we have a bug that we are unable
> to solve yet with the last version of openjdk. We are only able to
> compile VMKit with the version 7u23 of openjdk.
>
> >
> > Thanks for your time!
> > Regards,
> > Jeremy Bell
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> >
>
>
>
> --
> -------------------------------------------------------------------
> Gaël Thomas, Associate Professor, UPMC
> http://pagesperso-systeme.lip6.fr/Gael.Thomas/
> -------------------------------------------------------------------
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130919/b5f55c8f/attachment.html>