Samuel Crow
2009-Sep-01 00:23 UTC
[LLVMdev] accessing a bitcode library exported from C++ using the JIT
----- Original Message ----> From: Eli Friedman <eli.friedman at gmail.com> > To: Samuel Crow <samuraileumas at yahoo.com> > Cc: LLVM Developers Mailing List <llvmdev at cs.uiuc.edu> > Sent: Monday, August 31, 2009 3:49:01 PM > Subject: Re: [LLVMdev] accessing a bitcode library exported from C++ using the JIT > > On Mon, Aug 31, 2009 at 12:17 PM, Samuel Crowwrote: > > Hello, > > > > My partner and I am making a small app needs to access a C++ library from the > LLVM 2.5 JIT. We've made sure that there are no classes and have put 'extern > "c"' in front of the functions we need to access. In order to make this work, > we seem to need to have a bitcode version of libstdc++ so we can avoid writing > our own implementations of std::string and std::vector. I've been trying to get > the version of libstdc++ that came with llvm-gcc-4.2-2.5 to configure and > compile. I've set the environment variables to use llvm-gcc and llvm-g++ > respectively and placed the --emit-llvm flag in the CCFLAGS and CXXFLAGS > environment variables. Configure works but make fails with: > > > > from > /Users/samuraicrow/Documents/llvm-gcc4.2-2.5.source/libstdc++-v3/include/precompiled/extc++.h:60: > > > /Users/samuraicrow/Documents/llvm-gcc4.2-2.5.source/libstdc++-v3/include/ext/pb_ds/detail/priority_queue_base_dispatch.hpp:79: > error: ISO C++ forbids declaration of ‘rc_binomial_heap_’ with no type > > > /Users/samuraicrow/Documents/llvm-gcc4.2-2.5.source/libstdc++-v3/include/ext/pb_ds/detail/priority_queue_base_dispatch.hpp:79: > error: expected ‘;’ before ‘<’ token > > > > I'm building on MacOSX 10.5.8 and my parnter will be building on MinGW under > Windows XP. > > > > Is this a problem with the source or with my configuration? Has this been > solved in version 2.6 prerelease or ToT? > > It's probably a problem with your configuration, since llvm-gcc can > definitely build libstdc++ as part of the normal build. > > This seems like the wrong approach, though... can't you just make your > C APIs take char*/T* instead? > > -EliHello Eli, We're using the LLVM Value * class functions to box and unbox values and functions for our stack. The stack needs to be able to take indexing without changing the stack pointer so we're using a std::vector for that. The std::string section would be a lot easier to replace than the two I just mentioned since we store the length separately. This library is serving as part of an interpreted parser generator so it needs to be able to handle all types and functions that we choose to feed into it. For now it is i32, Float and String and in the future, Doubles and functions of various sorts need to go onto the stack. The only thing I doubt about our approach is that we might be able to get away with using an interpreter instead of a JIT since the code is fed through the LLVM assembly parser just as it gets executed. If you're wondering why we're doing an interpreted PEG parser generator rather than Boost Spirit 2.x, it's because we need it to be easier to debug the parser. Once the parser is debugged it can be fed into a compiled parser generator and "frozen" into stand-alone parser code. --Sam
Eli Friedman
2009-Sep-01 00:36 UTC
[LLVMdev] accessing a bitcode library exported from C++ using the JIT
On Tue, Sep 1, 2009 at 4:53 AM, Samuel Crow<samuraileumas at yahoo.com> wrote:> We're using the LLVM Value * class functions to box and unbox values and functions for our stack. The stack needs to be able to take indexing without changing the stack pointer so we're using a std::vector for that. The std::string section would be a lot easier to replace than the two I just mentioned since we store the length separately.If you really want to stick with std::string and std::vector, I'd suggest just writing some wrappers for the relevant functions. For example: extern "C" char* String_cstr(std::string*); extern "C" void Stack_pushback(std::vector*, void* value); You can even compile these to bitcode with llvm-g++ and inline them into your other IR. I don't think compiling libstdc++ to bitcode actually helps you here; you can call the relevant functions in libstdc++ whether or not they're bitcode. -Eli
Samuel Crow
2009-Sep-01 00:57 UTC
[LLVMdev] accessing a bitcode library exported from C++ using the JIT
----- Original Message ----> From: Eli Friedman <eli.friedman at gmail.com> > To: Samuel Crow <samuraileumas at yahoo.com> > Cc: LLVM Developers Mailing List <llvmdev at cs.uiuc.edu> > Sent: Monday, August 31, 2009 7:36:10 PM > Subject: Re: [LLVMdev] accessing a bitcode library exported from C++ using the JIT > > On Tue, Sep 1, 2009 at 4:53 AM, Samuel Crowwrote: > > We're using the LLVM Value * class functions to box and unbox values and > functions for our stack. The stack needs to be able to take indexing without > changing the stack pointer so we're using a std::vector for that. The > std::string section would be a lot easier to replace than the two I just > mentioned since we store the length separately. > > If you really want to stick with std::string and std::vector, I'd > suggest just writing some wrappers for the relevant functions. For > example: > > extern "C" char* String_cstr(std::string*); > extern "C" void Stack_pushback(std::vector*, void* value); > > You can even compile these to bitcode with llvm-g++ and inline them > into your other IR. > > I don't think compiling libstdc++ to bitcode actually helps you here; > you can call the relevant functions in libstdc++ whether or not > they're bitcode. > > -EliHi Eli, That's exactly what we are trying to do. All of the parameters are passed in as text and parsed using StringStreams into their relavent types in the Value stack. You can view the current code at http://mattathias.cvs.sourceforge.net/viewvc/mattathias/mattathias/modules/front_ends/Amos/BasicLib.cpp?view=markup and the BasicLib.h included in it only has a bunch of LLVM includes and an irrelevant previous attempt at code abstraction and a C struct to return the length-terminated string in. I think my mistake is that I'm using Value * as a return code in one place instead of a void *. I think that's causing the name mangling to kick in. I'll see what I can do to fix it. Thanks for talking me through the problem, --Sam
OvermindDL1
2009-Sep-01 01:47 UTC
[LLVMdev] accessing a bitcode library exported from C++ using the JIT
On Mon, Aug 31, 2009 at 6:23 PM, Samuel Crow<samuraileumas at yahoo.com> wrote:> If you're wondering why we're doing an interpreted PEG parser generator rather than Boost Spirit 2.x, it's because we need it to be easier to debug the parser. Once the parser is debugged it can be fed into a compiled parser generator and "frozen" into stand-alone parser code.You do know that Spirit2.1 has very nice debugging facilities, everything from error handlers so you can handle parsing failures to getting detailed info reports on the entire processing process (of which you can do something like dump it to cout or something and you get this nice tree of everything that is tried, what passes, what fails, etc...). It is exceedingly detailed, so I am not sure what your homegrown one would have over it, if you have an example then I could add that functionality to Boost.Spirit2.1.
Samuel Crow
2009-Sep-01 12:12 UTC
[LLVMdev] accessing a bitcode library exported from C++ using the JIT
Hello OvermindDL1, We are implementing an extensible language. That's one where you can add commands and constructs to the language without having to recompile the parser. We want compilation of the parser in order to "freeze" it but only as an option. One goal is to eventually get the macro functions of our language to the point where they are equivalent to the template functions of C++ so we can make a self-hosting langauge. Also, all of the action nodes in our example PEG parser are written in LLVM Assembly rather than only C++ (although they call functions from a C++ library) so I doubt it would work in Spirit2.1 without writing a syntax converter. The syntax converter may come in time but it's only optional. If you're really interested in a partially finished example parser, it is at http://mattathias.cvs.sourceforge.net/viewvc/mattathias/mattathias/modules/front_ends/Amos/ . It contains a PEG parser file and the source to a C++ library included in that PEG file. Thanks for your interest, --Sam ----- Original Message ----> From: OvermindDL1 <overminddl1 at gmail.com> > To: LLVM Developers Mailing List <llvmdev at cs.uiuc.edu> > Sent: Monday, August 31, 2009 8:47:05 PM > Subject: Re: [LLVMdev] accessing a bitcode library exported from C++ using the JIT > > On Mon, Aug 31, 2009 at 6:23 PM, Samuel Crowwrote: > > If you're wondering why we're doing an interpreted PEG parser generator rather > than Boost Spirit 2.x, it's because we need it to be easier to debug the parser. > Once the parser is debugged it can be fed into a compiled parser generator and > "frozen" into stand-alone parser code. > > You do know that Spirit2.1 has very nice debugging facilities, > everything from error handlers so you can handle parsing failures to > getting detailed info reports on the entire processing process (of > which you can do something like dump it to cout or something and you > get this nice tree of everything that is tried, what passes, what > fails, etc...). It is exceedingly detailed, so I am not sure what > your homegrown one would have over it, if you have an example then I > could add that functionality to Boost.Spirit2.1. > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reasonably Related Threads
- [LLVMdev] accessing a bitcode library exported from C++ using the JIT
- [LLVMdev] accessing a bitcode library exported from C++ using the JIT
- [LLVMdev] accessing a bitcode library exported from C++ using the JIT
- [LLVMdev] PEG parsers? (was Re: Regular Expressions)
- [LLVMdev] Regular Expression lib support