Chris Lattner
2007-Aug-14 18:46 UTC
[LLVMdev] Some questions about building an LLVM frontend
On Aug 13, 2007, at 9:20 PM, Talin wrote:> Hi, I spoke with you briefly after your talk at Google; I've since > been playing around with LLVM quite a bit and started working on a > front end for my experimental programming language, Tart.Cool, welcome :)> A couple of questions I wanted to ask (although feel free to answer > by saying "go ask on the mailing list".)The questions you have may be interesting to others, so I'm cc'ing the llvm dev list.> I notice in clang that you define your own set of classes for > representing Types, independent of the Type class in LLVM. I'm > trying to decide if I should do the same - is it generally > recommended that front ends use the LLVM type classes directly > during parsing and analysis, or do front ends generally define > their own data structures and the convert to LLVM form when they > communicate with the back end?I strongly recommend defining your own type system. The LLVM type system has been built to do one thing well: analyze and optimize programs in a low-level language independent form. You will not get much benefit from extending them, and you'd have to deal with a lot of complexity. I'd strongly suggest defining your own type system, then lowering your front-end types to the LLVM types when doing the "code generation" stage.> There are some concepts in my language which I can't quite grok how > to represent in terms of LLVM types. > > For example, one of the type modifiers is the 'nullable' modifier. > Normally references to objects aren't allowed to be NULL, however > in this language you can put a question mark after a type name to > indicate that this particular reference can be NULL. (Essentially > it declares a disjoint type between the object and the NULL type.) > Mixing nullable and non-nullable types produces a warning unless > the compiler can deduce that the value is indeed non-null. > > What I'd like to know is, is it possible to subclass or otherwise > customize an LLVM type to maintain this information, or should I > instead create a parallel type hierarchy that can store information > like this?One common question is "how do I expose certain high level information to the optimizer". Unfortunately this answer has many different possible answers depending on the specific information and the constraints on the problem. In LLVM, tend to try to avoid encoding information into the type system, preferring instead to encode it into the operations. In any case I'd suggest working on getting your language up and correct first, we can always worry about extending llvm as needed in the future.> Another question is how to handle builtin functions. In this > language, the various operators, such as '+', are just synonyms for > builtin functions such as operator.add, which are overloaded by > argument type. Generally, one would want to insure that these > functions are nearly always inlined. The question is, should the > front-end be responsible for the inlining, or should it rely on > LLVM to do it?There are actually multiple ways of doing this. I suspect that the LLVM inliner will have no problem inlining these operations, and defining them as function bodies with a standard C interface has a lot of advantages. A middle ground is to go the route of relying on the LLVM inliner to handle these, but then explicitly call the inliner on each one you want to ensure gets inlined. This way you get the convenience of dealing with calls in your front-end, but don't have to worry about the whims of the inliner (which could be significant when optimization is disabled for example). -Chris
I've been working some more on my front-end, incorporating the advice from your previous email. I've had some success, in that I can generate a Module object, populated it with functions and instructions, and print out the resulting LLVM IR. But I'm still a little puzzled on how to connect the front end with LLVM's backend code generator, I can't seem to figure out how to run the various optimization and code generation passes. I've looked extensively at the documentation on the web site, but this particular aspect of LLVM seems curiously absent - that is, I see lots of docs on how to *build* LLVM, how to create the inputs for LLVM, the *theory* of LLVM, and how to *modify* LLVM; But I can find little on how to *call* it. (I mean, as a library, not as a shell command.) I've also looked at the various examples and tools. Most of them seem focused on generating either bitcode or JITs, which I'm not interested in - I want to get an executable containing natively compiled code. The one example that seems close to what I want is the llc.cpp source file; But when I try to do the same thing in my code, I find that the TargetMachineRegistry is empty; And yet I see no code in llc.cpp that initializes this, nor do I see #includes or Makefile dependencies in the llc project that would do this implicitly. Sincerely Puzzled, -- Talin Chris Lattner wrote:> On Aug 13, 2007, at 9:20 PM, Talin wrote: >> Hi, I spoke with you briefly after your talk at Google; I've since >> been playing around with LLVM quite a bit and started working on a >> front end for my experimental programming language, Tart. > > Cool, welcome :) > >> A couple of questions I wanted to ask (although feel free to answer by >> saying "go ask on the mailing list".) > > The questions you have may be interesting to others, so I'm cc'ing the > llvm dev list. > >> I notice in clang that you define your own set of classes for >> representing Types, independent of the Type class in LLVM. I'm trying >> to decide if I should do the same - is it generally recommended that >> front ends use the LLVM type classes directly during parsing and >> analysis, or do front ends generally define their own data structures >> and the convert to LLVM form when they communicate with the back end? > > I strongly recommend defining your own type system. The LLVM type > system has been built to do one thing well: analyze and optimize > programs in a low-level language independent form. You will not get > much benefit from extending them, and you'd have to deal with a lot of > complexity. I'd strongly suggest defining your own type system, then > lowering your front-end types to the LLVM types when doing the "code > generation" stage. > >> There are some concepts in my language which I can't quite grok how to >> represent in terms of LLVM types. >> >> For example, one of the type modifiers is the 'nullable' modifier. >> Normally references to objects aren't allowed to be NULL, however in >> this language you can put a question mark after a type name to >> indicate that this particular reference can be NULL. (Essentially it >> declares a disjoint type between the object and the NULL type.) Mixing >> nullable and non-nullable types produces a warning unless the compiler >> can deduce that the value is indeed non-null. >> >> What I'd like to know is, is it possible to subclass or otherwise >> customize an LLVM type to maintain this information, or should I >> instead create a parallel type hierarchy that can store information >> like this? > > One common question is "how do I expose certain high level information > to the optimizer". Unfortunately this answer has many different > possible answers depending on the specific information and the > constraints on the problem. In LLVM, tend to try to avoid encoding > information into the type system, preferring instead to encode it into > the operations. In any case I'd suggest working on getting your > language up and correct first, we can always worry about extending llvm > as needed in the future. > >> Another question is how to handle builtin functions. In this language, >> the various operators, such as '+', are just synonyms for builtin >> functions such as operator.add, which are overloaded by argument type. >> Generally, one would want to insure that these functions are nearly >> always inlined. The question is, should the front-end be responsible >> for the inlining, or should it rely on LLVM to do it? > > There are actually multiple ways of doing this. I suspect that the LLVM > inliner will have no problem inlining these operations, and defining > them as function bodies with a standard C interface has a lot of > advantages. A middle ground is to go the route of relying on the LLVM > inliner to handle these, but then explicitly call the inliner on each > one you want to ensure gets inlined. This way you get the convenience > of dealing with calls in your front-end, but don't have to worry about > the whims of the inliner (which could be significant when optimization > is disabled for example). > > -Chris >
Rafael Espindola
2007-Aug-25 09:52 UTC
[LLVMdev] Some questions about building an LLVM frontend
> The one example that seems close to what I want is the llc.cpp source > file; But when I try to do the same thing in my code, I find that the > TargetMachineRegistry is empty; And yet I see no code in llc.cpp that > initializes this, nor do I see #includes or Makefile dependencies in the > llc project that would do this implicitly. > > Sincerely Puzzled,If i remember correctly, all you need to do is link with some of the target code generators. They use the static constructors to register them in TargetMachineRegistry.> -- TalinCheers, -- Rafael Avila de Espindola Google Ireland Ltd. Gordon House Barrow Street Dublin 4 Ireland Registered in Dublin, Ireland Registration Number: 368047
Chris Lattner
2007-Aug-25 20:02 UTC
[LLVMdev] Some questions about building an LLVM frontend
On Aug 24, 2007, at 7:02 PM, Talin wrote:> I've been working some more on my front-end, incorporating the advice > from your previous email. > > I've had some success, in that I can generate a Module object, > populated > it with functions and instructions, and print out the resulting > LLVM IR.Great!> But I'm still a little puzzled on how to connect the front end with > LLVM's backend code generator, I can't seem to figure out how to > run the > various optimization and code generation passes.You need to set up a passmanager or two, populate them with the passes you want, then tell them to run on the code, when appropriate.> The one example that seems close to what I want is the llc.cpp source > file; But when I try to do the same thing in my code, I find that the > TargetMachineRegistry is empty; And yet I see no code in llc.cpp that > initializes this, nor do I see #includes or Makefile dependencies > in the > llc project that would do this implicitly.The best example here is llvm-gcc. In particular, the llvm- backend.cpp file contains the top-level code that sets up the optimization pipeline and the code generator, and is the one that runs it at various times. -Chris