Perry E. Metzger via llvm-dev
2017-May-07 13:43 UTC
[llvm-dev] Email list just for front end developers?
On Sat, 6 May 2017 21:38:50 -0700 Sanjoy Das <sanjoy at playingwithpointers.com> wrote:> Hi Perry, > > On Sat, May 6, 2017 at 2:55 PM, Perry E. Metzger > <perry at piermont.com> wrote: > >> Let me put it this way -- I'm pretty sure if you take it on > >> yourself to answer beginner questions on llvm-dev promptly, > >> nobody will *complain*. In fact, you're going to make some > >> folks happy. :) > > > > Sure, but who will answer mine? Part of the reason I asked about > > this was selfish, as I'd like access to other people in the same > > situation as me who might know answers. (I don't think the people > > here now will want to answer more questions, and many of my > > questions doubtless border on inanely stupid but for the fact > > that it is hard to find answers in the docs.) > > I don't see how people who don't have time to answer on llvm-dev > will find time to answer on some other mailing list.My belief was they would be different populations. That is why there would be a separate list. The other list would attract mostly front end devs. I would expect front end devs would be both more interested in a list that discusses mostly things they care about (including IR changes they need to be aware of) and more interested in mutual assistance. But never mind, it seems that I'm the only person who likes that idea, so we're ignoring that possibility for the moment.> > That said, I'm willing to give it a try, as an experiment. How > > long should we attempt it for? If that's reasonably agreed to, > > I'll > > Depends on how much time you have -- that's the limiting resource > here.By time I meant the question of how long one tries a strategy before deciding to look for new strategies. :) Given that few questions about front end things arrive on the -dev list, trying to help answering them, for the moment, will only require that I attend more closely to the postings, and that other people make a bit of an effort to answer "dumb questions" as well. Hopefully that will simply work. My skepticism comes from the feeling that if it was really going to work, it would have already, but there's no reason not to give it a good faith try. On the wider issue: LLVM has been transformative. There are now a lot more people experimenting with compiling languages than ever before (including me!) because LLVM removes a lot of the pain from constructing everything past the IR. However, until I started working with it personally, I didn't understand how much work it was to learn how to use it, and much of that work is because there is a very big gap between the programmers guide + the tutorial (which makes it all look like you can learn the whole thing in an afternoon) and what you actually need to figure out to get things done. Much of what you need you have to puzzle out painstakingly on your own, trying to make sense of the doxygen docs and reading code, because there's less written down than one would hope. Even the IR language reference, which is seemingly complete at first glance, has some big gaps in it. Given good enough resources, though, even more people could be using LLVM to create compilers, which would be a great thing. And, selfishly, given better resources, my own work would be easier. :) Thus my overall interest.> However, I liked your idea of maintaining a wiki-like thing. Both > http://llvm.org/docs/Frontend/PerformanceTips.html and > http://llvm.org/docs/tutorial/ will be good additions there.Parts of the programmer's manual http://llvm.org/docs/ProgrammersManual.html are also essential, though for a front end developer, especially one using the C API, not everything is something you want to know, and it isn't initially obvious when you need to read closely. Anyway, given that you would prefer for it not to be an actual wiki, what would be lower barrier to entry and Wiki-ish? (A big issue with the current workflow, I think, is that for outsiders submitting documentation updates the process is very heavyweight. Even the use of svn vs. git increases the amount of work for casual users. It's just not worth it as it stands for someone to say "hey, please fix just this one sentence", especially if you've never contributed to LLVM before. However, often even single sentence edits improve comprehension. Having a low barrier for small fixes is one of the reasons Wikipedia grew so fast. Of course, it is also the reason that policing Wikipedia takes huge effort, so there's a tradeoff.) Perry -- Perry E. Metzger perry at piermont.com
Dean Michael Berris via llvm-dev
2017-May-11 10:14 UTC
[llvm-dev] Email list just for front end developers?
FWIW, I think the "modern" way of doing any of this is with any one of the following services (that don't need to be owned/maintained by the LLVM Foundation): - Stack Overflow (maybe if it gets large enough, get a "vertical" for just LLVM usage) - Reddit (if there isn't an /r/llvm yet, that might be a good "community" place for less llvm-dev worthy material) - Slack (if there isn't one yet, maybe an llvm-users slack instance for more persistent discussions) I suspect none of this has to get the blessing of the llvm-dev@ members nor the foundation, and if there indeed are a lot of people that might benefit from these resources, then you would have started that community for the users. Hope this helps. Cheers> On 7 May 2017, at 23:43, Perry E. Metzger via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > On Sat, 6 May 2017 21:38:50 -0700 Sanjoy Das > <sanjoy at playingwithpointers.com <mailto:sanjoy at playingwithpointers.com>> wrote: >> Hi Perry, >> >> On Sat, May 6, 2017 at 2:55 PM, Perry E. Metzger >> <perry at piermont.com> wrote: >>>> Let me put it this way -- I'm pretty sure if you take it on >>>> yourself to answer beginner questions on llvm-dev promptly, >>>> nobody will *complain*. In fact, you're going to make some >>>> folks happy. :) >>> >>> Sure, but who will answer mine? Part of the reason I asked about >>> this was selfish, as I'd like access to other people in the same >>> situation as me who might know answers. (I don't think the people >>> here now will want to answer more questions, and many of my >>> questions doubtless border on inanely stupid but for the fact >>> that it is hard to find answers in the docs.) >> >> I don't see how people who don't have time to answer on llvm-dev >> will find time to answer on some other mailing list. > > My belief was they would be different populations. That is why there > would be a separate list. The other list would attract mostly front > end devs. I would expect front end devs would be both more interested > in a list that discusses mostly things they care about (including IR > changes they need to be aware of) and more interested in mutual > assistance. > > But never mind, it seems that I'm the only person who likes that idea, > so we're ignoring that possibility for the moment. > >>> That said, I'm willing to give it a try, as an experiment. How >>> long should we attempt it for? If that's reasonably agreed to, >>> I'll >> >> Depends on how much time you have -- that's the limiting resource >> here. > > By time I meant the question of how long one tries a strategy before > deciding to look for new strategies. :) Given that few questions about > front end things arrive on the -dev list, trying to help answering > them, for the moment, will only require that I attend more closely to > the postings, and that other people make a bit of an effort to answer > "dumb questions" as well. > > Hopefully that will simply work. My skepticism comes from the feeling > that if it was really going to work, it would have already, but > there's no reason not to give it a good faith try. > > On the wider issue: > > LLVM has been transformative. There are now a lot more people > experimenting with compiling languages than ever before (including > me!) because LLVM removes a lot of the pain from constructing > everything past the IR. > > However, until I started working with it personally, I didn't > understand how much work it was to learn how to use it, and much of > that work is because there is a very big gap between the programmers > guide + the tutorial (which makes it all look like you can learn the > whole thing in an afternoon) and what you actually need to figure out > to get things done. Much of what you need you have to puzzle out > painstakingly on your own, trying to make sense of the doxygen docs > and reading code, because there's less written down than one would > hope. Even the IR language reference, which is seemingly complete at > first glance, has some big gaps in it. > > Given good enough resources, though, even more people could be using > LLVM to create compilers, which would be a great thing. And, > selfishly, given better resources, my own work would be easier. :) > Thus my overall interest. > >> However, I liked your idea of maintaining a wiki-like thing. Both >> http://llvm.org/docs/Frontend/PerformanceTips.html <http://llvm.org/docs/Frontend/PerformanceTips.html> and >> http://llvm.org/docs/tutorial/ <http://llvm.org/docs/tutorial/> will be good additions there. > > Parts of the programmer's manual > http://llvm.org/docs/ProgrammersManual.html <http://llvm.org/docs/ProgrammersManual.html> are also essential, though > for a front end developer, especially one using the C API, not > everything is something you want to know, and it isn't initially > obvious when you need to read closely. > > Anyway, given that you would prefer for it not to be an actual wiki, > what would be lower barrier to entry and Wiki-ish? > > (A big issue with the current workflow, I think, is that for outsiders > submitting documentation updates the process is very heavyweight. Even > the use of svn vs. git increases the amount of work for casual users. > It's just not worth it as it stands for someone to say "hey, please > fix just this one sentence", especially if you've never contributed to > LLVM before. However, often even single sentence edits improve > comprehension. Having a low barrier for small fixes is one of the > reasons Wikipedia grew so fast. Of course, it is also the reason that > policing Wikipedia takes huge effort, so there's a tradeoff.) > > > Perry > -- > Perry E. Metzger perry at piermont.com <mailto:perry at piermont.com> > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>-- Dean -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170511/3c2e4e5b/attachment-0001.html>
Sean Silva via llvm-dev
2017-May-11 22:51 UTC
[llvm-dev] Email list just for front end developers?
On Sun, May 7, 2017 at 6:43 AM, Perry E. Metzger via llvm-dev < llvm-dev at lists.llvm.org> wrote:> On Sat, 6 May 2017 21:38:50 -0700 Sanjoy Das > <sanjoy at playingwithpointers.com> wrote: > > Hi Perry, > > > > On Sat, May 6, 2017 at 2:55 PM, Perry E. Metzger > > <perry at piermont.com> wrote: > > >> Let me put it this way -- I'm pretty sure if you take it on > > >> yourself to answer beginner questions on llvm-dev promptly, > > >> nobody will *complain*. In fact, you're going to make some > > >> folks happy. :) > > > > > > Sure, but who will answer mine? Part of the reason I asked about > > > this was selfish, as I'd like access to other people in the same > > > situation as me who might know answers. (I don't think the people > > > here now will want to answer more questions, and many of my > > > questions doubtless border on inanely stupid but for the fact > > > that it is hard to find answers in the docs.) > > > > I don't see how people who don't have time to answer on llvm-dev > > will find time to answer on some other mailing list. > > My belief was they would be different populations. That is why there > would be a separate list. The other list would attract mostly front > end devs. I would expect front end devs would be both more interested > in a list that discusses mostly things they care about (including IR > changes they need to be aware of) and more interested in mutual > assistance. > > But never mind, it seems that I'm the only person who likes that idea, > so we're ignoring that possibility for the moment. > > > > That said, I'm willing to give it a try, as an experiment. How > > > long should we attempt it for? If that's reasonably agreed to, > > > I'll > > > > Depends on how much time you have -- that's the limiting resource > > here. > > By time I meant the question of how long one tries a strategy before > deciding to look for new strategies. :) Given that few questions about > front end things arrive on the -dev list, trying to help answering > them, for the moment, will only require that I attend more closely to > the postings, and that other people make a bit of an effort to answer > "dumb questions" as well. > > Hopefully that will simply work. My skepticism comes from the feeling > that if it was really going to work, it would have already, but > there's no reason not to give it a good faith try. > > On the wider issue: > > LLVM has been transformative. There are now a lot more people > experimenting with compiling languages than ever before (including > me!) because LLVM removes a lot of the pain from constructing > everything past the IR. > > However, until I started working with it personally, I didn't > understand how much work it was to learn how to use it, and much of > that work is because there is a very big gap between the programmers > guide + the tutorial (which makes it all look like you can learn the > whole thing in an afternoon) and what you actually need to figure out > to get things done. Much of what you need you have to puzzle out > painstakingly on your own, trying to make sense of the doxygen docs > and reading code, because there's less written down than one would > hope. Even the IR language reference, which is seemingly complete at > first glance, has some big gaps in it. > > Given good enough resources, though, even more people could be using > LLVM to create compilers, which would be a great thing. And, > selfishly, given better resources, my own work would be easier. :) > Thus my overall interest. >I'm still unclear about what sorts of questions/discussion this audience is interested in. Can you provide a few examples of questions that might be asked, or links to discussions/blog posts on the net that exemplify the type of thing that you would want to happen in the "frontend authors using LLVM" medium you are thinking of? For example, looking at the post you linked ( http://lists.llvm.org/pipermail/llvm-dev/2017-March/111526.html the author of which is in this thread I think, hi!), there's not much LLVM-specific about it. In order to generate LLVM IR, it is necessary to understand its semantics. The semantics are mostly C-like, and so the answer to questions like those in the post: - How to assign a text to a string variable? - How to access for example the 4th element in a heap-allocated array? - How to create local variables which are not part of the global scope? - How to create complex structures like classes and objects? is basically "how would you do it in C"? For example, to assign text to a "string variable", you first have to have a struct that represents you string type, then you do some C-level code to manipulate that data structure. For a class with inheritance, you would put a vptr in your structs and emit vtables. Etc. Then it's just a matter of writing some C code and running it through Clang to see how Clang lowers it to LLVM IR. It shouldn't be too hard to then trace back from the textual IR to the API's (at worst, requiring a small amount of trial and error, at worst looking at small parts of Clang's source code after grepping for AllocaInst or whatever). Something like https://f0rki.gitbooks.io/mapping-high-level-constructs-to-llvm-ir/content/ might be useful too, though you could get mostly the same effect by looking at Clang's output. For some of the specifics of LLVM IR's SSA form (like phi nodes) working through the Kaleidoscope tutorial should give a feel for how to do it. (for an intro, a video like https://www.youtube.com/watch?v=haQ2cijhvhE by our own Chandler Carruth might be useful). However, from your other posts, you say something like: "People working on front ends typically don't really know everything about the innards of LLVM, and thanks to LLVM's very nicely designed architecture, we don't need to, we mostly need to know how to generate IR and hook up to the rest of the system." So it sounds like you're more interested in API usage like IRBuilder, setting up codegen, setting up MC, configuring pass pipelines, etc.? I.e. you have a clear idea about the "big picture" and what you are trying to do, but want to know how that is done in LLVM (e.g. "how do I create a phi node?"). This is a very different kind of question from the ones above. On the other hand, "hook it up to the rest of the system" might be referring to general toolchain knowledge (which is again not LLVM specifc). For example object file semantics (i.e. the semantics of what comes out the LLVM backends; there's a lot more to it than just a single blob of machine code) and how that interacts with the rest of the system (e.g. how a malloc declaration in LLVM IR gets actually resolved and called at runtime). I'd really like to hear your feedback. I think that historically, Clang has been the dominant frontend and so there hasn't been much impetus for providing certain kinds of documention, but as you mention upthread, there is now a quite large tail of other frontends that are developed outside of the LLVM community, so that might have to change. -- Sean Silva> > > However, I liked your idea of maintaining a wiki-like thing. Both > > http://llvm.org/docs/Frontend/PerformanceTips.html and > > http://llvm.org/docs/tutorial/ will be good additions there. > > Parts of the programmer's manual > http://llvm.org/docs/ProgrammersManual.html are also essential, though > for a front end developer, especially one using the C API, not > everything is something you want to know, and it isn't initially > obvious when you need to read closely. > > Anyway, given that you would prefer for it not to be an actual wiki, > what would be lower barrier to entry and Wiki-ish? > > (A big issue with the current workflow, I think, is that for outsiders > submitting documentation updates the process is very heavyweight. Even > the use of svn vs. git increases the amount of work for casual users. > It's just not worth it as it stands for someone to say "hey, please > fix just this one sentence", especially if you've never contributed to > LLVM before. However, often even single sentence edits improve > comprehension. Having a low barrier for small fixes is one of the > reasons Wikipedia grew so fast. Of course, it is also the reason that > policing Wikipedia takes huge effort, so there's a tradeoff.) > > > Perry > -- > Perry E. Metzger perry at piermont.com > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170511/1d75f363/attachment-0001.html>
Perry E. Metzger via llvm-dev
2017-May-12 02:48 UTC
[llvm-dev] Email list just for front end developers?
So in writing this, I want to emphasize, in advance, that I'm grateful to the LLVM team for having created LLVM, I do not feel I am in any way entitled to anyone else's help, etc. The purpose of my original post was just to suggest that there be a way for people who are struggling with some parts of the APIs to find each other and answer each other's questions. If any of the following sounds like I'm demanding anything, that's not my intent. This may be longer than it should be, too. On Thu, 11 May 2017 15:51:14 -0700 Sean Silva <chisophugis at gmail.com> wrote:> I'm still unclear about what sorts of questions/discussion this > audience is interested in. Can you provide a few examples of > questions that might be asked, or links to discussions/blog posts > on the net that exemplify the type of thing that you would want to > happen in the "frontend authors using LLVM" medium you are thinking > of? > > For example, looking at the post you linked ( > http://lists.llvm.org/pipermail/llvm-dev/2017-March/111526.html the > author of which is in this thread I think, hi!), there's not much > LLVM-specific about it. In order to generate LLVM IR, it is > necessary to understand its semantics. The semantics are mostly > C-like, and so the answer to questions like those in the post: > > - How to assign a text to a string variable? > - How to access for example the 4th element in a heap-allocated > array? > - How to create local variables which are not part of the global > scope? > - How to create complex structures like classes and objects? > > is basically "how would you do it in C"?I think you're missing the import of his questions. This isn't "how you would do it in C". He's asking effectively how to use the IRBuilder APIs to do these things, not what in principle you want the machine to do or what you might write into a file of text form IR if you were bypassing the IRBuilder. The "how do I construct a string constant in the API" one cost me hours to figure out until someone on IRC took pity on me, which was unusual, and steered me to the right call (CreateGlobalString). Now I know what you're thinking. "Why didn't you just look for something like CreateGlobalString to begin with?" And the answer is, because I had no idea what I was looking for, what form it would take, or what it was called, and googling at random wasn't helping. I knew what sort of IR output I wanted but I didn't know how to generate it in the API. And yes, a naive newcomer can literally spend hours trying to figure out that just one call they wanted was CreateGlobalString. Maybe I missed the part of the extant documentation or the tutorial that pointed there, but that's kind of the point, isn't it? If it was there, I didn't find it, and I needed to ask for help. It would have been nice to have a place where I could have dropped the question and no one would have thought twice about just answering. And thus, my desire for a place for mutual aid for people who are figuring out front end stuff or who have already figured it out and are willing to help others.> For example, to assign text to a "string variable", you first have > to have a struct that represents you string type, then you do some > C-level code to manipulate that data structure.That's not what he's asking, or at least I don't think he is. He's asking about how to generate the right IR. He probably was looking for CreateGlobalString and how to put it together with other stuff. His other messages made that seem clear to someone else who had been struggling recently with the same issues,.> For some of the specifics of LLVM IR's SSA form (like phi nodes) > working through the Kaleidoscope tutorial should give a feel for > how to do it.A feel isn't enough to write code that compiles and does the right stuff. It gives you a tantalizing hint of what you want, but you need more than that. The tutorial makes it all look really easy, but it skips even a lot of details about what it is doing, and then you start to ask questions that go slightly past the edge of the tutorial and the answers _really_ aren't obvious. Here's just one a of a million examples: in a real compiler you don't want to just parse a number into a machine int or double or what have you, you want to use arbitrary precision stuff that's machine independent, and of course, LLVM has that because it needs it. Unluckily, even once you know it is there, it takes a while to figure out what you need to use it, and there's very little documentation at all. There was a point where I wanted to just punt and let the APFloat API parse a string from my front end's scanner (after all, why duplicate that code and less well?), only there's no documentation on what sorts of float formats it parses. If someone could have answered that, or if there had been a paragraph or two in the doxygen output, it would have saved me lots of time reading code. As it was, the best someone had to suggest to me was read the tests, so I went off and slogged through. (And yah, it turns out that it handles a variety of standard C-like formats including hex floats, which was something I wanted to figure out but couldn't just read in the doxygen docs because it isn't there.) So yah, you get a feel for what you want from the tutorial, but a feel isn't enough to write a compiler for something non-trivial. The programmer's guide helps a bit more, but you're quickly off the road and feeling stuff out. Note again that I'm not in any way suggesting that anyone should feel obligated to help others. No one is! I have no claim on anyone else's time, and again, thank all of you for creating LLVM in the first place. All I was asking for was just a place for those who are willing to help each other over the hump to contact each other. Puzzling things out by reading the test cases for a piece of compiler middle-end machinery isn't the most fun way to learn an API, and it is slow, too.> However, from your other posts, you say something like: > "People working on front ends typically don't really know everything > about the innards of LLVM, and thanks to LLVM's very nicely designed > architecture, we don't need to, we mostly need to know how to > generate IR and hook up to the rest of the system." > > So it sounds like you're more interested in API usage like > IRBuilder, setting up codegen, setting up MC, configuring pass > pipelines, etc.? I.e. you have a clear idea about the "big picture" > and what you are trying to do, but want to know how that is done in > LLVM (e.g. "how do I create a phi node?"). This is a very different > kind of question from the ones above.The questions that guy was asking seem to be part and parcel, though I could be wrong. There's a big IR Builder API, the C bindings for all this stuff are poorly documented, the C++ bindings are only slightly better documented, and dealing with the important bits is not trivial. The language reference superficially looks quite nice, but then you realize the only way to actually figure out what IR you need for many constructs is to write C code and get Clang to dump out the IR that results. (It took me a while to learn that trick, even though, yes, it should have been obvious, just like finding CreateGlobalString should have been obvious only I didn't know I was looking for it.) However, even once having seen what Clang generates, you still need to figure out what to put into IRBuilder to get the same results. Sure, that's probably somewhere in the innards of Clang, but try finding it. It's not trivial, and without much documentation it takes a while to figure out. I really would have like to have just asked someone about a bunch of stuff instead of deciphering things. Also take his second question about accessing parts of arrays. getelementptr is sufficiently weird that there actually _is_ a decent bit of documentation on it, but it isn't obvious that you're looking for that unless someone tells you. (Luckily I stumbled on it.) I get that this all seems really easy if you know it already. To someone who is coming at this completely naively, without much understanding of where to dig in, there's a very, very steep learning curve and little documentation once you're done with the tutorial. A lot has to be found in blog postings, bits of sample code you happen to find, reading the implementation, etc. And I note that even though I'm still at it, I don't yet have my front end doing very much. There's still a lot to learn. And yah, maybe I'm not the best at this. Maybe if I was smarter I'd just pick it up from reading Clang code or something, but I'm slow and middle aged and yet I have a job to get done anyway. So I keep beating my head against stuff until I figure it out. But I still wish there were people to ask questions.> I'd really like to hear your feedback. I think that historically, > Clang has been the dominant frontend and so there hasn't been much > impetus for providing certain kinds of documention, but as you > mention upthread, there is now a quite large tail of other > frontends that are developed outside of the LLVM community, so that > might have to change.If I had a magic wand, I'd have everything well documented, but that seems like it isn't going to happen. As a more practical second best, all I really want is to be in contact with other people who work on front ends who are sympathetic and remember learning the same thing. Perry -- Perry E. Metzger perry at piermont.com
Perry E. Metzger via llvm-dev
2017-May-12 03:03 UTC
[llvm-dev] Email list just for front end developers?
On Thu, 11 May 2017 20:14:11 +1000 Dean Michael Berris via llvm-dev <llvm-dev at lists.llvm.org> wrote:> FWIW, I think the "modern" way of doing any of this is with any one > of the following services (that don't need to be owned/maintained > by the LLVM Foundation): > > - Stack Overflow (maybe if it gets large enough, get a "vertical" > for just LLVM usage)Unfortunately, stack overflow works best for questions where there are a lot of people who know the answers and people are willing to actively log in day after day to answer questions. It works great for "how do I use SQL to do something" but less well for small niches. The LLVM questions I've seen posed there seem to usually have zero or one answers rather than the large number common things get.> - Reddit (if there isn't an /r/llvm yet, that might be a good > "community" place for less llvm-dev worthy material)Maybe. I like mailing lists because you don't have to poll them. Having to log in constantly to a service to see if there is discussion waiting is a bit of a pain. (I have to admit I've never been much of a reddit fan in general.)> - Slack (if there isn't one yet, maybe an llvm-users slack > instance for more persistent discussions)There is an IRC channel already, though my experience with it as a help mechanism has been mixed as it seems more geared towards the middle and back end. And even if people decided this was a good idea, without the LLVM foundation documenting it somewhere, no one will find it. All that said, I might be wrong and these might be the better path. Perry -- Perry E. Metzger perry at piermont.com