I've tried using SourceMgr in a parser to get the nice error reporting with automatic line/column tracking, and it works well. I'm thinking about extending that to try to get the nice error reporting in subsequent type checking, and that leads to a question. As I understand it, SourceMgr owns the memory buffers (be they allocated chunks of memory, or memory mapped files) containing the source, the idea being that when you finish parsing and return, the SourceMgr destructor automatically frees the memory buffers, which makes sense. It doesn't seem to have anything like a .clear function, presumably because the whole idea is you are supposed to let the destructor automatically do it? But that would seem to indicate that the memory buffers are not going to be available once parsing is finished and you've moved on to things like type checking; the SourceMgr object will be out of scope and destructed. What's the intended way to deal with this? I suppose you could allocate SourceMgr on the heap and manually destruct it once type checking is finished, but that doesn't seem to be how the example code I've looked at does things? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150928/ea5e5a15/attachment.html>
> On Sep 27, 2015, at 11:07 PM, Russell Wallace via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > I've tried using SourceMgr in a parser to get the nice error reporting with automatic line/column tracking, and it works well. > > I'm thinking about extending that to try to get the nice error reporting in subsequent type checking, and that leads to a question. > > As I understand it, SourceMgr owns the memory buffers (be they allocated chunks of memory, or memory mapped files) containing the source, the idea being that when you finish parsing and return, the SourceMgr destructor automatically frees the memory buffers, which makes sense. It doesn't seem to have anything like a .clear function, presumably because the whole idea is you are supposed to let the destructor automatically do it? > > But that would seem to indicate that the memory buffers are not going to be available once parsing is finished and you've moved on to things like type checking; the SourceMgr object will be out of scope and destructed. What's the intended way to deal with this? I suppose you could allocate SourceMgr on the heap and manually destruct it once type checking is finished, but that doesn't seem to be how the example code I've looked at does things?Hi Russell, The intent of SourceMgr is that you keep it around for as long as diagnostics need to be produced. The file buffers it manages are generally all mmap’d in, so after doing the initial lex, the kernel is free to page them back out to disk if the machine comes under memory pressure. -Chris
Okay, that makes sense; I can do the type checking after parsing but before returning from the parsing phase. On Mon, Sep 28, 2015 at 5:43 PM, Chris Lattner <clattner at apple.com> wrote:> > > On Sep 27, 2015, at 11:07 PM, Russell Wallace via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > > > > I've tried using SourceMgr in a parser to get the nice error reporting > with automatic line/column tracking, and it works well. > > > > I'm thinking about extending that to try to get the nice error reporting > in subsequent type checking, and that leads to a question. > > > > As I understand it, SourceMgr owns the memory buffers (be they allocated > chunks of memory, or memory mapped files) containing the source, the idea > being that when you finish parsing and return, the SourceMgr destructor > automatically frees the memory buffers, which makes sense. It doesn't seem > to have anything like a .clear function, presumably because the whole idea > is you are supposed to let the destructor automatically do it? > > > > But that would seem to indicate that the memory buffers are not going to > be available once parsing is finished and you've moved on to things like > type checking; the SourceMgr object will be out of scope and destructed. > What's the intended way to deal with this? I suppose you could allocate > SourceMgr on the heap and manually destruct it once type checking is > finished, but that doesn't seem to be how the example code I've looked at > does things? > > Hi Russell, > > The intent of SourceMgr is that you keep it around for as long as > diagnostics need to be produced. The file buffers it manages are generally > all mmap’d in, so after doing the initial lex, the kernel is free to page > them back out to disk if the machine comes under memory pressure. > > -Chris > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150929/bd661e1b/attachment.html>
Do we madvise(DONTNEED) them? Is it a good idea? Needing to emit diagnostics is, after all, hopefully a rare case. OTOH, the madvise() call takes time and most machines have plenty of RAM, and source code is small. On Mon, Sep 28, 2015 at 7:43 PM, Chris Lattner via llvm-dev < llvm-dev at lists.llvm.org> wrote:> > > On Sep 27, 2015, at 11:07 PM, Russell Wallace via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > > > > I've tried using SourceMgr in a parser to get the nice error reporting > with automatic line/column tracking, and it works well. > > > > I'm thinking about extending that to try to get the nice error reporting > in subsequent type checking, and that leads to a question. > > > > As I understand it, SourceMgr owns the memory buffers (be they allocated > chunks of memory, or memory mapped files) containing the source, the idea > being that when you finish parsing and return, the SourceMgr destructor > automatically frees the memory buffers, which makes sense. It doesn't seem > to have anything like a .clear function, presumably because the whole idea > is you are supposed to let the destructor automatically do it? > > > > But that would seem to indicate that the memory buffers are not going to > be available once parsing is finished and you've moved on to things like > type checking; the SourceMgr object will be out of scope and destructed. > What's the intended way to deal with this? I suppose you could allocate > SourceMgr on the heap and manually destruct it once type checking is > finished, but that doesn't seem to be how the example code I've looked at > does things? > > Hi Russell, > > The intent of SourceMgr is that you keep it around for as long as > diagnostics need to be produced. The file buffers it manages are generally > all mmap’d in, so after doing the initial lex, the kernel is free to page > them back out to disk if the machine comes under memory pressure. > > -Chris > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150930/f1ae1c47/attachment.html>