[resending since the previous copy was apparently dropped by the mailing list]>> Today I found a nice bug in the JIT global variable emitter. >> The problem may lead to an assert() failure when doing the following: >> 1) compile some function >> 2) emit a global variable >> 3) compile another function. an assert() may trigger in the JIT memory >> manager >> >> This happens because the JIT global variable emitter is using the >> MachineCodeEmitter::allocate() function, which uses memory allocated >> by the >> JIT memory manager (which should be used for functions only). > > No, this was a deliberate change, 54442. We have a situation where a > user > wants to emit JIT code on one machine, then send it off to another > machine to > execute. Putting statically allocated data in the same buffer as code > is the > easiest approach to make this work, although there may be others.Ok, thanks for the explanation. So my first patch doesn't work. Also, to be clear, this bug has nothing to do with overflowing the JIT memory buffer. I made another one that takes keeps the allocation of global variables in the JIT buffer, but it creates a new mem block if it doesn't exist (i.e. when dumping a global variable out of the scope of a function compilation). The patch is at: web.ist.utl.pt/nuno.lopes/llvm_jit_global_emitter2.txt The problem only happens when calling ExecutionEngine::getPointerToGlobal(someGV) from some non-llvm program. If the function is called when JITing a function, it works, since it will dump the global variable to the memory reserved to the function being JITed, which raises a question: shouldn't it generate the GV to some non-executable memory block?? My patch doesn't attempt to fix this last concern. Thanks, Nuno P.S.: the control flow of this bug is quite complex, so feel free to ask if you don't get what the problem is.
On Oct 14, 2008, at 2:32 PM, Nuno Lopes wrote:> [resending since the previous copy was apparently dropped by the > mailing > list] > >>> Today I found a nice bug in the JIT global variable emitter. >>> The problem may lead to an assert() failure when doing the >>> following: >>> 1) compile some function >>> 2) emit a global variable >>> 3) compile another function. an assert() may trigger in the JIT >>> memory >>> manager >>> >>> This happens because the JIT global variable emitter is using the >>> MachineCodeEmitter::allocate() function, which uses memory allocated >>> by the >>> JIT memory manager (which should be used for functions only). >> >> No, this was a deliberate change, 54442. We have a situation where a >> user >> wants to emit JIT code on one machine, then send it off to another >> machine to >> execute. Putting statically allocated data in the same buffer as >> code >> is the >> easiest approach to make this work, although there may be others. > > Ok, thanks for the explanation. So my first patch doesn't work. > Also, to be > clear, this bug has nothing to do with overflowing the JIT memory > buffer. > I made another one that takes keeps the allocation of global > variables in > the JIT buffer, but it creates a new mem block if it doesn't exist > (i.e. > when dumping a global variable out of the scope of a function > compilation). > The patch is at: > web.ist.utl.pt/nuno.lopes/llvm_jit_global_emitter2.txtSorry, I am still not able to understand the problem. Is there a bug in the default memory manager? From your patch it seems like there is a real bug. What is the assertion that you ran into?> > > The problem only happens when calling > ExecutionEngine::getPointerToGlobal(someGV) from some non-llvm > program. If > the function is called when JITing a function, it works, since it > will dump > the global variable to the memory reserved to the function being > JITed, > which raises a question: shouldn't it generate the GV to some non- > executable > memory block?? My patch doesn't attempt to fix this last concern.There are two ways to go about this. Either we enhance memory manager interface to create non-executable memory for GVs etc. Or we keep the memory manager simple and let the JIT change the privilege as it sees fit. I prefer the later unless there is a good argument for an alternative. Evan> > > Thanks, > Nuno > > P.S.: the control flow of this bug is quite complex, so feel free to > ask if > you don't get what the problem is. > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu llvm.cs.uiuc.edu > lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>> Today I found a nice bug in the JIT global variable emitter. >>>> The problem may lead to an assert() failure when doing the >>>> following: >>>> 1) compile some function >>>> 2) emit a global variable >>>> 3) compile another function. an assert() may trigger in the JIT >>>> memory >>>> manager >>>> >>>> This happens because the JIT global variable emitter is using the >>>> MachineCodeEmitter::allocate() function, which uses memory allocated >>>> by the >>>> JIT memory manager (which should be used for functions only). >>> >>> No, this was a deliberate change, 54442. We have a situation where a >>> user >>> wants to emit JIT code on one machine, then send it off to another >>> machine to >>> execute. Putting statically allocated data in the same buffer as >>> code >>> is the >>> easiest approach to make this work, although there may be others. >> >> Ok, thanks for the explanation. So my first patch doesn't work. >> Also, to be >> clear, this bug has nothing to do with overflowing the JIT memory >> buffer. >> I made another one that takes keeps the allocation of global >> variables in >> the JIT buffer, but it creates a new mem block if it doesn't exist >> (i.e. >> when dumping a global variable out of the scope of a function >> compilation). >> The patch is at: >> web.ist.utl.pt/nuno.lopes/llvm_jit_global_emitter2.txt > > Sorry, I am still not able to understand the problem. Is there a bug > in the default memory manager? From your patch it seems like there is > a real bug. What is the assertion that you ran into?This bug is a problem in the design of the JIT GV emitter, that appeared with the revision 54442, as Dale pointed. I was trying to avoid to make a test case, but I did one now. Please find it in attach (I hope the ML doesn't strip it). The source code has enough comments to explain what's going on. The module.bc is just a container to have 2 functions, but it's irrelevant to the problem. I just picked some random .bc file that had 2 functions.>> The problem only happens when calling >> ExecutionEngine::getPointerToGlobal(someGV) from some non-llvm >> program. If >> the function is called when JITing a function, it works, since it >> will dump >> the global variable to the memory reserved to the function being >> JITed, >> which raises a question: shouldn't it generate the GV to some non- >> executable >> memory block?? My patch doesn't attempt to fix this last concern. > > There are two ways to go about this. Either we enhance memory manager > interface to create non-executable memory for GVs etc. Or we keep the > memory manager simple and let the JIT change the privilege as it sees > fit. I prefer the later unless there is a good argument for an > alternative.I didn't check, but I think the JIT engine is mapping the whole function's memory as executable. As now most global variables end up being created when JITing functions, those GV end up in executable memory. Nuno -------------- next part -------------- A non-text attachment was scrubbed... Name: llvm-jit-memmanager.zip Type: application/x-zip-compressed Size: 1886 bytes Desc: not available URL: <lists.llvm.org/pipermail/llvm-dev/attachments/20081015/66bb60c8/attachment.bin>