>> Today I found a nice bug in the JIT global variable emitter. >> The problem may lead to an assert() failure when doing the following: >> 1) compile some function >> 2) emit a global variable >> 3) compile another function. an assert() may trigger in the JIT memory >> manager >> >> This happens because the JIT global variable emitter is using the >> MachineCodeEmitter::allocate() function, which uses memory allocated >> by the >> JIT memory manager (which should be used for functions only). > > No, this was a deliberate change, 54442. We have a situation where a > user > wants to emit JIT code on one machine, then send it off to another > machine to > execute. Putting statically allocated data in the same buffer as code > is the > easiest approach to make this work, although there may be others.Ok, thanks for the explanation. So my first patch doesn't work. Also, to be clear, this bug has nothing to do with overflowing the JIT memory buffer. I made another one that takes keeps the allocation of global variables in the JIT buffer, but it creates a new mem block if it doesn't exist (i.e. when dumping a global variable out of the scope of a function compilation). The patch is at: http://web.ist.utl.pt/nuno.lopes/llvm_jit_global_emitter2.txt The problem only happens when calling ExecutionEngine::getPointerToGlobal(someGV) from some non-llvm program. If the function is called when JITing a function, it works, since it will dump the global variable to the memory reserved to the function being JITed, which raises a question: shouldn't it generate the GV to some non-executable memory block?? My patch doesn't attempt to fix this last concern. Thanks, Nuno P.S.: the control flow of this bug is quite complex, so feel free to ask if you don't get what the problem is.
On Oct 14, 2008, at 4:47 AMPDT, Nuno Lopes wrote:>>> Today I found a nice bug in the JIT global variable emitter. >>> The problem may lead to an assert() failure when doing the >>> following: >>> 1) compile some function >>> 2) emit a global variable >>> 3) compile another function. an assert() may trigger in the JIT >>> memory >>> manager >>> >>> This happens because the JIT global variable emitter is using the >>> MachineCodeEmitter::allocate() function, which uses memory allocated >>> by the >>> JIT memory manager (which should be used for functions only). >> >> No, this was a deliberate change, 54442. We have a situation where a >> user >> wants to emit JIT code on one machine, then send it off to another >> machine to >> execute. Putting statically allocated data in the same buffer as >> code >> is the >> easiest approach to make this work, although there may be others. > > Ok, thanks for the explanation. So my first patch doesn't work. > Also, to be > clear, this bug has nothing to do with overflowing the JIT memory > buffer. > I made another one that takes keeps the allocation of global > variables in > the JIT buffer, but it creates a new mem block if it doesn't exist > (i.e. > when dumping a global variable out of the scope of a function > compilation). > The patch is at: > http://web.ist.utl.pt/nuno.lopes/llvm_jit_global_emitter2.txtOK, I think I finally understand this: the allocation in JIT::getOrEmitGlobalVariable doesn't call into JITEmitter but uses the MachineCodeEmitter directly. Thus, if you call it from outside the JITEmitter, it won't update the data structures the JITEmitter relies on. In the usual case, the JITEmitter calls into JIT and updates its own data structures, so that case works fine. I think adding a smarter allocateSpace to JITEmitter is the right general idea, and your patch looks OK. I'd like to get another pair of eyes on it though, as I find this area not straightfoward:) Please run the testsuite before committing. Thanks for analyzing this and the test case.> The problem only happens when calling > ExecutionEngine::getPointerToGlobal(someGV) from some non-llvm > program. If > the function is called when JITing a function, it works, since it > will dump > the global variable to the memory reserved to the function being > JITed, > which raises a question: shouldn't it generate the GV to some non- > executable > memory block?? My patch doesn't attempt to fix this last concern. > > Thanks, > Nuno > > P.S.: the control flow of this bug is quite complex, so feel free to > ask if > you don't get what the problem is. > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>> Ok, thanks for the explanation. So my first patch doesn't work. >> Also, to be >> clear, this bug has nothing to do with overflowing the JIT memory >> buffer. >> I made another one that takes keeps the allocation of global >> variables in >> the JIT buffer, but it creates a new mem block if it doesn't exist >> (i.e. >> when dumping a global variable out of the scope of a function >> compilation). >> The patch is at: >> http://web.ist.utl.pt/nuno.lopes/llvm_jit_global_emitter2.txt > > OK, I think I finally understand this: the allocation in > JIT::getOrEmitGlobalVariable doesn't call into JITEmitter but uses the > MachineCodeEmitter directly. Thus, if you call it from outside the > JITEmitter, it won't update the data structures the JITEmitter relies > on. In the usual case, the JITEmitter calls into JIT and updates its > own data structures, so that case works fine.exactly! :) Thank you for summarizing. I hope others can now follow this description.> I think adding a smarter allocateSpace to JITEmitter is the right > general idea, and your patch looks OK. I'd like to get another pair > of eyes on it though, as I find this area not straightfoward:) Please > run the testsuite before committing.Yes, I would also appreciate more reviewers. The JIT's code is particularly complex and I'm also quite new to the LLVM code.. Thanks, Nuno