Hi, Today I found a nice bug in the JIT global variable emitter. The problem may lead to an assert() failure when doing the following: 1) compile some function 2) emit a global variable 3) compile another function. an assert() may trigger in the JIT memory manager This happens because the JIT global variable emitter is using the MachineCodeEmitter::allocate() function, which uses memory allocated by the JIT memory manager (which should be used for functions only). The assert() is triggered because the global variable is dumped to the header of the free blocks list (provided that ThisAllocated becomes 1). My proposed patch: http://web.ist.utl.pt/nuno.lopes/llvm_jit_global_emitter.txt (similar to what other JIT variable emitters do). Please comment. Thanks, Nuno
On Oct 13, 2008, at 2:42 PMPDT, Nuno Lopes wrote:> Hi, > > Today I found a nice bug in the JIT global variable emitter. > The problem may lead to an assert() failure when doing the following: > 1) compile some function > 2) emit a global variable > 3) compile another function. an assert() may trigger in the JIT memory > manager > > This happens because the JIT global variable emitter is using the > MachineCodeEmitter::allocate() function, which uses memory allocated > by the > JIT memory manager (which should be used for functions only).No, this was a deliberate change, 54442. We have a situation where a user wants to emit JIT code on one machine, then send it off to another machine to execute. Putting statically allocated data in the same buffer as code is the easiest approach to make this work, although there may be others. Is this the known situation that the JIT memory manager doesn't handle buffer overflow well? You could get into problems with that when it contained only code, although it obviously gets hit more often now.> The assert() > is triggered because the global variable is dumped to the header of > the free > blocks list (provided that ThisAllocated becomes 1). > > My proposed patch: > http://web.ist.utl.pt/nuno.lopes/llvm_jit_global_emitter.txt > (similar to > what other JIT variable emitters do). > > Please comment. > > Thanks, > Nuno > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>> Today I found a nice bug in the JIT global variable emitter. >> The problem may lead to an assert() failure when doing the following: >> 1) compile some function >> 2) emit a global variable >> 3) compile another function. an assert() may trigger in the JIT memory >> manager >> >> This happens because the JIT global variable emitter is using the >> MachineCodeEmitter::allocate() function, which uses memory allocated >> by the >> JIT memory manager (which should be used for functions only). > > No, this was a deliberate change, 54442. We have a situation where a > user > wants to emit JIT code on one machine, then send it off to another > machine to > execute. Putting statically allocated data in the same buffer as code > is the > easiest approach to make this work, although there may be others.Ok, thanks for the explanation. So my first patch doesn't work. Also, to be clear, this bug has nothing to do with overflowing the JIT memory buffer. I made another one that takes keeps the allocation of global variables in the JIT buffer, but it creates a new mem block if it doesn't exist (i.e. when dumping a global variable out of the scope of a function compilation). The patch is at: http://web.ist.utl.pt/nuno.lopes/llvm_jit_global_emitter2.txt The problem only happens when calling ExecutionEngine::getPointerToGlobal(someGV) from some non-llvm program. If the function is called when JITing a function, it works, since it will dump the global variable to the memory reserved to the function being JITed, which raises a question: shouldn't it generate the GV to some non-executable memory block?? My patch doesn't attempt to fix this last concern. Thanks, Nuno P.S.: the control flow of this bug is quite complex, so feel free to ask if you don't get what the problem is.
[resending since the previous copy was apparently dropped by the mailing list]>> Today I found a nice bug in the JIT global variable emitter. >> The problem may lead to an assert() failure when doing the following: >> 1) compile some function >> 2) emit a global variable >> 3) compile another function. an assert() may trigger in the JIT memory >> manager >> >> This happens because the JIT global variable emitter is using the >> MachineCodeEmitter::allocate() function, which uses memory allocated >> by the >> JIT memory manager (which should be used for functions only). > > No, this was a deliberate change, 54442. We have a situation where a > user > wants to emit JIT code on one machine, then send it off to another > machine to > execute. Putting statically allocated data in the same buffer as code > is the > easiest approach to make this work, although there may be others.Ok, thanks for the explanation. So my first patch doesn't work. Also, to be clear, this bug has nothing to do with overflowing the JIT memory buffer. I made another one that takes keeps the allocation of global variables in the JIT buffer, but it creates a new mem block if it doesn't exist (i.e. when dumping a global variable out of the scope of a function compilation). The patch is at: http://web.ist.utl.pt/nuno.lopes/llvm_jit_global_emitter2.txt The problem only happens when calling ExecutionEngine::getPointerToGlobal(someGV) from some non-llvm program. If the function is called when JITing a function, it works, since it will dump the global variable to the memory reserved to the function being JITed, which raises a question: shouldn't it generate the GV to some non-executable memory block?? My patch doesn't attempt to fix this last concern. Thanks, Nuno P.S.: the control flow of this bug is quite complex, so feel free to ask if you don't get what the problem is.
> Putting statically allocated data in the same buffer as code > is the > easiest approach to make this work, although there may be others. >That's likely to result is trouble on platforms where the code pages are not writable, though, right?