Hi David, Following up on the problems we discussed yesterday on IRC regarding TLS with MCJIT, I've put together the attached experimental patch. This patch makes three changes: 1. SectionMemoryManager is changed to request memory below the 2GB boundary by default. 2. sys::Memory::allocateMappedMemory is changed to set the MAP_32BIT flag if the requested "near" block is below the 2GB boundary. 3. RuntimeDyldELF is changed to recognize the possibility of external data symbols. Of these changes, items 2 and 3 are probably reasonable things to commit into trunk, and depending on how this turns out I will do so. Item 1 is a bit heavy-handed as presented here, but it suggests the type of thing that subclasses of SectionMemoryManager could do to make this work. If we had a way to communicate the code model to the memory manager from RuntimeDyld/MCJIT (and we obviously should!) then SectionMemoryManager could do something like this when small or medium memory models are selected on applicable platforms. When I tried this patch with the test case you provided yesterday it got through the compilation phase with lli using the small code model and the static relocation model, but it ultimately failed (but failed gracefully) because it couldn't resolve the '_ThreadRuneLocale' symbol. Resolution of external symbols is meant to be handled by the memory manager, so I thought perhaps you could get something working with this patch. Please give this a try and let me know how it works. Thanks, Andy -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130508/17655ea1/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: tls-experimental.patch Type: application/octet-stream Size: 2471 bytes Desc: tls-experimental.patch URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130508/17655ea1/attachment.obj>
Hi, Unfortunately, I can't compile this patch. MAP_32BIT is a Linuxism that doesn't work on FreeBSD (or OS X, or, as far as I can tell, anywhere except Linux). We can consider adding something similar to FreeBSD (although I'm hesitant to encourage anything that increases the determinism of the memory layout of JITed code, for security reasons), but it doesn't seem ideal. David On 8 May 2013, at 16:54, "Kaylor, Andrew" <andrew.kaylor at intel.com> wrote:> Hi David, > > Following up on the problems we discussed yesterday on IRC regarding TLS with MCJIT, I’ve put together the attached experimental patch. > > This patch makes three changes: > > 1. SectionMemoryManager is changed to request memory below the 2GB boundary by default. > 2. sys::Memory::allocateMappedMemory is changed to set the MAP_32BIT flag if the requested “near” block is below the 2GB boundary. > 3. RuntimeDyldELF is changed to recognize the possibility of external data symbols. > > Of these changes, items 2 and 3 are probably reasonable things to commit into trunk, and depending on how this turns out I will do so. Item 1 is a bit heavy-handed as presented here, but it suggests the type of thing that subclasses of SectionMemoryManager could do to make this work. If we had a way to communicate the code model to the memory manager from RuntimeDyld/MCJIT (and we obviously should!) then SectionMemoryManager could do something like this when small or medium memory models are selected on applicable platforms. > > When I tried this patch with the test case you provided yesterday it got through the compilation phase with lli using the small code model and the static relocation model, but it ultimately failed (but failed gracefully) because it couldn’t resolve the ‘_ThreadRuneLocale’ symbol. Resolution of external symbols is meant to be handled by the memory manager, so I thought perhaps you could get something working with this patch. > > Please give this a try and let me know how it works. > > Thanks, > Andy > <tls-experimental.patch>
Can you try it without the MAP_32BIT part? It won't be as reliable, but if the memory addresses it is asking for are available it could work. I agree that there are good reasons not to lock in on a single memory address, but I'm curious as to what other obstacles might be lurking behind the ones we know about. If the patch works when memory is loaded below 2GB then it would be possible to right a sophisticated memory manager that surveys the available memory in that space and selects an appropriate block in some non-deterministic manner. -Andy -----Original Message----- From: Dr D. Chisnall [mailto:dc552 at hermes.cam.ac.uk] On Behalf Of David Chisnall Sent: Wednesday, May 08, 2013 6:53 PM To: Kaylor, Andrew Cc: LLVM Developers Mailing List Subject: Re: TLS with MCJIT (an experimental patch) Hi, Unfortunately, I can't compile this patch. MAP_32BIT is a Linuxism that doesn't work on FreeBSD (or OS X, or, as far as I can tell, anywhere except Linux). We can consider adding something similar to FreeBSD (although I'm hesitant to encourage anything that increases the determinism of the memory layout of JITed code, for security reasons), but it doesn't seem ideal. David On 8 May 2013, at 16:54, "Kaylor, Andrew" <andrew.kaylor at intel.com> wrote:> Hi David, > > Following up on the problems we discussed yesterday on IRC regarding TLS with MCJIT, I've put together the attached experimental patch. > > This patch makes three changes: > > 1. SectionMemoryManager is changed to request memory below the 2GB boundary by default. > 2. sys::Memory::allocateMappedMemory is changed to set the MAP_32BIT flag if the requested "near" block is below the 2GB boundary. > 3. RuntimeDyldELF is changed to recognize the possibility of external data symbols. > > Of these changes, items 2 and 3 are probably reasonable things to commit into trunk, and depending on how this turns out I will do so. Item 1 is a bit heavy-handed as presented here, but it suggests the type of thing that subclasses of SectionMemoryManager could do to make this work. If we had a way to communicate the code model to the memory manager from RuntimeDyld/MCJIT (and we obviously should!) then SectionMemoryManager could do something like this when small or medium memory models are selected on applicable platforms. > > When I tried this patch with the test case you provided yesterday it got through the compilation phase with lli using the small code model and the static relocation model, but it ultimately failed (but failed gracefully) because it couldn't resolve the '_ThreadRuneLocale' symbol. Resolution of external symbols is meant to be handled by the memory manager, so I thought perhaps you could get something working with this patch. > > Please give this a try and let me know how it works. > > Thanks, > Andy > <tls-experimental.patch>