2009/11/22 Jon Harrop <jon at ffconsultancy.com>:> What about parallelization?I thought about that for a while, but if you keep your classes/files small, intra-unit parallelization gains are probably not worth the time invested. Compiling multiple files is embarrassingly parallel. [1] MHO is that, though inter-unit optimizations can take much longer, the benefits are worthwhile. Multiple threads/processes with a message passing interface in between them would be a start, but compiling a unix kernel that way would be tricky memory-wise. ;) cheers, --renato [1] http://en.wikipedia.org/wiki/Embarrassingly_parallel
Renato Golin wrote:> 2009/11/22 Jon Harrop<jon at ffconsultancy.com>: >> What about parallelization? > > I thought about that for a while, but if you keep your classes/files > small, intra-unit parallelization gains are probably not worth the > time invested. Compiling multiple files is embarrassingly parallel. > [1]Compiling multiple files is embarrassingly parallel ... which doesn't matter if your developers are doing a build+edit cycle on a single file. I don't know about the backend, but I've thought about how to make the middle parallelized. We already have the notion of FunctionPass, but this running FPs in parallel will encounter a lot of contention around the reference counts for things like ConstantInt 'i32 0'. That led me to my next idea, which is to turn the use lists for llvm::Constant (except llvm::GlobalValue) off. Just remove that and you lose the lock contention. That means that you can't look at a Constant and iterate over its list of users, but then again, you already weren't doing that. I might experiment with trying out a parallel optimizing LLVM -- mostly to find out what the problems really are -- but I suspect it won't really matter for performance until the backend is parallel. Nick
The place to start would be to make 'llc' parallel on functions. This would directly benefit lto and normal single file compiles. -Chris On Nov 23, 2009, at 12:21 AM, Nick Lewycky <nicholas at mxc.ca> wrote:> Renato Golin wrote: >> 2009/11/22 Jon Harrop<jon at ffconsultancy.com>: >>> What about parallelization? >> >> I thought about that for a while, but if you keep your classes/files >> small, intra-unit parallelization gains are probably not worth the >> time invested. Compiling multiple files is embarrassingly parallel. >> [1] > > Compiling multiple files is embarrassingly parallel ... which doesn't > matter if your developers are doing a build+edit cycle on a single > file. > > I don't know about the backend, but I've thought about how to make the > middle parallelized. We already have the notion of FunctionPass, but > this running FPs in parallel will encounter a lot of contention around > the reference counts for things like ConstantInt 'i32 0'. That led > me to > my next idea, which is to turn the use lists for llvm::Constant > (except > llvm::GlobalValue) off. Just remove that and you lose the lock > contention. That means that you can't look at a Constant and iterate > over its list of users, but then again, you already weren't doing > that. > > I might experiment with trying out a parallel optimizing LLVM -- > mostly > to find out what the problems really are -- but I suspect it won't > really matter for performance until the backend is parallel. > > Nick > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev