search for: propell

...m <tmsriram at google.com> > > Sent: Thursday, September 26, 2019 5:31 PM > > To: Eli Friedman <efriedma at quicinc.com> > > Cc: Xinliang David Li <xinliangli at gmail.com>; llvm-dev <llvm-dev at lists.llvm.org> > > Subject: [EXT] Re: [llvm-dev] [RFC] Propeller: A frame work for Post Link > > Optimizations > > > > For more details, We strongly considered this. We could run something > > like a thin link in thin lto figure out the global layout and hand out > > the relevant subsets of the global decision to each module....

...lam <tmsriram at google.com> > > Sent: Friday, September 27, 2019 9:43 AM > > To: Eli Friedman <efriedma at quicinc.com> > > Cc: Xinliang David Li <xinliangli at gmail.com>; llvm-dev <llvm-dev at lists.llvm.org> > > Subject: [EXT] Re: [llvm-dev] [RFC] Propeller: A frame work for Post Link > > Optimizations > > > > > > > > > > Why are you proposing to add a bunch of options to clang to manipulate > > LLVM > > > > code generation, given none of those options are actually used for Propeller > > &g...

...of BOLT and the memory and time overhead shoots up quickly for large binaries. Inspired by the performance gains and to address the scalability issue of BOLT, we went about designing a scalable infrastructure that can perform BOLT-like post-link optimizations. In this RFC, we discuss our system, “Propeller”, which can perform profile guided link time binary optimizations in a scalable way and is friendly to distributed build systems. Our system leverages the existing capabilities of the compiler tool-chain and is not a stand alone tool. Like BOLT, our system boosts the performance of optimized bin...

...gt; > Sent: Friday, September 27, 2019 9:43 AM > > > > To: Eli Friedman <efriedma at quicinc.com> > > > > Cc: Xinliang David Li <xinliangli at gmail.com>; llvm-dev < > llvm-dev at lists.llvm.org> > > > > Subject: [EXT] Re: [llvm-dev] [RFC] Propeller: A frame work for Post > Link > > > > Optimizations > > > > > > > > > > > > > > > > Why are you proposing to add a bunch of options to clang to > manipulate > > > > LLVM > > > > > > code generation, gi...

...rom: Xinliang David Li <xinliangli at gmail.com> > Sent: Wednesday, September 25, 2019 5:58 PM > To: Eli Friedman <efriedma at quicinc.com> > Cc: Sriraman Tallam <tmsriram at google.com>; llvm-dev <llvm-dev at lists.llvm.org> > Subject: [EXT] Re: [llvm-dev] [RFC] Propeller: A frame work for Post Link Optimizations > > > > > > > > On Wed, Sep 25, 2019 at 5:02 PM Eli Friedman via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > My biggest question about this architecture is about when propeller runs basic block reordering within a...

...ember 27, 2019 9:43 AM > >> > > > To: Eli Friedman <efriedma at quicinc.com> > >> > > > Cc: Xinliang David Li <xinliangli at gmail.com>; llvm-dev < > llvm-dev at lists.llvm.org> > >> > > > Subject: [EXT] Re: [llvm-dev] [RFC] Propeller: A frame work for > Post Link > >> > > > Optimizations > >> > > > > >> > > > > > >> > > > > > > Why are you proposing to add a bunch of options to clang to > manipulate > >> > > > LLVM >...

On Wed, Sep 25, 2019 at 5:02 PM Eli Friedman via llvm-dev < llvm-dev at lists.llvm.org> wrote: > My biggest question about this architecture is about when propeller runs > basic block reordering within a function. It seems like a lot of the > complexity comes from using the proposed -fbasicblock-sections to generated > mangled ELF, and then re-parsing the mangled ELF as a separate step. I'm > not sure that's the right approach, long-ter...

...>> >> > > > To: Eli Friedman <efriedma at quicinc.com> > >> >> > > > Cc: Xinliang David Li <xinliangli at gmail.com>; llvm-dev < > llvm-dev at lists.llvm.org> > >> >> > > > Subject: [EXT] Re: [llvm-dev] [RFC] Propeller: A frame work > for Post Link > >> >> > > > Optimizations > >> >> > > > > >> >> > > > > > >> >> > > > > > > Why are you proposing to add a bunch of options to clang > to manipulate &g...

...m <tmsriram at google.com> > > Sent: Thursday, September 26, 2019 3:24 PM > > To: Eli Friedman <efriedma at quicinc.com> > > Cc: Xinliang David Li <xinliangli at gmail.com>; llvm-dev <llvm-dev at lists.llvm.org> > > Subject: [EXT] Re: [llvm-dev] [RFC] Propeller: A frame work for Post Link > > Optimizations > > > > On Thu, Sep 26, 2019 at 12:39 PM Eli Friedman <efriedma at quicinc.com> wrote: > > > > > > > > > > > > From: Xinliang David Li <xinliangli at gmail.com> > > > Sent: Wed...

...> > To: Eli Friedman <efriedma at quicinc.com> >>> >> >> > > > Cc: Xinliang David Li <xinliangli at gmail.com>; llvm-dev < >>> llvm-dev at lists.llvm.org> >>> >> >> > > > Subject: [EXT] Re: [llvm-dev] [RFC] Propeller: A frame work >>> for Post Link >>> >> >> > > > Optimizations >>> >> >> > > > >>> >> >> > > > > >>> >> >> > > > > > > Why are you proposing to add a bunch o...

I met with the Propeller team today (we work for the same company but it was my first time meeting two members on the team:) ). One thing I have been reassured: * There is no general disassembly work. General disassembly work would assuredly frighten off developers. (Inherently unreliable, memory usage heavy and diffic...

I have a data file that looks like this class pigeon falcon propeller jet wing fly birds 25 37 0 0 2 1 planes 0 1 28 40 1 3 birds 19 41 0 1 4 6 planes 0 0 25 50 5 5 pla...

...a loop would have to know what it's > doing (i.e. would have to be aware of the loop structure and what the rules > were), there was no risk that the metadata would become accidentally > separated from the loop. > Here is an analogy for your consideration. "Where do I put the propeller?" LLVM (the optimizer/codegen) is like a car design. It's got a good engine, frame, and other parts. It's got an active community of people around refining the design and using it for many purposes. It's a good car for these people. OpenMP and similar pragmas are like a helicop...

...6 iterations and only 4 branches had to be grown back. On Mon, Oct 7, 2019 at 11:17 AM Sriraman Tallam <tmsriram at google.com> wrote: > > The next concern we want to address is about C++ Exceptions which was > considered a major omission: > > TLDR; This can be implemented in propeller easily. It was considered > low priority because exception handling code is assumed in general not > performance critical . > > Currently, we do not create basic block sections for any block that is > involved in exception handling. There is nothing fundamentally > difficult a...

Is there large value from deferring the block ordering to link time? That is, does the block layout algorithm need to consider global layout issues when deciding which blocks to put together and which to relegate to the far-away part of the code? Or, could the propellor-optimized compile step instead split each function into only 2 pieces -- one containing an "optimally-ordered" set of hot blocks from the function, and another containing the cold blocks? The linker would have less flexibility in placement, but maybe it doesn't actually need that fl...

..., we have prototyped a "Thin" version that optimizes Clang-7 in > under > > 15 seconds using less than 4GB of memory. No modifications to the linker or > > compiler were required. And by the way, that appears to be faster than > just the > > re-linking phase of the Propeller. Larger loads show similar improvements > > providing 2x-5x savings over the original processing specs. > > > > Let me reiterate that current BOLT requires large amounts of memory not > because > > it's a fundamental limitation, unlike LTO. For us, system memory was...

...system at Google scale is shown where 5 million binary and test builds are performed every day on several thousands of machines, each with a limitation of 12G of memory per process and 15 minute time-out on tests. Memory overheads of 35G (clang) are well above these thresholds. We have developed Propeller like ThinLTO that can be used to obtain similar performance gains like BOLT in such environments. Thanks Sri On Fri, Oct 11, 2019 at 11:25 AM Xinliang David Li via llvm-dev < llvm-dev at lists.llvm.org> wrote: > > > On Fri, Oct 11, 2019 at 10:46 AM James Y Knight via llvm-dev &...

...uploaded the binaries, BOLT's yaml files and > perf.data files and the commands so that you can independently verify our > results and check the binaries. We have gzipped all the files to keep it > under 2G limit for git lfs, everything is here : > https://github.com/google/llvm-propeller/tree/plo-dev/clang-bolt-experiment > We have run our experiments on a 192G machine with Intel 18 core. > > > > We built llvm-bolt with most recent sources and is *pristine* with none of > our patches and uploading the binary we used here, > https://github.com/google/llvm-prop...

...ransparency* we have uploaded the binaries, BOLT's yaml files and perf.data files and the commands so that you can independently verify our results and check the binaries. We have gzipped all the files to keep it under 2G limit for git lfs, everything is here : https://github.com/google/llvm-propeller/tree/plo-dev/clang-bolt-experiment We have run our experiments on a 192G machine with Intel 18 core. We built llvm-bolt with most recent sources and is *pristine* with none of our patches and uploading the binary we used here, https://github.com/google/llvm-propeller/blob/plo-dev/clang-bolt-expe...

On Wed, Oct 2, 2019 at 8:41 PM Maksim Panchenko via llvm-dev < llvm-dev at lists.llvm.org> wrote: > *Pessimization/overhead for stack unwinding used by system-wide profilers > and > for exception handling* > > Larger CFI programs put an extra burden on unwinding at runtime as more CFI > (and thus native) instructions have to be executed. This will cause more > overhead