search for: ghash

We don't generate any .lib as those don't work well with incremental linking (and give zero advantages when linking AFAIK), and it would be pretty easy to have a modern format for having a .ghash for multiple files, something simple like size prefixed name and then size prefixed ghash blobs. On Fri, Jan 26, 2018 at 8:44 PM, Zachary Turner <zturner at google.com> wrote: > We considered that early on, but most object files actually end up in .lib > files so unless there were a w...

Look for this code in lld/coff/pdb.cpp if (Config->DebugGHashes) { ArrayRef<GloballyHashedType> Hashes; std::vector<GloballyHashedType> OwnedHashes; if (Optional<ArrayRef<uint8_t>> DebugH = getDebugH(File)) Hashes = getHashesFromDebugH(*DebugH); else { OwnedHashes = GloballyHashedType::hashTypes(Types); Hashes = OwnedHashes; } In the...

-lldb-dev, +llvm-dev lldb-dev is specifically for the llvm debugger, and llvm-dev is for everything else (including lld) Did you compile the object files with clang and use -mllvm -emit-codeview-ghash-section? It sounds like probably not. If you don’t do that /DEBUG:GHASH will indeed be much slower. I wonder if we should emit a linker diagnostic in this case On Fri, Jan 19, 2018 at 12:09 PM Leonardo Santagada via lldb-dev < lldb-dev at lists.llvm.org> wrote: > Hi all, > > At...

...ns and as the shadow .debug$H is not really a section anymore (its just some bytes at the end of the file) it doesn't change anything else that does matter. With that I could do my first test with a subset of our code base, and the results are not good. I found one of our sources that break the ghash computation, I will get more info on this and post a proper bug report, but I guess its type information that is generated only by msvc. The other more alarming problem is that linking is way slower with the ghahes... my guess is that we have a bunch of pdb files for some third party libraries and...

On Fri, Jan 19, 2018 at 12:30 PM Andrew Kelley <superjoe30 at gmail.com> wrote: > Zachary, > > Do you generally recommend using -mllvm -emit-codeview-ghash-section and > /DEBUG:GHASH when linking for Windows? > I definitely want you to try it out and let me know how it goes :) It's behind the undocumented -emit-codeview-ghash-section for now until we have some more data indicating it's safe to make default. I'd like to eventually...

...ch > faster can it get? I might try that as well, as 10% speed improvement might > be handy. > > On Sun, Jan 28, 2018 at 11:14 PM, Zachary Turner <zturner at google.com> > wrote: > >> Look for this code in lld/coff/pdb.cpp >> >> >> if (Config->DebugGHashes) { >> ArrayRef<GloballyHashedType> Hashes; >> std::vector<GloballyHashedType> OwnedHashes; >> if (Optional<ArrayRef<uint8_t>> DebugH = getDebugH(File)) >> Hashes = getHashesFromDebugH(*DebugH); >> else { >> OwnedHashes = GloballyHashedTy...

Zachary, Do you generally recommend using -mllvm -emit-codeview-ghash-section and /DEBUG:GHASH when linking for Windows? On Fri, Jan 19, 2018 at 3:17 PM, Zachary Turner via llvm-dev < llvm-dev at lists.llvm.org> wrote: > -lldb-dev, +llvm-dev > > lldb-dev is specifically for the llvm debugger, and llvm-dev is for > everything else (including lld) &...

...link lld-link.exe with LTO+PGO to see how much faster can it get? I might try that as well, as 10% speed improvement might be handy. On Sun, Jan 28, 2018 at 11:14 PM, Zachary Turner <zturner at google.com> wrote: > Look for this code in lld/coff/pdb.cpp > > > if (Config->DebugGHashes) { > ArrayRef<GloballyHashedType> Hashes; > std::vector<GloballyHashedType> OwnedHashes; > if (Optional<ArrayRef<uint8_t>> DebugH = getDebugH(File)) > Hashes = getHashesFromDebugH(*DebugH); > else { > OwnedHashes = GloballyHashedType::hashTypes(Types); &g...

How do you compile LLD? There's a big difference between when using MSVC vs Clang. The parallel ghash patch I was mentioning is almost 2x as fast when using Clang 7.0+ vs. MSVC 15.9+, I don't know exactly why. I also suggest you use the Release target. You should also grab this patch: https://reviews.llvm.org/D55056 - I had to revert it because it was causing issues with LLDB. But it will gi...

Leonardo, to answer to your questions, yes to all of them  You can take a look at this prototype/proposal: https://reviews.llvm.org/D55585 Overall, computing ghashes in parallel at link-time and merging Types with them is less costly that the current approach to merging. The 35sec you’re seeing for merging should go down to about 15sec. The patch doesn’t parallelize (yet) the Type merging itself, but we have an alternate multithread-suitable implementation of...

...eonardo Santagada <santagada at gmail.com> wrote: > > I cleaned up my tests and figured that the obj file generated with > problems was only with msvc 2015, so trying again with msvc 2017 I get: > > lld-link: 4s > lld-link /debug: 1m30s and ~20gb of ram > lld-link /debug:ghash: 59s and ~20gb of ram > link: 13s > link /debug:fastlink: 43s and 1gb of ram > link specialpdb: 1m10s and 4gb of ram > link /debug: 9m16s min and >14gb of ram > > link incremental: 8s when it works. > > > *specialpdb is created with passing to a set of compilation unit...

I cleaned up my tests and figured that the obj file generated with problems was only with msvc 2015, so trying again with msvc 2017 I get: lld-link: 4s lld-link /debug: 1m30s and ~20gb of ram lld-link /debug:ghash: 59s and ~20gb of ram link: 13s link /debug:fastlink: 43s and 1gb of ram link specialpdb: 1m10s and 4gb of ram link /debug: 9m16s min and >14gb of ram link incremental: 8s when it works. *specialpdb is created with passing to a set of compilation units (eg a folder) the same pdb to be written...

...x = 0; >>> void b(int); >>> void a(int) { >>> if (x) >>> b(x); >>> } >>> int main(int argc, char **argv) { >>> a(argc); >>> return x; >>> } >>> >>> >>> clang-cl /Z7 /c a.cpp /Foa.noghash.obj >>> clang-cl /Z7 /c a.cpp -mllvm -emit-codeview-ghash-section >>> /Foa.ghash.good.obj >>> llvm-objcopy a.noghash.obj a.ghash.bad.obj >>> obj2yaml a.ghash.good.obj > a.ghash.good.yaml >>> obj2yaml a.ghash.bad.obj > a.ghash.bad.yaml >>>...

...t;> void a(int) { >>>> if (x) >>>> b(x); >>>> } >>>> int main(int argc, char **argv) { >>>> a(argc); >>>> return x; >>>> } >>>> >>>> >>>> clang-cl /Z7 /c a.cpp /Foa.noghash.obj >>>> clang-cl /Z7 /c a.cpp -mllvm -emit-codeview-ghash-section >>>> /Foa.ghash.good.obj >>>> llvm-objcopy a.noghash.obj a.ghash.bad.obj >>>> obj2yaml a.ghash.good.obj > a.ghash.good.yaml >>>> obj2yaml a.ghash.bad.obj > a.ghash.ba...

...ail.com> > wrote: > >> >> I cleaned up my tests and figured that the obj file generated with >> problems was only with msvc 2015, so trying again with msvc 2017 I get: >> >> lld-link: 4s >> lld-link /debug: 1m30s and ~20gb of ram >> lld-link /debug:ghash: 59s and ~20gb of ram >> link: 13s >> link /debug:fastlink: 43s and 1gb of ram >> link specialpdb: 1m10s and 4gb of ram >> link /debug: 9m16s min and >14gb of ram >> >> link incremental: 8s when it works. >> >> >> *specialpdb is created with...

...>>> >>> I cleaned up my tests and figured that the obj file generated with >>> problems was only with msvc 2015, so trying again with msvc 2017 I get: >>> >>> lld-link: 4s >>> lld-link /debug: 1m30s and ~20gb of ram >>> lld-link /debug:ghash: 59s and ~20gb of ram >>> link: 13s >>> link /debug:fastlink: 43s and 1gb of ram >>> link specialpdb: 1m10s and 4gb of ram >>> link /debug: 9m16s min and >14gb of ram >>> >>> link incremental: 8s when it works. >>> >>> >&...

it does. I just had an epiphany: why not just write a .ghash file and have lld read those if they exist for an .obj file? Seem much simpler than trying to wire up a 20 year old file format. I will try to do this, is something like this acceptable for LLD? The cool thing is that I can generate .ghash for .lib or any obj lying around (maybe even for pdb in th...

We considered that early on, but most object files actually end up in .lib files so unless there were a way to connect the objects in the .lib to the corresponding .ghash files, this would disable ghash usage for a large amount of inputs. Supporting both is an option, but it adds a bit of complexity and I’m not totally convinced it’s worth it On Fri, Jan 26, 2018 at 11:38 AM Leonardo Santagada <santagada at gmail.com> wrote: > it does. > > I just had...

...; if (x) >>>>> b(x); >>>>> } >>>>> int main(int argc, char **argv) { >>>>> a(argc); >>>>> return x; >>>>> } >>>>> >>>>> >>>>> clang-cl /Z7 /c a.cpp /Foa.noghash.obj >>>>> clang-cl /Z7 /c a.cpp -mllvm -emit-codeview-ghash-section >>>>> /Foa.ghash.good.obj >>>>> llvm-objcopy a.noghash.obj a.ghash.bad.obj >>>>> obj2yaml a.ghash.good.obj > a.ghash.good.yaml >>>>> obj2yaml a.ghash.bad...

...t;> I cleaned up my tests and figured that the obj file generated with >>>> problems was only with msvc 2015, so trying again with msvc 2017 I get: >>>> >>>> lld-link: 4s >>>> lld-link /debug: 1m30s and ~20gb of ram >>>> lld-link /debug:ghash: 59s and ~20gb of ram >>>> link: 13s >>>> link /debug:fastlink: 43s and 1gb of ram >>>> link specialpdb: 1m10s and 4gb of ram >>>> link /debug: 9m16s min and >14gb of ram >>>> >>>> link incremental: 8s when it works. >&gt...