search for: lto_module_dispose

...ect files are loaded (which means no undefined symbols are left), the linker then calls: lto_codegen_create() and then in a for-loop calls lto_codegen_add_module() on each module previously loaded. 3) After lto_codegen_compile() has returned, the linker does clean up and deletes each module with lto_module_dispose(). It sounds like the linker could call lto_module_dispose() right after lto_codegen_add_module() to help reduce the memory footprint. That would be a simple linker change. A slightly larger linker change would be to immediately call lto_codegen_add_module() right after lto_module_create_from_me...

> It sounds like the linker could call lto_module_dispose() right after > lto_codegen_add_module() to help reduce the memory footprint. That would be > a simple linker change. A slightly larger linker change would be to > immediately call lto_codegen_add_module() right after > lto_module_create_from_memory(), then lto_module_dispose(). That...

On Tue, Nov 12, 2013 at 6:07 PM, Manman Ren <manman.ren at gmail.com> wrote: > Hi Chandler, > > I don't quite get why you think sharing is not buying us anything... > It reduces the memory footprint of the source modules (there is sharing > among the source modules) and the number of MDNodes created for the > destination module (we do not need to re-create the MDNodes

On Wed, Nov 13, 2013 at 7:58 PM, Rafael Espíndola < rafael.espindola at gmail.com> wrote: > > It sounds like the linker could call lto_module_dispose() right after > > lto_codegen_add_module() to help reduce the memory footprint. That > would be > > a simple linker change. A slightly larger linker change would be to > > immediately call lto_codegen_add_module() right after > > lto_module_create_from_memory(), then lt...

On Tue, Nov 12, 2013 at 4:59 PM, Chandler Carruth <chandlerc at google.com>wrote: > On Tue, Nov 12, 2013 at 4:46 PM, Manman Ren <manman.ren at gmail.com> wrote: > >> >> >> >> On Tue, Nov 12, 2013 at 4:38 PM, Chandler Carruth <chandlerc at google.com>wrote: >> >>> >>> On Tue, Nov 12, 2013 at 4:29 PM, Manman Ren <manman.ren

...lue* const*, unsigned int, llvm::MDNode::FunctionLocalness, bool) (Metadata.cpp:226) The file itself is 4.5MB, so 36MB from just this allocation spot is a lot. Having 71,717 blocks also suggests that we should be using some pool. For now I would be more then happy if I was just able to free that. lto_module_dispose is being called and the llvm module is being destructed, but it looks like the only way to free the metadata is to destroy the LLVMContext. Is this by design? why? Thanks, Rafael

...ptors open? That is odd. I will debug it in a sec, but we have in the claim_file_hook: if (code_gen) { if (lto_codegen_add_module(code_gen, M)) { (*message)(LDPL_ERROR, "Error linking module: %s", lto_get_error_message()); return LDPS_ERR; } } lto_module_dispose(M); In fact, with current gold we call get_view, so the plugin uses the same fd as gold. It might actually be a bug with gold trying to cache too many open files. How are you trying to build it? Cheers, Rafael

LTOCodeGenerator::addModule (which is wrapped by lto_codegen_add_module) takes an LTOModule as an argument and links the latter's llvm::Module into its own. It does not change the latter's llvm::Module. The lto_module_dispose API call destroys an LTOModule. This frees the LTOModule's llvm::Module, but also invalidates strings returned by, e.g., lto_module_get_symbol_name. I propose that LTOCodeGenerator::addModule be changed to also deallocate the (consumed and redundant) llvm::Module and llvm::TargetMachine objec...

On Nov 14, 2013, at 8:19 AM, Rafael Espíndola <rafael.espindola at gmail.com> wrote: > But gold has at most 2 objects loaded at any time. Are you sure about that? I haven't looked into it but while building Chromium with LTO, I get: ../../third_party/gold/gold64: fatal error: out of file descriptors and couldn't close any clang: error: linker command failed with exit code 1

...in a sec, but we have in the claim_file_hook: > > if (code_gen) { > if (lto_codegen_add_module(code_gen, M)) { > (*message)(LDPL_ERROR, "Error linking module: %s", > lto_get_error_message()); > return LDPS_ERR; > } > } > > lto_module_dispose(M); > > In fact, with current gold we call get_view, so the plugin uses the > same fd as gold. It might actually be a bug with gold trying to cache > too many open files. > > How are you trying to build it? The standard Chromium build system was modified to add -flto -Os to cfl...

...Because many of the `scope:` chains currently terminate at `MDFile`s or `null` instead of `MDCompileUnit`s. Why? Because LTO type uniquing needs scope chains to be identical. (I have a vague plan for fixing this, too: (1) move ownership of Metadata to the Module (so metadata isn't leaked by `lto_module_dispose()`), (2) add a "StringRef -> MDType" map to the Module (only for types with an ODR-style UUID), (3) delete the concept of `MDString`-based type refs and update lib/Linker to rely on the "StringRef -> MDType" map in the destination Module, (4) make all `scope:` chains termi...

Hi all, libLTO is exposing a very “stable” (in the sense of immutable) C API to be used by linkers (and binutils tools) that manipulate bitcode (like when performing LTO). I’m looking into relaxing the stability concern and design a policy for this API that would allow to deprecate and remove some the APIs exposed here. The MacOS linker (ld64) is one the users of libLTO, but there are others

...MDNode::FunctionLocalness, > bool) (Metadata.cpp:226) > > The file itself is 4.5MB, so 36MB from just this allocation spot is a > lot. Having 71,717 blocks also suggests that we should be using some pool. > > For now I would be more then happy if I was just able to free that. > lto_module_dispose is being called and the llvm module is being > destructed, but it looks like the only way to free the metadata is to > destroy the LLVMContext. Is this by design? why? Types, constants, and metadata are all associated with the LLVMContext; the primary reason for this is that it maintains poi...

...e_hook: >> >> if (code_gen) { >> if (lto_codegen_add_module(code_gen, M)) { >> (*message)(LDPL_ERROR, "Error linking module: %s", >> lto_get_error_message()); >> return LDPS_ERR; >> } >> } >> >> lto_module_dispose(M); >> >> In fact, with current gold we call get_view, so the plugin uses the >> same fd as gold. It might actually be a bug with gold trying to cache >> too many open files. >> >> How are you trying to build it? > > The standard Chromium build system was m...

...ields would probably work? (Which means I shouldn't have merged the two types of "file" nodes when I introduced the new hierarchy. Boo.) > > (I have a vague plan for fixing this, too: (1) move > ownership of Metadata to the Module (so metadata isn't leaked by > `lto_module_dispose()`), (2) add a "StringRef -> MDType" map to the > Module (only for types with an ODR-style UUID), (3) delete the concept > of `MDString`-based type refs and update lib/Linker to rely on the > "StringRef -> MDType" map in the destination Module, (4) make all > `...

...`scope:` chains currently terminate at `MDFile`s or `null` instead of > `MDCompileUnit`s. Why? Because LTO type uniquing needs scope chains > to be identical. (I have a vague plan for fixing this, too: (1) move > ownership of Metadata to the Module (so metadata isn't leaked by > `lto_module_dispose()`), (2) add a "StringRef -> MDType" map to the > Module (only for types with an ODR-style UUID), (3) delete the concept > of `MDString`-based type refs and update lib/Linker to rely on the > "StringRef -> MDType" map in the destination Module, (4) make all > `...