[llvm-dev] Using LLD to create a .lib from a .def

Andrew Kelley via llvm-dev
Fri, 16 Jun 2017 20:38:21 -0000

It's a pretty standard build, all I did is change the prefix path:

cmake .. -DCMAKE_INSTALL_PREFIX=/home/andy/local
-DCMAKE_PREFIX_PATH=/home/andy/local -DCMAKE_BUILD_TYPE=Debug

On Fri, Jun 16, 2017 at 4:29 PM, Rui Ueyama <ruiu at google.com> wrote:

> It looks like it is an unrelated build issue (probably related
> to LLVM_BUILD_LLVM_DYLIB), no?
>
> On Fri, Jun 16, 2017 at 11:53 AM, Andrew Kelley <superjoe30 at
gmail.com>
> wrote:
>
>> I saw this change come in yesterday:
>>
>> commit 572ad839e2f66eaa82ffc71b1061eb3d06a4d126
>> Author: Saleem Abdulrasool <compnerd at compnerd.org>
>> Date:   Thu Jun 15 20:39:58 2017 +0000
>>
>>     COFF: add support for lib mode usage
>>
>>     When link is invoked with `/def:` and no input files, it behaves as
if
>>     `lib.exe` was invoked.  Emulate this behaviour, generating the
import
>>     library from the def file that was passed.  Because there is no
input
>> to
>>     actually generate the dll, we simply process the def file early and
>> exit
>>     once we have created the import library.
>>
>>
>> Thank you for this, Saleem.
>>
>> I still have this problem though, with latest master of llvm-project,
>> which is bd8af0ee30bc1a308bf286089b172f2ae87f3de5
>>
>> llvm-config: error: component libraries and shared library
>>
>> llvm-config: error: missing:
/home/andy/local/lib/libLLVMTestingSupport.a
>>
>> I'm pretty sure it's not just me, anyone should be able to
reproduce this
>> issue with a clean build.
>>
>> Regards,
>> Andrew
>>
>> On Thu, Jun 15, 2017 at 12:29 PM, Andrew Kelley <superjoe30 at
gmail.com>
>> wrote:
>>
>>> When I build llvm-project from source (revision
53a3cb539f7e92e70d4eef8d1b219cbef7c8ba1c
>>> from https://github.com/llvm-project/llvm-project-20170507)
>>>
>>> The build and install seemed to work correctly. But then when I run
>>> `llvm-config --libfiles`, I get this:
>>>
>>> llvm-config: error: component libraries and shared library
>>>
>>> llvm-config: error: missing: /home/andy/local/lib/libLLVMTe
>>> stingSupport.a
>>>
>>>
>>> The library got built: ./llvm/build/lib/libLLVMTestingSupport.a
>>> But `make install` didn't copy it to the prefix path.
>>>
>>>
>>> On Thu, Jun 15, 2017 at 10:33 AM, Andrew Kelley <superjoe30 at
gmail.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Wed, Jun 14, 2017 at 7:37 PM, Rui Ueyama <ruiu at
google.com> wrote:
>>>>
>>>>> On Wed, Jun 14, 2017 at 4:24 PM, Andrew Kelley via llvm-dev
<
>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>
>>>>>> I'm copying some LLD code into my codebase like
this:
>>>>>>
>>>>>> // workaround for LLD not exposing ability to convert
.def to .lib
>>>>>>
>>>>>> #include <set>
>>>>>>
>>>>>> namespace lld {
>>>>>> namespace coff {
>>>>>>
>>>>>> class SymbolBody;
>>>>>> class StringChunk;
>>>>>> struct Symbol;
>>>>>>
>>>>>> struct Export {
>>>>>>   StringRef Name;       // N in /export:N or
/export:E=N
>>>>>>   StringRef ExtName;    // E in /export:E=N
>>>>>>   SymbolBody *Sym = nullptr;
>>>>>>   uint16_t Ordinal = 0;
>>>>>>   bool Noname = false;
>>>>>>   bool Data = false;
>>>>>>   bool Private = false;
>>>>>>
>>>>>>   // If an export is a form of /export:foo=dllname.bar,
that means
>>>>>>   // that foo should be exported as an alias to bar in
the DLL.
>>>>>>   // ForwardTo is set to "dllname.bar" part.
Usually empty.
>>>>>>   StringRef ForwardTo;
>>>>>>   StringChunk *ForwardChunk = nullptr;
>>>>>>
>>>>>>   // True if this /export option was in .drectves
section.
>>>>>>   bool Directives = false;
>>>>>>   StringRef SymbolName;
>>>>>>   StringRef ExportName; // Name in DLL
>>>>>>
>>>>>>   bool operator==(const Export &E) {
>>>>>>     return (Name == E.Name && ExtName ==
E.ExtName &&
>>>>>>             Ordinal == E.Ordinal && Noname ==
E.Noname &&
>>>>>>             Data == E.Data && Private ==
E.Private);
>>>>>>   }
>>>>>> };
>>>>>>
>>>>>> enum class DebugType {
>>>>>>   None  = 0x0,
>>>>>>   CV    = 0x1,  /// CodeView
>>>>>>   PData = 0x2,  /// Procedure Data
>>>>>>   Fixup = 0x4,  /// Relocation Table
>>>>>> };
>>>>>>
>>>>>> struct Configuration {
>>>>>>   enum ManifestKind { SideBySide, Embed, No };
>>>>>>   llvm::COFF::MachineTypes Machine =
llvm::COFF::IMAGE_FILE_MACHINE
>>>>>> _UNKNOWN;
>>>>>>   bool Verbose = false;
>>>>>>   llvm::COFF::WindowsSubsystem Subsystem
>>>>>> llvm::COFF::IMAGE_SUBSYSTEM_UNKNOWN;
>>>>>>   SymbolBody *Entry = nullptr;
>>>>>>   bool NoEntry = false;
>>>>>>   std::string OutputFile;
>>>>>>   bool DoGC = true;
>>>>>>   bool DoICF = true;
>>>>>>   bool Relocatable = true;
>>>>>>   bool Force = false;
>>>>>>   bool Debug = false;
>>>>>>   bool WriteSymtab = true;
>>>>>>   unsigned DebugTypes =
static_cast<unsigned>(DebugType::None);
>>>>>>   StringRef PDBPath;
>>>>>>
>>>>>>   // Symbols in this set are considered as live by the
garbage
>>>>>> collector.
>>>>>>   std::set<SymbolBody *> GCRoot;
>>>>>>
>>>>>>   std::set<StringRef> NoDefaultLibs;
>>>>>>   bool NoDefaultLibAll = false;
>>>>>>
>>>>>>   // True if we are creating a DLL.
>>>>>>   bool DLL = false;
>>>>>>   StringRef Implib;
>>>>>>   std::vector<Export> Exports;
>>>>>>   std::set<std::string> DelayLoads;
>>>>>>   std::map<std::string, int> DLLOrder;
>>>>>>   SymbolBody *DelayLoadHelper = nullptr;
>>>>>>
>>>>>>   // Used for SafeSEH.
>>>>>>   Symbol *SEHTable = nullptr;
>>>>>>   Symbol *SEHCount = nullptr;
>>>>>>
>>>>>>   // Used for /opt:lldlto=N
>>>>>>   unsigned LTOOptLevel = 2;
>>>>>>
>>>>>>   // Used for /opt:lldltojobs=N
>>>>>>   unsigned LTOJobs = 1;
>>>>>>
>>>>>>   // Used for /merge:from=to (e.g. /merge:.rdata=.text)
>>>>>>   std::map<StringRef, StringRef> Merge;
>>>>>>
>>>>>>   // Used for /section=.name,{DEKPRSW} to set section
attributes.
>>>>>>   std::map<StringRef, uint32_t> Section;
>>>>>>
>>>>>>   // Options for manifest files.
>>>>>>   ManifestKind Manifest = SideBySide;
>>>>>>   int ManifestID = 1;
>>>>>>   StringRef ManifestDependency;
>>>>>>   bool ManifestUAC = true;
>>>>>>   std::vector<std::string> ManifestInput;
>>>>>>   StringRef ManifestLevel =
"'asInvoker'";
>>>>>>   StringRef ManifestUIAccess =
"'false'";
>>>>>>   StringRef ManifestFile;
>>>>>>
>>>>>>   // Used for /failifmismatch.
>>>>>>   std::map<StringRef, StringRef> MustMatch;
>>>>>>
>>>>>>   // Used for /alternatename.
>>>>>>   std::map<StringRef, StringRef> AlternateNames;
>>>>>>
>>>>>>   uint64_t ImageBase = -1;
>>>>>>   uint64_t StackReserve = 1024 * 1024;
>>>>>>   uint64_t StackCommit = 4096;
>>>>>>   uint64_t HeapReserve = 1024 * 1024;
>>>>>>   uint64_t HeapCommit = 4096;
>>>>>>   uint32_t MajorImageVersion = 0;
>>>>>>   uint32_t MinorImageVersion = 0;
>>>>>>   uint32_t MajorOSVersion = 6;
>>>>>>   uint32_t MinorOSVersion = 0;
>>>>>>   bool DynamicBase = true;
>>>>>>   bool AllowBind = true;
>>>>>>   bool NxCompat = true;
>>>>>>   bool AllowIsolation = true;
>>>>>>   bool TerminalServerAware = true;
>>>>>>   bool LargeAddressAware = false;
>>>>>>   bool HighEntropyVA = false;
>>>>>>
>>>>>>   // This is for debugging.
>>>>>>   bool DebugPdb = false;
>>>>>>   bool DumpPdb = false;
>>>>>> };
>>>>>>
>>>>>> extern Configuration *Config;
>>>>>>
>>>>>> void writeImportLibrary();
>>>>>> void parseModuleDefs(MemoryBufferRef MB);
>>>>>>
>>>>>> } // namespace coff
>>>>>> } // namespace lld
>>>>>>
>>>>>>
//=========================================================>>>>>>
==============>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> This is so that I can write the following user code:
>>>>>>
>>>>>>
>>>>>>
>>>>>> // writes the output to dll_path with .dll replaced
with .lib
>>>>>> void ZigLLDDefToLib(Buf *def_contents, Buf *dll_path) {
>>>>>>     lld::coff::Config = new lld::coff::Configuration;
>>>>>>     auto mem_buf =
MemoryBuffer::getMemBuffer(buf_ptr(def_contents));
>>>>>>     MemoryBufferRef mbref(*mem_buf);
>>>>>>     lld::coff::parseModuleDefs(mbref);
>>>>>>     lld::coff::Config->OutputFile =
buf_ptr(dll_path);
>>>>>>     lld::coff::writeImportLibrary();
>>>>>> }
>>>>>>
>>>>>>
>>>>>> Then I give it def_contents that looks like:
>>>>>> LIBRARY kernel32
>>>>>> EXPORTS
>>>>>> ExitProcess
>>>>>> GetConsoleMode
>>>>>> GetStdHandle
>>>>>> GetFileInformationByHandleEx
>>>>>> WriteFile
>>>>>> GetLastError
>>>>>>
>>>>>>
>>>>>> with dll_path set to ./zig-cache/all.dll. This
generates
>>>>>> ./zig-cache/all.lib.
>>>>>>
>>>>>> The generated LLVM IR looks like:
>>>>>>
>>>>>> ; Function Attrs: noreturn nounwind
>>>>>> declare void @ExitProcess(i32) #6
>>>>>> ; Function Attrs: nounwind
>>>>>> declare i1 @GetConsoleMode(i8* nonnull, i32* nonnull)
#3
>>>>>> ; Function Attrs: nounwind
>>>>>> declare i8* @GetStdHandle(i32) #3
>>>>>> ; Function Attrs: nounwind
>>>>>> declare i1 @GetFileInformationByHandleEx(i8* nonnull,
i32, i8*
>>>>>> nonnull, i32) #3
>>>>>> ; Function Attrs: nounwind
>>>>>> declare i1 @WriteFile(i8* nonnull, i8* nonnull
readonly, i32, i32*,
>>>>>> %OVERLAPPED*) #3
>>>>>> ; Function Attrs: nounwind
>>>>>> declare i32 @GetLastError() #3
>>>>>>
>>>>>> ...with code you would expect to call these functions.
>>>>>>
>>>>>> Then I link with
>>>>>> lld -NOLOGO -MACHINE:X64 -OUT:hello.exe -NODEFAULTLIB
-ENTRY:_start
>>>>>> ./zig-cache/hello.obj ./zig-cache/builtin.obj
./zig-cache/compiler_rt.obj
>>>>>> ./zig-cache/all.lib
>>>>>>
>>>>>>
>>>>>> and I get the following errors:
>>>>>> ./zig-cache/hello.obj: undefined symbol: ExitProcess
>>>>>> ./zig-cache/hello.obj: undefined symbol: GetConsoleMode
>>>>>> ./zig-cache/hello.obj: undefined symbol: GetStdHandle
>>>>>> ./zig-cache/hello.obj: undefined symbol:
GetFileInformationByHandleEx
>>>>>> ./zig-cache/hello.obj: undefined symbol: GetLastError
>>>>>> ./zig-cache/hello.obj: undefined symbol: WriteFile
>>>>>> error: link failed
>>>>>>
>>>>>>
>>>>>>
>>>>>> 1. Is there something else I need to be doing to make
this work
>>>>>> correctly?
>>>>>>
>>>>>
>>>>> The first thing I would check is to make sure that you
created your
>>>>> .lib file for x86-64. Windows uses different name mangling
scheme for x86
>>>>> and x86-64, so if you mix the two, it could result in an
"undefined symbol"
>>>>> error.
>>>>>
>>>>
>>>> I tested this by setting the machine in the config directly to
AMD64.
>>>>
>>>>
>>>>>
>>>>>
>>>>>> (Note: I already tried renaming "all" to
"kernel32". Ideally I could
>>>>>> generate 1 .lib file for all the DLL calls needed.)
>>>>>> 2. Can LLD expose this ability directly so I don't
have to copy paste
>>>>>> a bunch of LLD internal stuff?
>>>>>>
>>>>>
>>>>> Martell recently factored out the code to parse and
generate module
>>>>> definition files. You might be able to use that. See
>>>>> llvm/Object/COFFModuleDefinition.h.
>>>>>
>>>>> The location at where LLD uses COFFModuleDefinition.h is in
>>>>> parseModuleDefs and createImportLibrary in COFF/Driver.cpp.
>>>>>
>>>>
>>>> OK, thanks. I'll get a build going with latest trunk and
see if I can
>>>> get it to work.
>>>>
>>>>
>>>>>
>>>>> Regards,
>>>>>> Andrew Kelley
>>>>>> ziglang.org
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> LLVM Developers mailing list
>>>>>> llvm-dev at lists.llvm.org
>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170616/154a39b3/attachment.html>