llvm dev - May 2008 - [LLVMdev] building llvm on Windows

Not that I sympathize with the OP's manners but...

Bill Wendling <isanbard at gmail.com> writes:
> On May 10, 2008, at 7:55 PM, kr512 wrote:
>
>> See how gcc is invoked to generate the final executable
>> file.  This means LLVM is an incomplete backend,
>> unfortunately.
>>
> That's only a convenience. GCC generates assembly code too and calls  
> the assembler and linker as part of it's execution. You are perfectly  
> able to call the assembler & linker yourself.
This means that LLVM requires an assembler and linker. Call it GCC or
binutils, it is irrelevant. The OP point is that LLVM is not a
self-sufficient tool on this aspect.

Of course, if this is a serious problem for the OP, the correct way of
dealing with it is to take constructive, polite actions for correcting
it :-)
>> Bill Wendling wrote:
>>> I don't know about your computer, by mine comes with an
>>> assembler.
>>
>> MS Windows does not come with an assembler, AFAIK.
>>
> You should really learn how to use Google. Got this as the top hit for  
> "Microsoft Assembler":
>
>
http://www.microsoft.com/downloads/details.aspx?FamilyID=7a1c9da0-0510-44a2-b042-7ef370530c64&displaylang=en
This does not *comes* with MS Windows. It is a separate download, which
depends on more downloads (Visual C++ Express Edition) and is for
non-commercial use only. Furthermore, you can't re-distribute it.

OTOH, I'm curious about why the OP needs to produce dlls on the
fly. Isn't the JIT ok?

-- 
Oscar

On May 11, 2008, at 9:36 AM, Óscar Fuentes wrote:
>
> Not that I sympathize with the OP's manners but...
>
> Bill Wendling <isanbard at gmail.com> writes:
>>
>> That's only a convenience. GCC generates assembly code too and
calls
>> the assembler and linker as part of it's execution. You are
perfectly
>> able to call the assembler & linker yourself.
>
> This means that LLVM requires an assembler and linker. Call it GCC or
> binutils, it is irrelevant. The OP point is that LLVM is not a
> self-sufficient tool on this aspect.
>
Actually, I think Bill's right on target.  The OP was implying that  
LLVM is incomplete because it depends on GCC in the backend, which is  
incorrect.  It depends on binutils, _which GCC also depends on_.  Thus  
LLVM is certainly no less complete than GCC in that sense.

I think the point is that implementing our own linker is definitely  
beyond the purview of LLVM, and implementing our own object file  
writer/assembler is honestly pretty low on the priority queue.  GCC,  
the primary alternative, also depends on these things, so having them  
would be a competitive advantage, not a requirement for parity.
>
>>> Bill Wendling wrote:
>>>> I don't know about your computer, by mine comes with an
>>>> assembler.
>>>
>>> MS Windows does not come with an assembler, AFAIK.
>>>
>> You should really learn how to use Google. Got this as the top hit  
>> for
>> "Microsoft Assembler":
>>
>>
http://www.microsoft.com/downloads/details.aspx?FamilyID=7a1c9da0-0510-44a2-b042-7ef370530c64&displaylang=en
>
> This does not *comes* with MS Windows. It is a separate download,  
> which
> depends on more downloads (Visual C++ Express Edition) and is for
> non-commercial use only. Furthermore, you can't re-distribute it.
>
 From what I read, it's included with copies of Visual Studio.  I  
assume that a paid copy of Visual Studio (as opposed to the free  
Express edition) includes a version that's licensed for commercial use.

Additionally, LLVM's support for Win32 outside of MinGW/Cygwin is ...  
spotty at best.  And in those environments, the competition is, once  
again, GCC, which still depends on gas and ld.

The fundamental problem is that both executable formats and linking  
problems are EXTREMELY platform specific, so it's typically best to  
leave it up to the system tools.  Assembly/object file generation is a  
_possibility_ for us only because there is some commonality between  
platforms (i.e. ELF is used on many platforms).  But it's still low on  
the queue because there already system-provided tools that do it  
better on every platform we care about (*nix + MinGW/Cygwin).

--Owen

P.S.  I do agree that, if he has a desperate need to run LLVM outside  
of MinGW/Cygwin, JIT is his best bet.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4260 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20080511/87c1ff42/attachment.bin>

Owen Anderson <resistor at mac.com> writes:
>>> That's only a convenience. GCC generates assembly code too and
calls
>>> the assembler and linker as part of it's execution. You are
perfectly
>>> able to call the assembler & linker yourself.
>>
>> This means that LLVM requires an assembler and linker. Call it GCC or
>> binutils, it is irrelevant. The OP point is that LLVM is not a
>> self-sufficient tool on this aspect.
>>
>
> Actually, I think Bill's right on target.  The OP was implying that
> LLVM is incomplete because it depends on GCC in the backend, which is
> incorrect.  It depends on binutils, _which GCC also depends on_.  Thus
> LLVM is certainly no less complete than GCC in that sense.
There is a culture schism here. The Windows folks are accustomed to
monolithic packages that does all the work. For Unix folks, having
"tool-chains" is the natural way. For a Windows folk, having extra
requirements goes against what he is accustomed to. If LLVM is
perceived as a backend, but it fails short into producing the final
result, the typical Windows folk will be disappointed.

The OP talks about GCC because that's what he sees, irrespectively of
the other tools behind. In Windows, installing GCC implies installing
binutils, either via MinGW or Cygwin. This is deeply rooted into the
Windows development culture: a "compiler" has all needed parts for
creating an executable. They talk about the Visual C++ compiler, or the
Borland compiler, never about the Borland compiler and linker.
> I think the point is that implementing our own linker is definitely
> beyond the purview of LLVM, and implementing our own object file
> writer/assembler is honestly pretty low on the priority queue.  GCC,
> the primary alternative, also depends on these things, so having them
> would be a competitive advantage, not a requirement for parity.
Absolutely agreed. I'm just trying to explain from where the OP (seems
to) come.
>>> "Microsoft Assembler":
>>>
>>>
http://www.microsoft.com/downloads/details.aspx?FamilyID=7a1c9da0-0510-44a2-b042-7ef370530c64&displaylang=en
>>
>> This does not *comes* with MS Windows. It is a separate download,
>> which
>> depends on more downloads (Visual C++ Express Edition) and is for
>> non-commercial use only. Furthermore, you can't re-distribute it.
>>
>
> From what I read, it's included with copies of Visual Studio.  I
> assume that a paid copy of Visual Studio (as opposed to the free
> Express edition) includes a version that's licensed for commercial
> use.
Exactly. And this is very different from the GNU/Linux and BSD case,
where the existence of a compiler, assembler and linker along the OS is
considered a "reasonable requirement" and part of the standard
installation in most cases. If the OP plans to deploy his compiler on
Windows, he can't assume the existence of an assembler/linker on most
user's machines. He becomes dependent of third-party packages, hence
LLVM is not a "complete backend" from his (limited) point of
view. (Leaving aside the fact that a compiler backend is not required to
generate executable files, but his insistence on this point is related
to the "monolithic package" paradigm I mentioned above).
> Additionally, LLVM's support for Win32 outside of MinGW/Cygwin is
> ... spotty at best.
Sure. And, AFAIK, even if you want to use MASM, you need to add support
for his assembler format to LLVM.
> And in those environments, the competition is, once again, GCC, which
> still depends on gas and ld.
>
> The fundamental problem is that both executable formats and linking
> problems are EXTREMELY platform specific,
[snip]

Agreed. I'm just trying to explain that, from the POV of the OP's
culture, the demands he expressed are quite reasonable. This narrow view
of the world and being... ummmh... too demanding, are the only problems
with the OP message :-)

Expect more messages like this as LLVM becomes more popular outside
professional and academical circles.

-- 
Oscar

On May 11, 2008, at 7:36 AM, Óscar Fuentes wrote:
>
> Not that I sympathize with the OP's manners but...
>
> Bill Wendling <isanbard at gmail.com> writes:
>
>> On May 10, 2008, at 7:55 PM, kr512 wrote:
>>
>>> See how gcc is invoked to generate the final executable
>>> file.  This means LLVM is an incomplete backend,
>>> unfortunately.
>>>
>> That's only a convenience. GCC generates assembly code too and
calls
>> the assembler and linker as part of it's execution. You are
perfectly
>> able to call the assembler & linker yourself.
>
> This means that LLVM requires an assembler and linker. Call it GCC or
> binutils, it is irrelevant. The OP point is that LLVM is not a
> self-sufficient tool on this aspect.
FWIW, I wouldn't be surprised if the LLVM project eventually grew more  
of native toolchain support (e.g. assembler, linker, etc).  It would  
fit naturally with the growing scope of the project.  I don't think  
that it is a high priority for anyone though.

One advantage of having at least simple assembler support is being  
able to handle inline asm in the (currently incomplete and buggy)  
macho/elf writers.

-Chris

I tried to compile on Windows ( using the win32/llvm.sln in MSVC Express 
2008 ) the new 2.3 branch and here are the results:

1. I needed to copy the file Configure.exe.embed.manifest to 
Configure.exe.intermediate.manifest . This is a bug from the previous llvm 
release (2.2)

2. I copied the file SimplifyLibCalls.cpp from lib/Transforms/Scalar to 
lib/Transforms/IPO. I think the position of this file was updated but the 
MSVC project wasn't.

3. I put a "return SDOperand();" on X86ISelLowering.cpp, on function 
"SDOperand X86TargetLowering::LowerVAARG(SDOperand Op, SelectionDAG
&DAG)"
after "abort()", to make compiler happy to compile it.

4. I needed to give "Build Solution" more times, because there are
some
incorrect dependencies between projects and some of them expect results from 
the future builded ones.

After over 4500 warnings, I think most of them because of the automatically 
generated code, I succeded to compile almost all the projects, except opt 
and bugpoint. At these two I had an error:

error LNK2019: unresolved external symbol "class llvm::FunctionPass * 
__cdecl llvm::createLibCallAliasAnalysisPass(class llvm::LibCallInfo *)" 
(?createLibCallAliasAnalysisPass at llvm@@YAPAVFunctionPass at 1@PAVLibCallInfo
at 1@@Z)
referenced in function "public: __thiscall `anonymous 
namespace'::ForcePassLinking::ForcePassLinking(void)" 
(??0ForcePassLinking@?A0xd1580448@@QAE at XZ)

which I wasn't able to fix it.

I hope these will be fixed on the final 2.3 release so it can be compiled 
out of the box on MSVC Express 2008.

Now I'm checking on some Windows assemblers and other tools which can 
complete llvm to make it a full backend (with the final result an exe or dll 
file).

Razvan

PS: I also tried to compile llvm 2.3 in MSYS using the latest MinGW with gcc 
4.3.0. This wasn't able to compile llvm at all because some errors like:

d:\msys\bin\../lib/gcc/i386-pc-mingw32/4.3.0/include/c++/bits/boost_concept_check.h:
In member function 'bool __gnu_cxx::_EqualOpConcept<_First, 
_Second>::__constraints_() [with _First = llvm::BasicBlock, _Second = 
llvm::BasicBlock*]':
d:\msys\bin\../lib/gcc/i386-pc-mingw32/4.3.0/include/c++/bits/boost_concept_check.h:296:
instantiated from 'void __gnu_cxx::_EqualOpConcept<_First, 
_Second>::__constraints() [with _First = llvm::BasicBlock, _Second = 
llvm::BasicBlock*]'
d:\msys\bin\../lib/gcc/i386-pc-mingw32/4.3.0/include/c++/bits/boost_concept_check.h:62:
instantiated from 'void __gnu_cxx::__function_requires() [with _Concept = 
__gnu_cxx::_EqualOpConcept<llvm::BasicBlock, llvm::BasicBlock*>]'
d:\msys\bin\../lib/gcc/i386-pc-mingw32/4.3.0/include/c++/bits/stl_algo.h:3810: 
instantiated from '_IIter std::find(_IIter, _IIter, const _Tp&) [with
_IIter
= llvm::PredIterator<llvm::BasicBlock,
llvm::value_use_iterator<llvm::User>
 >, _Tp = llvm::BasicBlock*]'
BasicBlock.cpp:180:   instantiated from here
d:\msys\bin\../lib/gcc/i386-pc-mingw32/4.3.0/include/c++/bits/boost_concept_check.h:296:
error: no match for 'operator==' in 
'((__gnu_cxx::_EqualOpConcept<llvm::BasicBlock, 
llvm::BasicBlock*>*)this)->__gnu_cxx::_EqualOpConcept<llvm::BasicBlock,
llvm::BasicBlock*>::__a == ((__gnu_cxx::_EqualOpConcept<llvm::BasicBlock, 
llvm::BasicBlock*>*)this)->__gnu_cxx::_EqualOpConcept<llvm::BasicBlock,
llvm::BasicBlock*>::__b'
D:/msys/home/Razvan/release_23/include/llvm/ADT/APInt.h:1242: note: 
candidates are: bool llvm::operator==(uint64_t, const llvm::APInt&)
make[1]: *** [/home/Razvan/release_23/lib/VMCore/Release/BasicBlock.o] Error 
1

On May 11, 2008, at 7:36 AM, Óscar Fuentes wrote:
>
> Not that I sympathize with the OP's manners but...
>
> Bill Wendling <isanbard at gmail.com> writes:
>
>> On May 10, 2008, at 7:55 PM, kr512 wrote:
>>
>>> See how gcc is invoked to generate the final executable
>>> file.  This means LLVM is an incomplete backend,
>>> unfortunately.
>>>
>> That's only a convenience. GCC generates assembly code too and
calls
>> the assembler and linker as part of it's execution. You are
perfectly
>> able to call the assembler & linker yourself.
>
> This means that LLVM requires an assembler and linker. Call it GCC or
> binutils, it is irrelevant. The OP point is that LLVM is not a
> self-sufficient tool on this aspect.
>
Then the exact same thing is true for GCC.
> Of course, if this is a serious problem for the OP, the correct way of
> dealing with it is to take constructive, polite actions for correcting
> it :-)
>
>>> Bill Wendling wrote:
>>>> I don't know about your computer, by mine comes with an
>>>> assembler.
>>>
>>> MS Windows does not come with an assembler, AFAIK.
>>>
>> You should really learn how to use Google. Got this as the top hit  
>> for
>> "Microsoft Assembler":
>>
>>
http://www.microsoft.com/downloads/details.aspx?FamilyID=7a1c9da0-0510-44a2-b042-7ef370530c64&displaylang=en
>
> This does not *comes* with MS Windows. It is a separate download,  
> which
> depends on more downloads (Visual C++ Express Edition) and is for
> non-commercial use only. Furthermore, you can't re-distribute it.
>
It's a separate download for me too. And there are other tools  
available for MS Windows which achieve the same goals. The point is  
that he was wrong.
> OTOH, I'm curious about why the OP needs to produce dlls on the
> fly. Isn't the JIT ok?
>
I doubt he knows.

-bw

On May 11, 2008, at 12:59 PM, Razvan Aciu wrote:
> I tried to compile on Windows ( using the win32/llvm.sln in MSVC  
> Express
> 2008 ) the new 2.3 branch and here are the results:
It would be great to get any windows improvements into the 2.3 release  
if possible.
> 1. I needed to copy the file Configure.exe.embed.manifest to
> Configure.exe.intermediate.manifest . This is a bug from the  
> previous llvm
> release (2.2)
Please send a patch.
> 2. I copied the file SimplifyLibCalls.cpp from lib/Transforms/Scalar  
> to
> lib/Transforms/IPO. I think the position of this file was updated  
> but the
> MSVC project wasn't.
Please update the project file and send a patch for it.
> 3. I put a "return SDOperand();" on X86ISelLowering.cpp, on
function
> "SDOperand X86TargetLowering::LowerVAARG(SDOperand Op, SelectionDAG  
> &DAG)"
> after "abort()", to make compiler happy to compile it.
This went into mainline, Tanya, can we pull this patch into 2.3?
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20080512/062268.html
> 4. I needed to give "Build Solution" more times, because there
are
> some
> incorrect dependencies between projects and some of them expect  
> results from
> the future builded ones.
Please fix them :)
> After over 4500 warnings, I think most of them because of the  
> automatically
> generated code, I succeded to compile almost all the projects,  
> except opt
> and bugpoint. At these two I had an error:
>
> error LNK2019: unresolved external symbol "class llvm::FunctionPass *
> __cdecl llvm::createLibCallAliasAnalysisPass(class llvm::LibCallInfo  
> *)"
> (? 
> createLibCallAliasAnalysisPass 
> @llvm@@YAPAVFunctionPass at 1@PAVLibCallInfo at 1@@Z)
> referenced in function "public: __thiscall `anonymous
> namespace'::ForcePassLinking::ForcePassLinking(void)"
> (??0ForcePassLinking@?A0xd1580448@@QAE at XZ)
>
> which I wasn't able to fix it.
This sounds like lib/Analysis/LibCallAliasAnalysis.cpp needs to be  
added to the project file.
> I hope these will be fixed on the final 2.3 release so it can be  
> compiled
> out of the box on MSVC Express 2008.
We would love to have this fixed, but since most of the core  
developers aren't using VC++, the only way it will get fixed for LLVM  
2.3 is if someone sends up patches.  Can you please do this?

-Chris

On May 11, 2008, at 12:59 PM, Razvan Aciu wrote:
> 3. I put a "return SDOperand();" on X86ISelLowering.cpp, on
function
> "SDOperand X86TargetLowering::LowerVAARG(SDOperand Op, SelectionDAG  
> &DAG)"
> after "abort()", to make compiler happy to compile it.
This one is now fixed on trunk and the 2.3 release branch. Thanks!

Dan

> This means that LLVM requires an assembler and linker. Call it
> GCC or binutils, it is irrelevant. The OP point is that LLVM
> is not a self-sufficient tool on this aspect.
>
> Of course, if this is a serious problem for the OP, the
> correct way of dealing with it is to take constructive, polite
> actions for correcting it :-)
I know one compiler (Free Pascal) that directly can write ELF and 
files and also Windows EXE/DLL files.

So it is actually possible to do this with limited resources (the 
FPC developer community isn't that large). It just has to be 
done.

On 13 May 2008, at 09:21, Holger Schurig wrote:
> I know one compiler (Free Pascal) that directly can write ELF and
> files and also Windows EXE/DLL files.
>
> So it is actually possible to do this with limited resources (the
> FPC developer community isn't that large). It just has to be
> done.
FWIW, our internal assembler (only available for i386) is more or less  
an integrated Pascal translation of nasm. The linker (only available  
for Windows/i386) is, afaik, an original work. The main reason for the  
latter was that at the time it was written, the "--gc-sections" option
didn't exist yet for GNU ld, and performing dead code stripping by  
linking in static libraries (which we could already generate using our  
internal assembler) containing a single object file for each global  
symbol is excruciatingly slow with GNU ld.

Further, before we had an internal assembler and linker for Windows,  
we also used the GNU assembler and linker there (and they are still an  
option). And to confirm what others said earlier on: you do not have  
to ship an entire MingW or Cygwin distribution to be able to use them.  
In fact, when you ship the MingW versions of those tools there are no  
external dependencies whatsoever.

You can find the whole bunch (as.exe, ar.exe, ld.exe, strip.exe etc)  
precompiled in e.g.
ftp://ftp.freepascal.org/pub/fpc/dist/i386-win32-2.0.4/separate/base.i386-win32.zip
  (along with some other fpc-related binaries -- note that 2.0.4 is  
not the latest fpc release, but later releases only have a big  
installer available for download and no separate zipfiles with just  
the binaries)


Jonas