llvm dev - Sep 2012 - [LLVMdev] RFC: Adding an option to llvm-link to allow it to get a list of input bitcode file names from a file

I am proposing to add an option to llvm-link allow it to get a list of input
bitcode file names from a file.

The reason is that there is a limitation for command line length which limits
the number of input bitcode files that can be passed to llvm-link. By adding
this option we can bypass such limitation.

The name of the option can be discussed. My initial proposal would be
-input-file-list.

Each line of the list file is a path to an input bitcode file.

If this option is given, no input bitcode file names are allowed in the command
line. If input bitcode file names are given in the command line, this option is
not allowed.

The implementation is simple. Only llvm-link.cpp is affected.

Thanks.

Sam Liu

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120913/d8ff1478/attachment.html>

Why is there a restriction on the usage between command line and input file? It
seems that they both should be able to co-exist and the resulting linking is
done as a set of all the bitcode files.

From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On
Behalf Of Liu, Yaxun (Sam)
Sent: Thursday, September 13, 2012 9:51 AM
To: llvmdev at cs.uiuc.edu
Subject: [LLVMdev] RFC: Adding an option to llvm-link to allow it to get a list
of input bitcode file names from a file

I am proposing to add an option to llvm-link allow it to get a list of input
bitcode file names from a file.

The reason is that there is a limitation for command line length which limits
the number of input bitcode files that can be passed to llvm-link. By adding
this option we can bypass such limitation.

The name of the option can be discussed. My initial proposal would be
-input-file-list.

Each line of the list file is a path to an input bitcode file.

If this option is given, no input bitcode file names are allowed in the command
line. If input bitcode file names are given in the command line, this option is
not allowed.

The implementation is simple. Only llvm-link.cpp is affected.

Thanks.

Sam Liu

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120913/580bd723/attachment.html>

My assumption is that a user will either specify input file names in the command
line or passing them by a list file, but usually would not do both. Making these
two different ways of specifying input files mutually exclusive may avoid
confusion and misuse.

Sam

From: Villmow, Micah
Sent: Thursday, September 13, 2012 12:54 PM
To: Liu, Yaxun (Sam); llvmdev at cs.uiuc.edu
Subject: RE: [LLVMdev] RFC: Adding an option to llvm-link to allow it to get a
list of input bitcode file names from a file

Why is there a restriction on the usage between command line and input file? It
seems that they both should be able to co-exist and the resulting linking is
done as a set of all the bitcode files.

From: llvmdev-bounces at cs.uiuc.edu<mailto:llvmdev-bounces at
cs.uiuc.edu> [mailto:llvmdev-bounces at
cs.uiuc.edu]<mailto:[mailto:llvmdev-bounces at cs.uiuc.edu]> On Behalf Of
Liu, Yaxun (Sam)
Sent: Thursday, September 13, 2012 9:51 AM
To: llvmdev at cs.uiuc.edu<mailto:llvmdev at cs.uiuc.edu>
Subject: [LLVMdev] RFC: Adding an option to llvm-link to allow it to get a list
of input bitcode file names from a file

I am proposing to add an option to llvm-link allow it to get a list of input
bitcode file names from a file.

The reason is that there is a limitation for command line length which limits
the number of input bitcode files that can be passed to llvm-link. By adding
this option we can bypass such limitation.

The name of the option can be discussed. My initial proposal would be
-input-file-list.

Each line of the list file is a path to an input bitcode file.

If this option is given, no input bitcode file names are allowed in the command
line. If input bitcode file names are given in the command line, this option is
not allowed.

The implementation is simple. Only llvm-link.cpp is affected.

Thanks.

Sam Liu

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120913/c1bac848/attachment.html>

Liu, Yaxun (Sam) wrote:
> I am proposing to add an option to llvm-link allow it to get a list of
> input bitcode file names from a file.
>
> The reason is that there is a limitation for command line length which
> limits the number of input bitcode files that can be passed to
> llvm-link. By adding this option we can bypass such limitation.
>
> The name of the option can be discussed. My initial proposal would be
> –input-file-list.
It's common for tools to accept "@file" as an argument that means
"read
options from file", where options may include both flags and positional 
arguments (ie., more input files). For instance, gcc, as and ld all 
support @file.

I realize this is more complicated than just reading in a list of input 
bitcode files, but I think the right fix is to teach llvm to support 
this. (It looks like clang already does? What are they doing?)

Nick
> Each line of the list file is a path to an input bitcode file.
>
> If this option is given, no input bitcode file names are allowed in the
> command line. If input bitcode file names are given in the command line,
> this option is not allowed.
>
> The implementation is simple. Only llvm-link.cpp is affected.
>
> Thanks.
>
> Sam Liu
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

It seems clang supports @file.

http://llvm.org/svn/llvm-project/cfe/trunk/tools/driver/driver.cpp

So maybe it is a better idea to let llvm-link support @file instead of adding a
new option.

Sam
-----Original Message-----
From: Nick Lewycky [mailto:nicholas at mxc.ca] 
Sent: Thursday, September 13, 2012 2:33 PM
To: Liu, Yaxun (Sam)
Cc: llvmdev at cs.uiuc.edu
Subject: Re: [LLVMdev] RFC: Adding an option to llvm-link to allow it to get a
list of input bitcode file names from a file

Liu, Yaxun (Sam) wrote:
> I am proposing to add an option to llvm-link allow it to get a list of 
> input bitcode file names from a file.
>
> The reason is that there is a limitation for command line length which 
> limits the number of input bitcode files that can be passed to 
> llvm-link. By adding this option we can bypass such limitation.
>
> The name of the option can be discussed. My initial proposal would be 
> -input-file-list.
It's common for tools to accept "@file" as an argument that means
"read options from file", where options may include both flags and
positional arguments (ie., more input files). For instance, gcc, as and ld all
support @file.

I realize this is more complicated than just reading in a list of input bitcode
files, but I think the right fix is to teach llvm to support this. (It looks
like clang already does? What are they doing?)

Nick
> Each line of the list file is a path to an input bitcode file.
>
> If this option is given, no input bitcode file names are allowed in 
> the command line. If input bitcode file names are given in the command 
> line, this option is not allowed.
>
> The implementation is simple. Only llvm-link.cpp is affected.
>
> Thanks.
>
> Sam Liu
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

Just found llvm-link (and other llvm commands) have already supported @file
through cl::ParseCommandLineOptions, therefore no change is needed. Thanks.

Sam

-----Original Message-----
From: Liu, Yaxun (Sam) 
Sent: Thursday, September 13, 2012 3:06 PM
To: 'Nick Lewycky'
Cc: llvmdev at cs.uiuc.edu
Subject: RE: [LLVMdev] RFC: Adding an option to llvm-link to allow it to get a
list of input bitcode file names from a file

It seems clang supports @file.

http://llvm.org/svn/llvm-project/cfe/trunk/tools/driver/driver.cpp

So maybe it is a better idea to let llvm-link support @file instead of adding a
new option.

Sam
-----Original Message-----
From: Nick Lewycky [mailto:nicholas at mxc.ca]
Sent: Thursday, September 13, 2012 2:33 PM
To: Liu, Yaxun (Sam)
Cc: llvmdev at cs.uiuc.edu
Subject: Re: [LLVMdev] RFC: Adding an option to llvm-link to allow it to get a
list of input bitcode file names from a file

Liu, Yaxun (Sam) wrote:
> I am proposing to add an option to llvm-link allow it to get a list of 
> input bitcode file names from a file.
>
> The reason is that there is a limitation for command line length which 
> limits the number of input bitcode files that can be passed to 
> llvm-link. By adding this option we can bypass such limitation.
>
> The name of the option can be discussed. My initial proposal would be 
> -input-file-list.
It's common for tools to accept "@file" as an argument that means
"read options from file", where options may include both flags and
positional arguments (ie., more input files). For instance, gcc, as and ld all
support @file.

I realize this is more complicated than just reading in a list of input bitcode
files, but I think the right fix is to teach llvm to support this. (It looks
like clang already does? What are they doing?)

Nick
> Each line of the list file is a path to an input bitcode file.
>
> If this option is given, no input bitcode file names are allowed in 
> the command line. If input bitcode file names are given in the command 
> line, this option is not allowed.
>
> The implementation is simple. Only llvm-link.cpp is affected.
>
> Thanks.
>
> Sam Liu
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

Well, although cl::ParseCommandLineOptions contains support for @file. By
default it is disabled, which is the case for llvm-link.

To enable @file support in llvm-link, a small change is needed:

--- llvm-link.cpp.orig  2012-09-20 16:10:50.000000000 -0400
+++ llvm-link.cpp       2012-09-20 16:11:24.000000000 -0400
@@ -83,7 +83,7 @@

   LLVMContext &Context = getGlobalContext();
   llvm_shutdown_obj Y;  // Call llvm_shutdown() on exit.
-  cl::ParseCommandLineOptions(argc, argv, "llvm linker\n");
+  cl::ParseCommandLineOptions(argc, argv, "llvm linker\n", true);

   unsigned BaseArg = 0;
   std::string ErrorMessage;

Sam

-----Original Message-----
From: Liu, Yaxun (Sam) 
Sent: Thursday, September 20, 2012 2:59 PM
To: 'Nick Lewycky'
Cc: 'llvmdev at cs.uiuc.edu'
Subject: RE: [LLVMdev] RFC: Adding an option to llvm-link to allow it to get a
list of input bitcode file names from a file

Just found llvm-link (and other llvm commands) have already supported @file
through cl::ParseCommandLineOptions, therefore no change is needed. Thanks.

Sam

-----Original Message-----
From: Liu, Yaxun (Sam)
Sent: Thursday, September 13, 2012 3:06 PM
To: 'Nick Lewycky'
Cc: llvmdev at cs.uiuc.edu
Subject: RE: [LLVMdev] RFC: Adding an option to llvm-link to allow it to get a
list of input bitcode file names from a file

It seems clang supports @file.

http://llvm.org/svn/llvm-project/cfe/trunk/tools/driver/driver.cpp

So maybe it is a better idea to let llvm-link support @file instead of adding a
new option.

Sam
-----Original Message-----
From: Nick Lewycky [mailto:nicholas at mxc.ca]
Sent: Thursday, September 13, 2012 2:33 PM
To: Liu, Yaxun (Sam)
Cc: llvmdev at cs.uiuc.edu
Subject: Re: [LLVMdev] RFC: Adding an option to llvm-link to allow it to get a
list of input bitcode file names from a file

Liu, Yaxun (Sam) wrote:
> I am proposing to add an option to llvm-link allow it to get a list of 
> input bitcode file names from a file.
>
> The reason is that there is a limitation for command line length which 
> limits the number of input bitcode files that can be passed to 
> llvm-link. By adding this option we can bypass such limitation.
>
> The name of the option can be discussed. My initial proposal would be 
> -input-file-list.
It's common for tools to accept "@file" as an argument that means
"read options from file", where options may include both flags and
positional arguments (ie., more input files). For instance, gcc, as and ld all
support @file.

I realize this is more complicated than just reading in a list of input bitcode
files, but I think the right fix is to teach llvm to support this. (It looks
like clang already does? What are they doing?)

Nick
> Each line of the list file is a path to an input bitcode file.
>
> If this option is given, no input bitcode file names are allowed in 
> the command line. If input bitcode file names are given in the command 
> line, this option is not allowed.
>
> The implementation is simple. Only llvm-link.cpp is affected.
>
> Thanks.
>
> Sam Liu
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev