llvm dev - Nov 2019 - Unable to parse command line more than once using llvm libraries?

Hi,

I am running a server that does regular builds using llc. To begin with I was
doing a fork/exec to launch llc, passing the files to compile and other switches
as command line, etc. then waiting for the subprocess to complete. This was fine
but seemed an unnecessary overhead. After all llc is mostly a fairly thin front
end over the llc libraries, which do the actual work right?

So I took llc.cpp, and the llvm-c and llvm headers and llvm libraries and linked
them all into my server program (which is a C based server). I modified llc.cpp
to change...

int llcmain(int argc, char **argv) {
...
}

...to...

extern "C" int llcmain(int argc, char **argv) {
...
}


Now I am calling this function directly from my server code.  This works fine
for the first build, however any other builds fail in

CommandLine.cpp:1101

CommandLineParser::ParseCommandLineOptions

assert(hasOptions() && "No options specified!"); <--fails
here


I'm trying to work out why this is breaking, I am assuming there is some
global state left after processing that llc.cpp doesn't normally need to
worry about?

Is there a way to reset this global state cleanly? I'm assuming there's
something about command line processing that is written to only work once, not
repeatedly?

Thanks for any advice/help.

I'm not entirely sure what it is that is causing your problem. It's
worth
noting that the unit tests for CommandLine.cpp can call
cl::ParseCommandLineOptions multiple times within a process, although it
looks like the options parsed are explicitly removed at the end of each
test to avoid polluting the environment of the next test.

Probably the right thing to do is to call the
"ResetAllOptionOccurrences"
function of the command line parser. This should allow you to use it again
cleanly. If you don't call that, you might end up setting options multiple
times, which could lead to unexpected behaviour. Note that the comment in
this function says that it is explicitly intended to allow parsing
different command lines multiple times.

Hope that helps.

James

On Sun, 17 Nov 2019 at 15:17, Carl Peto via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi,
>
> I am running a server that does regular builds using llc. To begin with I
> was doing a fork/exec to launch llc, passing the files to compile and other
> switches as command line, etc. then waiting for the subprocess to complete.
> This was fine but seemed an unnecessary overhead. After all llc is mostly a
> fairly thin front end over the llc libraries, which do the actual work
> right?
>
> So I took llc.cpp, and the llvm-c and llvm headers and llvm libraries and
> linked them all into my server program (which is a C based server). I
> modified llc.cpp to change...
>
> int llcmain(int argc, char **argv) {
> ...
> }
>
> ...to...
>
> extern "C" int llcmain(int argc, char **argv) {
> ...
> }
>
>
> Now I am calling this function directly from my server code.  This works
> fine for the first build, however any other builds fail in
>
> CommandLine.cpp:1101
>
> CommandLineParser::ParseCommandLineOptions
>
> assert(hasOptions() && "No options specified!");
<--fails here
>
>
> I'm trying to work out why this is breaking, I am assuming there is
some
> global state left after processing that llc.cpp doesn't normally need
to
> worry about?
>
> Is there a way to reset this global state cleanly? I'm assuming
there's
> something about command line processing that is written to only work once,
> not repeatedly?
>
> Thanks for any advice/help.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191118/61989751/attachment.html>

Thanks,


I tried calling ResetAllOptionOccurrences after the run like this…




  // Compile the module TimeCompilations times to give better compile time
  // metrics.
  for (unsigned I = TimeCompilations; I; --I)
    if (int RetVal = compileModule(argv, Context))
      return RetVal;

  if (YamlFile)
    YamlFile->keep();

  cl::ResetAllOptionOccurrences();

  return 0;
}





Unfortunately it is still crashing out when the next build starts. Any other
thoughts?

> On 18 Nov 2019, at 10:13, James Henderson via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> I'm not entirely sure what it is that is causing your problem. It's
worth noting that the unit tests for CommandLine.cpp can call
cl::ParseCommandLineOptions multiple times within a process, although it looks
like the options parsed are explicitly removed at the end of each test to avoid
polluting the environment of the next test.
> 
> Probably the right thing to do is to call the
"ResetAllOptionOccurrences" function of the command line parser. This
should allow you to use it again cleanly. If you don't call that, you might
end up setting options multiple times, which could lead to unexpected behaviour.
Note that the comment in this function says that it is explicitly intended to
allow parsing different command lines multiple times.
> 
> Hope that helps.
> 
> James
> 
> On Sun, 17 Nov 2019 at 15:17, Carl Peto via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> Hi,
> 
> I am running a server that does regular builds using llc. To begin with I
was doing a fork/exec to launch llc, passing the files to compile and other
switches as command line, etc. then waiting for the subprocess to complete. This
was fine but seemed an unnecessary overhead. After all llc is mostly a fairly
thin front end over the llc libraries, which do the actual work right?
> 
> So I took llc.cpp, and the llvm-c and llvm headers and llvm libraries and
linked them all into my server program (which is a C based server). I modified
llc.cpp to change...
> 
> int llcmain(int argc, char **argv) {
> ...
> }
> 
> ...to...
> 
> extern "C" int llcmain(int argc, char **argv) {
> ...
> }
> 
> 
> Now I am calling this function directly from my server code.  This works
fine for the first build, however any other builds fail in
> 
> CommandLine.cpp:1101
> 
> CommandLineParser::ParseCommandLineOptions
> 
> assert(hasOptions() && "No options specified!");
<--fails here
> 
> 
> I'm trying to work out why this is breaking, I am assuming there is
some global state left after processing that llc.cpp doesn't normally need
to worry about?
> 
> Is there a way to reset this global state cleanly? I'm assuming
there's something about command line processing that is written to only work
once, not repeatedly?
> 
> Thanks for any advice/help.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191118/d2901a7f/attachment.html>