llvm dev - Jun 2012 - [LLVMdev] [llvm-commits] [PATCH] Allow per-thread re-direction of outs()/errs()

On Jun 1, 2012, at 7:41 PM, Justin Holewinski wrote:
> >
> > Unfortunately, the use of outs() and (especially) errs() is rampant -
a simple grep of the 3.1 source tree shows about 1,500 instances.  One of the
first things we had to implement in order to make LLVM usable is something very
similar to what Justin has proposed.  Centralizing control of the output in
outs()/errs() would seem to be an ideal way to let users of LLVM as a library
customize it as they see fit without requiring massive source changes.
> 
> Virtually all of those are in DEBUG statements or in llvm/tools.  Neither
are relevant to the discussion.
> 
> In a way, they are.  What if I want a debug trace in a multi-threaded
context?  Right now, all threads would just spew to the same stream and the
result would be unreadable.  If you allow threads to setup their own outs() and
errs() streams (transparently to the rest of LLVM), you can intercept these as
you see fit, perhaps dumping each to a separate file.  I acknowledge that you
could also enforce single-threaded execution for debug runs, but what if you are
in the context of a library with no valid stdout/stderr buffers?
> 
This doesn't make any sense.  There is very little sense in trying to use
DEBUG (which gets compiled out of optimized builds) in a multithreaded context. 
Polluting the purity of outs() and errs() because of this sort of use case
doesn't make any sense to me.

outs() and errs() are useful independent of how LLVM happens to use them.  The
extensions that you are proposing doesn't make any sense give nthe design of
raw_ostream.

-Chris

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120601/87766969/attachment.html>

On Fri, Jun 1, 2012 at 10:17 PM, Chris Lattner <clattner at apple.com>
wrote:
>
> On Jun 1, 2012, at 7:41 PM, Justin Holewinski wrote:
>
> >
>> > Unfortunately, the use of outs() and (especially) errs() is
rampant - a
>> simple grep of the 3.1 source tree shows about 1,500 instances.  One of
the
>> first things we had to implement in order to make LLVM usable is
something
>> very similar to what Justin has proposed.  Centralizing control of the
>> output in outs()/errs() would seem to be an ideal way to let users of
LLVM
>> as a library customize it as they see fit without requiring massive
source
>> changes.
>>
>> Virtually all of those are in DEBUG statements or in llvm/tools. 
Neither
>> are relevant to the discussion.
>>
>
> In a way, they are.  What if I want a debug trace in a multi-threaded
> context?  Right now, all threads would just spew to the same stream and the
> result would be unreadable.  If you allow threads to setup their own outs()
> and errs() streams (transparently to the rest of LLVM), you can intercept
> these as you see fit, perhaps dumping each to a separate file.  I
> acknowledge that you could also enforce single-threaded execution for debug
> runs, but what if you are in the context of a library with no valid
> stdout/stderr buffers?
>
>
> This doesn't make any sense.  There is very little sense in trying to
use
> DEBUG (which gets compiled out of optimized builds) in a multithreaded
> context.  Polluting the purity of outs() and errs() because of this sort of
> use case doesn't make any sense to me.
>
> outs() and errs() are useful independent of how LLVM happens to use them.
>  The extensions that you are proposing doesn't make any sense give nthe
> design of raw_ostream.
>
Is the problem more that outs() and errs() should always point to valid
stdout/stderr streams, regardless of how the hosting compiler is
implemented?

If so, how do you feel about introducing a new global stream, logs()?  This
could replace all uses of outs()/errs() in the LLVM libraries, and default
to errs().  Existing command-line tools could just use this default, but
compilers that are hosted inside of libraries or other non-command line
scenarios can attach a custom stream to logs() that captures all LLVM
output, including DEBUG and any legitimate pass output.  Command-line tools
can continue to use outs()/errs() just as they do now.  Nothing would
really change in the way LLVM operates by default.

>
> -Chris
>
>

-- 

Thanks,

Justin Holewinski
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120601/3562c559/attachment.html>

On Jun 1, 2012, at 11:13 PM, Justin Holewinski wrote:
>> In a way, they are.  What if I want a debug trace in a multi-threaded
context?  Right now, all threads would just spew to the same stream and the
result would be unreadable.  If you allow threads to setup their own outs() and
errs() streams (transparently to the rest of LLVM), you can intercept these as
you see fit, perhaps dumping each to a separate file.  I acknowledge that you
could also enforce single-threaded execution for debug runs, but what if you are
in the context of a library with no valid stdout/stderr buffers?
>> 
> 
> This doesn't make any sense.  There is very little sense in trying to
use DEBUG (which gets compiled out of optimized builds) in a multithreaded
context.  Polluting the purity of outs() and errs() because of this sort of use
case doesn't make any sense to me.
> 
> outs() and errs() are useful independent of how LLVM happens to use them. 
The extensions that you are proposing doesn't make any sense give nthe
design of raw_ostream.
> 
> Is the problem more that outs() and errs() should always point to valid
stdout/stderr streams, regardless of how the hosting compiler is implemented?
No.  outs() and errs() are equivalent to stdout and stderr.  Your request to
change how they work to be thread local is like saying that we should change how
printf works because LLVM is misusing printf somewhere.
> If so, how do you feel about introducing a new global stream, logs()?  This
could replace all uses of outs()/errs() in the LLVM libraries, and default to
errs().  Existing command-line tools could just use this default, but compilers
that are hosted inside of libraries or other non-command line scenarios can
attach a custom stream to logs() that captures all LLVM output, including DEBUG
and any legitimate pass output.  Command-line tools can continue to use
outs()/errs() just as they do now.  Nothing would really change in the way LLVM
operates by default
In fact, before we had raw_ostream, we had a dbgs() sort of stream, which was
intended to be used in debug statements.  I'm not strongly opposed to this
(it would be much better than violating the sanctity of outs() :), but I'm
still unclear why you care so much about DEBUG output.  If we had a dbgs()
again, it should *only* be used from within DEBUG macros (for example, the
linker shouldn't use it).  Would that be enough to achieve your goal?

-Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120602/0c9d0f61/attachment.html>