llvm dev - Aug 2018 - Default compression level for -compress-debug-info=zlib?

> As to (3), in most cases, I believe it is rare to distribute executables
with debug info widely
>  I think it is at least less important than (1).
Agreed.
> I think (1) is definitely the case, and that's also true for a
distributed build system with which a lot of object files are copied
between machines.
> My suggestion was to use compression level 9 when both -O2 and
-compress-debug-section=zlib are specified.

Ok great, I'm less concerned if it still requires an explicit
-compress-debug-section=zlib
even with -O2 (I thought you were proposing to add to O2)

Still for informational / advisory purposes it would be good for us to
produce link time vs compression level vs total exe size, ideally with a
couple of different storage types (at least PCIe SSD vs spinning) and CPUs.

Thanks,

Simon

On Thu, Aug 2, 2018 at 1:17 PM, Rui Ueyama <ruiu at google.com> wrote:
> On Thu, Aug 2, 2018 at 10:24 AM Simon Whittaker via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi Rui,
>>
>> What's the intended advantage for compressing the debug sections? -
(i)
>> Improved link time through smaller IO / (ii) Improved Load / startup
time
>> for the debugger / (iii) Smaller exe with debug info for distribution /
>> disk space?
>>
>
> I think (1) is definitely the case, and that's also true for a
distributed
> build system with which a lot of object files are copied between machines.
>
> I doubt (2) is true. Does compressing debug sections improve debugger load
> time? Of course, as you mentioned, it depends on the ratio of CPU speed and
> IO speed, but since linked debug info isn't as large as the total of
input
> files, I think it is at least less important than (1).
>
> As to (3), in most cases, I believe it is rare to distribute executables
> with debug info widely. Only developers need debug info.
>
> For i) and ii) how much this is worth it depends on balance of storage
>> bandwidth to compression (i) / decompression (ii) bandwidth. For
spinning
>> drives it *might* be a win but for SATA and especially PCIe / NVMe SSD
it
>> could be a CPU bottleneck? Though we should also bear in mind that
>> compression can be pipelined with writes in i) and debug info loading
could
>> be lazy in ii)
>>
>> (e.g. for highly compressible data we've generally seen ~10MiB/s
output
>> bandwidth on single thread i7 @3.2GHz memory to memory for zlib9 with
32KiB
>> window, that doesn't stack up well against modern IO)
>>
>> How is the compression implemented in lld? Is it chunked and therefore
>> paralellizable (and able to be pipelined with IO) or more serial?
>>
>> I think the intention is i) so we'd be happy to link a few of our
game
>> titles with varying compression levels vs storage types and let you
know
>> the results. Might be a couple of weeks...
>>
>> > I wonder what is the best compression level when -O2 is passed to
lld.
>>
>> Just to check, if the default is changed to compress at -O2 we'll
still
>> be able to override to disable compression with
-compress-debug-section=none
>> ?
>>
>
> My suggestion was to use compression level 9 when both -O2 and
> -compress-debug-section=zlib are specified.
>
> Thanks,
>>
>> Simon
>>
>>
>> On Thu, Aug 2, 2018 at 7:00 AM, via llvm-dev <llvm-dev at
lists.llvm.org>
>> wrote:
>>
>>> More data on different compression levels will be good.  In this
case
>>> we're compressing fairly consistent looking input data (a DWARF
section) so
>>> I think we stand a good chance of being able to pick a very
reasonable
>>> level.
>>>
>>> I cringe at the thought of yet another user-facing knob, though.
>>>
>>> --paulr
>>>
>>>
>>>
>>> *From:* llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On
Behalf Of
>>> *James Henderson via llvm-dev
>>> *Sent:* Thursday, August 02, 2018 6:32 AM
>>> *To:* Pavel Labath
>>> *Cc:* LLVM Dev
>>> *Subject:* Re: [llvm-dev] Default compression level for
>>> -compress-debug-info=zlib?
>>>
>>>
>>>
>>> Also not an expert, but would it make sense for this to be
configurable
>>> at a fine-grained level, perhaps with another option, or an
extension to
>>> the compress-debug-sections switch interface? That way users who
care about
>>> the finer details can configure it themselves. And we should pick
sensible
>>> options for the default.
>>>
>>>
>>>
>>> James
>>>
>>>
>>>
>>> On 2 August 2018 at 11:08, Pavel Labath via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>> I don't claim to be an expert, but I did some zlib compression
>>> benchmarks in the past. IIRC, my conclusion from that was that the
>>> "DEFAULT" zlib level (6) is indeed a very good default
for a lot of
>>> cases -- it does not generate much larger outputs, while being
>>> significantly faster than the max level. This all depends on the
data
>>> set and what you intend to do with the resulting data, of course,
but
>>> I guess my point is you don't have to choose only between 1 and
9. I
>>> think it would be interesting to at least get the data for the
default
>>> level before making choice.
>>>
>>> cheers,
>>> pl
>>>
>>> On Thu, 2 Aug 2018 at 01:57, Rui Ueyama via llvm-dev
>>> <llvm-dev at lists.llvm.org> wrote:
>>> >
>>> > Folks,
>>> >
>>> > I'd like to get expert's opinion on which compression
level is
>>> suitable for lld's -compress-debug-section=zlib option, which
let the
>>> linker compress .debug_* sections using zlib.
>>> >
>>> > Currently, lld uses compression level 9 which produces the
smallest
>>> output in exchange for a longer link time. My question is, is this
what
>>> people actually want? We didn't consciously choose compression
level 9.
>>> That was just the default compression level for zlib::compress
function.
>>> >
>>> > For an experiment, I created a patch to use compression level
1
>>> instead of 9 and linked clang using that modified lld. By default,
lld
>>> takes 1m4s to link clang with --compress-debug-sections=zlib. With
that
>>> patch, it took only 31s.
>>> >
>>> > Here is a comparison of clang executable size with various
>>> configurations:
>>> >
>>> > no debug sections:    275 MB
>>> > level 9 compression:  855 MB
>>> > level 1 compression:  922 MB
>>> > no compression:      2044 MB
>>> >
>>> > Given that the best compression takes significantly longer
time than
>>> the fastest compression, we probably should change the default to
level 1.
>>> Any objections?
>>> >
>>> > I wonder what is the best compression level when -O2 is passed
to lld.
>>> We could use level 9 when -O2 is passed, but is there any reason to
>>> compress debug sections that hard in the first place?
>>>
>>> > _______________________________________________
>>> > LLVM Developers mailing list
>>> > llvm-dev at lists.llvm.org
>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180802/139721c0/attachment.html>

Hi Rui,
> My suggestion was to use compression level 9 when both -O2 and
-compress-debug-section=zlib are specified.
The data clearly shows that beyond level 6 there is a huge increase in compute
time (roughly triples) for nearly no benefit (incremental ~0.1% size reduction).
If we're going to do anything more than level 1, then I think 6 is the right
place to stop.

In my experience, building for release really should not be a different process
than a team normally uses.  So, we cannot rely on an argument that
"building for release is rare so it's okay to make it incredibly
expensive."  This reinforces the idea of using compression level 6 instead
of level 9.
> As to (3), in most cases, I believe it is rare to distribute executables
with debug info widely
Agreed, however debug executables are (or should be!) archived, and compressed
debug info looks like it would more than double the number of versions one can
archive on the same media.  Long-term storage may be cheap but it's not
free.

--paulr

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180802/267abd64/attachment.html>

On Thu, Aug 2, 2018 at 2:05 PM Simon Whittaker <simon.f.whittaker at
gmail.com>
wrote:
>
> > As to (3), in most cases, I believe it is rare to distribute
> executables with debug info widely
> >  I think it is at least less important than (1).
>
> Agreed.
>
> > I think (1) is definitely the case, and that's also true for a
> distributed build system with which a lot of object files are copied
> between machines.
> > My suggestion was to use compression level 9 when both -O2 and
> -compress-debug-section=zlib are specified.
>
> Ok great, I'm less concerned if it still requires an explicit  
-compress-debug-section=zlib
> even with -O2 (I thought you were proposing to add to O2)
>
> Still for informational / advisory purposes it would be good for us to
> produce link time vs compression level vs total exe size, ideally with a
> couple of different storage types (at least PCIe SSD vs spinning) and CPUs.
>
Debug sections are compresssed using zlib, so I think such benchmark would
be testing the performance of zlib itself on various conditions.

Thanks,
>
> Simon
>
> On Thu, Aug 2, 2018 at 1:17 PM, Rui Ueyama <ruiu at google.com>
wrote:
>
>> On Thu, Aug 2, 2018 at 10:24 AM Simon Whittaker via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> Hi Rui,
>>>
>>> What's the intended advantage for compressing the debug
sections? - (i)
>>> Improved link time through smaller IO / (ii) Improved Load /
startup time
>>> for the debugger / (iii) Smaller exe with debug info for
distribution /
>>> disk space?
>>>
>>
>> I think (1) is definitely the case, and that's also true for a
>> distributed build system with which a lot of object files are copied
>> between machines.
>>
>> I doubt (2) is true. Does compressing debug sections improve debugger
>> load time? Of course, as you mentioned, it depends on the ratio of CPU
>> speed and IO speed, but since linked debug info isn't as large as
the total
>> of input files, I think it is at least less important than (1).
>>
>> As to (3), in most cases, I believe it is rare to distribute
executables
>> with debug info widely. Only developers need debug info.
>>
>> For i) and ii) how much this is worth it depends on balance of storage
>>> bandwidth to compression (i) / decompression (ii) bandwidth. For
spinning
>>> drives it *might* be a win but for SATA and especially PCIe / NVMe
SSD it
>>> could be a CPU bottleneck? Though we should also bear in mind that
>>> compression can be pipelined with writes in i) and debug info
loading could
>>> be lazy in ii)
>>>
>>> (e.g. for highly compressible data we've generally seen
~10MiB/s output
>>> bandwidth on single thread i7 @3.2GHz memory to memory for zlib9
with 32KiB
>>> window, that doesn't stack up well against modern IO)
>>>
>>> How is the compression implemented in lld? Is it chunked and
therefore
>>> paralellizable (and able to be pipelined with IO) or more serial?
>>>
>>> I think the intention is i) so we'd be happy to link a few of
our game
>>> titles with varying compression levels vs storage types and let you
know
>>> the results. Might be a couple of weeks...
>>>
>>> > I wonder what is the best compression level when -O2 is passed
to lld.
>>>
>>> Just to check, if the default is changed to compress at -O2
we'll still
>>> be able to override to disable compression with
-compress-debug-section=none
>>> ?
>>>
>>
>> My suggestion was to use compression level 9 when both -O2 and
>> -compress-debug-section=zlib are specified.
>>
>> Thanks,
>>>
>>> Simon
>>>
>>>
>>> On Thu, Aug 2, 2018 at 7:00 AM, via llvm-dev <llvm-dev at
lists.llvm.org>
>>> wrote:
>>>
>>>> More data on different compression levels will be good.  In
this case
>>>> we're compressing fairly consistent looking input data (a
DWARF section) so
>>>> I think we stand a good chance of being able to pick a very
reasonable
>>>> level.
>>>>
>>>> I cringe at the thought of yet another user-facing knob,
though.
>>>>
>>>> --paulr
>>>>
>>>>
>>>>
>>>> *From:* llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org]
*On Behalf
>>>> Of *James Henderson via llvm-dev
>>>> *Sent:* Thursday, August 02, 2018 6:32 AM
>>>> *To:* Pavel Labath
>>>> *Cc:* LLVM Dev
>>>> *Subject:* Re: [llvm-dev] Default compression level for
>>>> -compress-debug-info=zlib?
>>>>
>>>>
>>>>
>>>> Also not an expert, but would it make sense for this to be
configurable
>>>> at a fine-grained level, perhaps with another option, or an
extension to
>>>> the compress-debug-sections switch interface? That way users
who care about
>>>> the finer details can configure it themselves. And we should
pick sensible
>>>> options for the default.
>>>>
>>>>
>>>>
>>>> James
>>>>
>>>>
>>>>
>>>> On 2 August 2018 at 11:08, Pavel Labath via llvm-dev <
>>>> llvm-dev at lists.llvm.org> wrote:
>>>>
>>>> I don't claim to be an expert, but I did some zlib
compression
>>>> benchmarks in the past. IIRC, my conclusion from that was that
the
>>>> "DEFAULT" zlib level (6) is indeed a very good
default for a lot of
>>>> cases -- it does not generate much larger outputs, while being
>>>> significantly faster than the max level. This all depends on
the data
>>>> set and what you intend to do with the resulting data, of
course, but
>>>> I guess my point is you don't have to choose only between 1
and 9. I
>>>> think it would be interesting to at least get the data for the
default
>>>> level before making choice.
>>>>
>>>> cheers,
>>>> pl
>>>>
>>>> On Thu, 2 Aug 2018 at 01:57, Rui Ueyama via llvm-dev
>>>> <llvm-dev at lists.llvm.org> wrote:
>>>> >
>>>> > Folks,
>>>> >
>>>> > I'd like to get expert's opinion on which
compression level is
>>>> suitable for lld's -compress-debug-section=zlib option,
which let the
>>>> linker compress .debug_* sections using zlib.
>>>> >
>>>> > Currently, lld uses compression level 9 which produces the
smallest
>>>> output in exchange for a longer link time. My question is, is
this what
>>>> people actually want? We didn't consciously choose
compression level 9.
>>>> That was just the default compression level for zlib::compress
function.
>>>> >
>>>> > For an experiment, I created a patch to use compression
level 1
>>>> instead of 9 and linked clang using that modified lld. By
default, lld
>>>> takes 1m4s to link clang with --compress-debug-sections=zlib.
With that
>>>> patch, it took only 31s.
>>>> >
>>>> > Here is a comparison of clang executable size with various
>>>> configurations:
>>>> >
>>>> > no debug sections:    275 MB
>>>> > level 9 compression:  855 MB
>>>> > level 1 compression:  922 MB
>>>> > no compression:      2044 MB
>>>> >
>>>> > Given that the best compression takes significantly longer
time than
>>>> the fastest compression, we probably should change the default
to level 1.
>>>> Any objections?
>>>> >
>>>> > I wonder what is the best compression level when -O2 is
passed to
>>>> lld. We could use level 9 when -O2 is passed, but is there any
reason to
>>>> compress debug sections that hard in the first place?
>>>>
>>>> > _______________________________________________
>>>> > LLVM Developers mailing list
>>>> > llvm-dev at lists.llvm.org
>>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180802/e7bbf7f2/attachment.html>

Not really, as well as some sensitivity to the input data overall
performance of the link with compression will depend on how this is
implemented in lld - how is it parallelized?  How is it chunked? Is it
effectively pipelined with IO?

Or, I wouldn't feel comfortable being able to make a recommendation to our
end-users on whether to use this option or not based on my existing
extensive benchmarking of zlib in isolation. It's necessary to test in real
conditions.

Thanks,

Simon


On Thu, 2 Aug 2018, 15:22 Rui Ueyama, <ruiu at google.com> wrote:
> On Thu, Aug 2, 2018 at 2:05 PM Simon Whittaker <
> simon.f.whittaker at gmail.com> wrote:
>
>>
>> > As to (3), in most cases, I believe it is rare to distribute
>> executables with debug info widely
>> >  I think it is at least less important than (1).
>>
>> Agreed.
>>
>> > I think (1) is definitely the case, and that's also true for a
>> distributed build system with which a lot of object files are copied
>> between machines.
>> > My suggestion was to use compression level 9 when both -O2 and
>> -compress-debug-section=zlib are specified.
>>
>> Ok great, I'm less concerned if it still requires an explicit  
-compress-debug-section=zlib
>> even with -O2 (I thought you were proposing to add to O2)
>>
>> Still for informational / advisory purposes it would be good for us to
>> produce link time vs compression level vs total exe size, ideally with
a
>> couple of different storage types (at least PCIe SSD vs spinning) and
CPUs.
>>
>
> Debug sections are compresssed using zlib, so I think such benchmark would
> be testing the performance of zlib itself on various conditions.
>
> Thanks,
>>
>> Simon
>>
>> On Thu, Aug 2, 2018 at 1:17 PM, Rui Ueyama <ruiu at google.com>
wrote:
>>
>>> On Thu, Aug 2, 2018 at 10:24 AM Simon Whittaker via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> Hi Rui,
>>>>
>>>> What's the intended advantage for compressing the debug
sections? - (i)
>>>> Improved link time through smaller IO / (ii) Improved Load /
startup time
>>>> for the debugger / (iii) Smaller exe with debug info for
distribution /
>>>> disk space?
>>>>
>>>
>>> I think (1) is definitely the case, and that's also true for a
>>> distributed build system with which a lot of object files are
copied
>>> between machines.
>>>
>>> I doubt (2) is true. Does compressing debug sections improve
debugger
>>> load time? Of course, as you mentioned, it depends on the ratio of
CPU
>>> speed and IO speed, but since linked debug info isn't as large
as the total
>>> of input files, I think it is at least less important than (1).
>>>
>>> As to (3), in most cases, I believe it is rare to distribute
executables
>>> with debug info widely. Only developers need debug info.
>>>
>>> For i) and ii) how much this is worth it depends on balance of
storage
>>>> bandwidth to compression (i) / decompression (ii) bandwidth.
For spinning
>>>> drives it *might* be a win but for SATA and especially PCIe /
NVMe SSD it
>>>> could be a CPU bottleneck? Though we should also bear in mind
that
>>>> compression can be pipelined with writes in i) and debug info
loading could
>>>> be lazy in ii)
>>>>
>>>> (e.g. for highly compressible data we've generally seen
~10MiB/s output
>>>> bandwidth on single thread i7 @3.2GHz memory to memory for
zlib9 with 32KiB
>>>> window, that doesn't stack up well against modern IO)
>>>>
>>>> How is the compression implemented in lld? Is it chunked and
therefore
>>>> paralellizable (and able to be pipelined with IO) or more
serial?
>>>>
>>>> I think the intention is i) so we'd be happy to link a few
of our game
>>>> titles with varying compression levels vs storage types and let
you know
>>>> the results. Might be a couple of weeks...
>>>>
>>>> > I wonder what is the best compression level when -O2 is
passed to
>>>> lld.
>>>>
>>>> Just to check, if the default is changed to compress at -O2
we'll still
>>>> be able to override to disable compression with
-compress-debug-section=none
>>>> ?
>>>>
>>>
>>> My suggestion was to use compression level 9 when both -O2 and
>>> -compress-debug-section=zlib are specified.
>>>
>>> Thanks,
>>>>
>>>> Simon
>>>>
>>>>
>>>> On Thu, Aug 2, 2018 at 7:00 AM, via llvm-dev <llvm-dev at
lists.llvm.org>
>>>> wrote:
>>>>
>>>>> More data on different compression levels will be good.  In
this case
>>>>> we're compressing fairly consistent looking input data
(a DWARF section) so
>>>>> I think we stand a good chance of being able to pick a very
reasonable
>>>>> level.
>>>>>
>>>>> I cringe at the thought of yet another user-facing knob,
though.
>>>>>
>>>>> --paulr
>>>>>
>>>>>
>>>>>
>>>>> *From:* llvm-dev [mailto:llvm-dev-bounces at
lists.llvm.org] *On Behalf
>>>>> Of *James Henderson via llvm-dev
>>>>> *Sent:* Thursday, August 02, 2018 6:32 AM
>>>>> *To:* Pavel Labath
>>>>> *Cc:* LLVM Dev
>>>>> *Subject:* Re: [llvm-dev] Default compression level for
>>>>> -compress-debug-info=zlib?
>>>>>
>>>>>
>>>>>
>>>>> Also not an expert, but would it make sense for this to be
>>>>> configurable at a fine-grained level, perhaps with another
option, or an
>>>>> extension to the compress-debug-sections switch interface?
That way users
>>>>> who care about the finer details can configure it
themselves. And we should
>>>>> pick sensible options for the default.
>>>>>
>>>>>
>>>>>
>>>>> James
>>>>>
>>>>>
>>>>>
>>>>> On 2 August 2018 at 11:08, Pavel Labath via llvm-dev <
>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>
>>>>> I don't claim to be an expert, but I did some zlib
compression
>>>>> benchmarks in the past. IIRC, my conclusion from that was
that the
>>>>> "DEFAULT" zlib level (6) is indeed a very good
default for a lot of
>>>>> cases -- it does not generate much larger outputs, while
being
>>>>> significantly faster than the max level. This all depends
on the data
>>>>> set and what you intend to do with the resulting data, of
course, but
>>>>> I guess my point is you don't have to choose only
between 1 and 9. I
>>>>> think it would be interesting to at least get the data for
the default
>>>>> level before making choice.
>>>>>
>>>>> cheers,
>>>>> pl
>>>>>
>>>>> On Thu, 2 Aug 2018 at 01:57, Rui Ueyama via llvm-dev
>>>>> <llvm-dev at lists.llvm.org> wrote:
>>>>> >
>>>>> > Folks,
>>>>> >
>>>>> > I'd like to get expert's opinion on which
compression level is
>>>>> suitable for lld's -compress-debug-section=zlib option,
which let the
>>>>> linker compress .debug_* sections using zlib.
>>>>> >
>>>>> > Currently, lld uses compression level 9 which produces
the smallest
>>>>> output in exchange for a longer link time. My question is,
is this what
>>>>> people actually want? We didn't consciously choose
compression level 9.
>>>>> That was just the default compression level for
zlib::compress function.
>>>>> >
>>>>> > For an experiment, I created a patch to use
compression level 1
>>>>> instead of 9 and linked clang using that modified lld. By
default, lld
>>>>> takes 1m4s to link clang with
--compress-debug-sections=zlib. With that
>>>>> patch, it took only 31s.
>>>>> >
>>>>> > Here is a comparison of clang executable size with
various
>>>>> configurations:
>>>>> >
>>>>> > no debug sections:    275 MB
>>>>> > level 9 compression:  855 MB
>>>>> > level 1 compression:  922 MB
>>>>> > no compression:      2044 MB
>>>>> >
>>>>> > Given that the best compression takes significantly
longer time than
>>>>> the fastest compression, we probably should change the
default to level 1.
>>>>> Any objections?
>>>>> >
>>>>> > I wonder what is the best compression level when -O2
is passed to
>>>>> lld. We could use level 9 when -O2 is passed, but is there
any reason to
>>>>> compress debug sections that hard in the first place?
>>>>>
>>>>> > _______________________________________________
>>>>> > LLVM Developers mailing list
>>>>> > llvm-dev at lists.llvm.org
>>>>> >
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>
>>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180802/a19f2e4f/attachment.html>