llvm dev - Jan 2013 - [LLVMdev] Obsolete PTX is NOT completely removed in 3.2 release

On Sun, Jan 13, 2013 at 01:00:55PM -0600, Pawel Wodnicki
wrote:
> Brooks,
> 
> > On Fri, Jan 11, 2013 at 02:47:01PM -0600, Pawel Wodnicki wrote:
> >> On 1/11/2013 2:40 PM, Brooks Davis wrote:
> >>> On Fri, Jan 11, 2013 at 09:33:17PM +0100, Benjamin Kramer
> >>> wrote:
> >>>> 
> >>>> On 11.01.2013, at 21:31, Justin Holewinski 
> >>>> <justin.holewinski at gmail.com> wrote:
> >>>> 
> >>>>> On Fri, Jan 11, 2013 at 3:26 PM, Benjamin Kramer 
> >>>>> <benny.kra at gmail.com> wrote:
> >>>>> 
> >>>>> On 11.01.2013, at 07:36, ????????? (Wei-Ren Chen) 
> >>>>> <chenwj at iis.sinica.edu.tw> wrote:
> >>>>> 
> >>>>>> Hi Pawel,
> >>>>>> 
> >>>>>> PTX already be replaced with NVPTX. However, PTX
> >>>>>> subdirectory still sit in lib/Target in 3.2
release. Do
> >>>>>> you think update the release tarball is a good
idea? Also
> >>>>>> could you remove it from the trunk?
> >>>>> 
> >>>>> Please do not, under no circumstances, change the 3.2
> >>>>> release tarballs at this point. They are mirrored
around
> >>>>> the world now with cryptographic hashes and
signatures.
> >>>>> Changing them will break things for many people,
especially
> >>>>> for an extremely minor thing like an empty directory.
> >>>>> 
> >>>>> I'm not sure if Pawel's tarball change should
be reverted
> >>>>> now as it already caused uproar, so changing it back
might
> >>>>> only make matters worse.
> >>>>> 
> >>>>> The tarballs were changed?
> >>>> 
> >>>> r172208
> >>> 
> >>> I finally updated the FreeBSD ports yesterday and today a user
> >>>  complained about distfile changes.  IMO, this revision should
> >>> be reverted or all the other BSDs will have to chase checksums
> >>> as well.
> >>> 
> >>> If you really want to remove the directory, ship a 3.2.1
> >>> tarball rather than screwing all the downstream consumers
who's
> >>>  infrastructure exists to detect trojan'd tarballs.
> >> 
> >> Tarball is signed, it is not trjoan. Your infrastructure should
> >> be able to deal with it?
> > 
> > The FreeBSD ports collection maintains a set of sha256 hashes for 
> > each distfile.  The system can deal with them changing, but it's
> > an inconvenience to port maintainers and users.  Even if we did
> > have infrastructure to verify the signatures we'd still have to
> > check the contents and would not just trust the signature since
> > there's always a risk of keys being compromised.
> > 
> > -- Brooks
> > 
> 
>  At first your reaction has puzzled me, but despite a few fire
> breathing dragons getting loose and after carefully re-reading
> these posts over the weekend I think I understand the underlying
> issue.
> 
>  Simply put we both take this release seriously.
> 
>  The problem is the release process is not entirely
> defined. Which is reflected in question from David Blaikie
> (the other post I am quoting here).
> 
> > Which release process document are you referring to that indicates
> > that?
> > 
> 
>  Simple answer is the one on the llvm.org website.
> But the real answer is, it all depends on how you read it.
> Apparently, we read it differently which is not that
> surprising as even our well defined compilers have
> a bit of a problem with undefined behavior.
> 
> So what can we expect from a release process document?
> My hope is that we have reached a sort of sequence
> point and the ambiguity of the release process
> can be resolved despite all the side effects.
> 
> 
> David also stated this:
> 
> > Generally, so far as I know, this kind of post-release change is
> > unprecedented.
> 
>  I agree with this conclusion but even more unprecedented
> is to a expect timely and problem free release with zero and
> I'll repeat again zero involvement from the wider community that
> depends on the clang+llvm!
>  This is how the 3.2 release has happened, just compare
> the list of 3.2 to 3.1 binaries and BTW not a single
> 3.2 binary has been produced with the help of a respective
> project, distribution or who ever.
> 
>  I hear the cries of your wounded infrastructures
> and I understand that in the days of forged root certificates
> my signature on the tarballs may not guarantee much of a
> security.
> 
>  But why didn't you get involved in the 3.2 release?
>
>  We could have resolved all this in the 2 months since
> the 3.2 release got rolling. Everybody knew about the
> changes for the 3.2 release, simple e-mail to the new
> LLVMRM would go a long way to resolve any security or
> other release related issues not to mention a bit
> of a courtesy.
I don't understand how my lack of involvement with this release has
anything to do with your decision to fix what appears to me to be
a non-problem in the release tarball well after the release and in
the process do something (replacing release tarballs) that is quite
simply NOT DONE to by projects who wish to treat their downstreams
with respect.  This is something many projects end up learning due to
the painful way so it's not something you should feel bad about as
long as the project learns the right lesson(s).

If anything I'd be more annoyed if I'd actually gotten the port updated
sooner since then it would have worked for more than a day.  As it
is, I could really use a final decision on which tarball will be at
http://www.llvm.org/releases/3.2/llvm-3.2.src.tar.gz in perpetuity so I
can change the port or not as required and avoid churn.

-- Brooks
> 
>  I am doing this as service to the LLVM community
> but frankly I'd rather be chasing my dragons of
> probability now.
> 
> Pawe??
> 
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130114/71088329/attachment.sig>

On 1/14/2013 12:40 PM, Brooks Davis wrote:
> On Sun, Jan 13, 2013 at 01:00:55PM -0600, Pawel Wodnicki wrote:
>> Brooks,
>>
>>> On Fri, Jan 11, 2013 at 02:47:01PM -0600, Pawel Wodnicki wrote:
>>>> On 1/11/2013 2:40 PM, Brooks Davis wrote:
>>>>> On Fri, Jan 11, 2013 at 09:33:17PM +0100, Benjamin Kramer
>>>>> wrote:
>>>>>>
>>>>>> On 11.01.2013, at 21:31, Justin Holewinski 
>>>>>> <justin.holewinski at gmail.com> wrote:
>>>>>>
>>>>>>> On Fri, Jan 11, 2013 at 3:26 PM, Benjamin Kramer 
>>>>>>> <benny.kra at gmail.com> wrote:
>>>>>>>
>>>>>>> On 11.01.2013, at 07:36, ????????? (Wei-Ren Chen) 
>>>>>>> <chenwj at iis.sinica.edu.tw> wrote:
>>>>>>>
>>>>>>>> Hi Pawel,
>>>>>>>>
>>>>>>>> PTX already be replaced with NVPTX. However,
PTX
>>>>>>>> subdirectory still sit in lib/Target in 3.2
release. Do
>>>>>>>> you think update the release tarball is a good
idea? Also
>>>>>>>> could you remove it from the trunk?
>>>>>>>
>>>>>>> Please do not, under no circumstances, change the
3.2
>>>>>>> release tarballs at this point. They are mirrored
around
>>>>>>> the world now with cryptographic hashes and
signatures.
>>>>>>> Changing them will break things for many people,
especially
>>>>>>> for an extremely minor thing like an empty
directory.
>>>>>>>
>>>>>>> I'm not sure if Pawel's tarball change
should be reverted
>>>>>>> now as it already caused uproar, so changing it
back might
>>>>>>> only make matters worse.
>>>>>>>
>>>>>>> The tarballs were changed?
>>>>>>
>>>>>> r172208
>>>>>
>>>>> I finally updated the FreeBSD ports yesterday and today a
user
>>>>>  complained about distfile changes.  IMO, this revision
should
>>>>> be reverted or all the other BSDs will have to chase
checksums
>>>>> as well.
>>>>>
>>>>> If you really want to remove the directory, ship a 3.2.1
>>>>> tarball rather than screwing all the downstream consumers
who's
>>>>>  infrastructure exists to detect trojan'd tarballs.
>>>>
>>>> Tarball is signed, it is not trjoan. Your infrastructure should
>>>> be able to deal with it?
>>>
>>> The FreeBSD ports collection maintains a set of sha256 hashes for 
>>> each distfile.  The system can deal with them changing, but
it's
>>> an inconvenience to port maintainers and users.  Even if we did
>>> have infrastructure to verify the signatures we'd still have to
>>> check the contents and would not just trust the signature since
>>> there's always a risk of keys being compromised.
>>>
>>> -- Brooks
>>>
>>
>>  At first your reaction has puzzled me, but despite a few fire
>> breathing dragons getting loose and after carefully re-reading
>> these posts over the weekend I think I understand the underlying
>> issue.
>>
>>  Simply put we both take this release seriously.
>>
>>  The problem is the release process is not entirely
>> defined. Which is reflected in question from David Blaikie
>> (the other post I am quoting here).
>>
>>> Which release process document are you referring to that indicates
>>> that?
>>>
>>
>>  Simple answer is the one on the llvm.org website.
>> But the real answer is, it all depends on how you read it.
>> Apparently, we read it differently which is not that
>> surprising as even our well defined compilers have
>> a bit of a problem with undefined behavior.
>>
>> So what can we expect from a release process document?
>> My hope is that we have reached a sort of sequence
>> point and the ambiguity of the release process
>> can be resolved despite all the side effects.
>>
>>
>> David also stated this:
>>
>>> Generally, so far as I know, this kind of post-release change is
>>> unprecedented.
>>
>>  I agree with this conclusion but even more unprecedented
>> is to a expect timely and problem free release with zero and
>> I'll repeat again zero involvement from the wider community that
>> depends on the clang+llvm!
>>  This is how the 3.2 release has happened, just compare
>> the list of 3.2 to 3.1 binaries and BTW not a single
>> 3.2 binary has been produced with the help of a respective
>> project, distribution or who ever.
>>
>>  I hear the cries of your wounded infrastructures
>> and I understand that in the days of forged root certificates
>> my signature on the tarballs may not guarantee much of a
>> security.
>>
>>  But why didn't you get involved in the 3.2 release?
>>
>>  We could have resolved all this in the 2 months since
>> the 3.2 release got rolling. Everybody knew about the
>> changes for the 3.2 release, simple e-mail to the new
>> LLVMRM would go a long way to resolve any security or
>> other release related issues not to mention a bit
>> of a courtesy.
> 
> I don't understand how my lack of involvement with this release has
> anything to do with your decision to fix what appears to me to be
> a non-problem in the release tarball well after the release and in
> the process do something (replacing release tarballs) that is quite
> simply NOT DONE to by projects who wish to treat their downstreams
> with respect.  This is something many projects end up learning due to
> the painful way so it's not something you should feel bad about as
> long as the project learns the right lesson(s).
Yessir! - lesson learned. Right one? Depends on a point of view.

Seems that downstream might also learn a thing or two about
participating in the release, and it does not have that painful.

> 
> If anything I'd be more annoyed if I'd actually gotten the port
updated
> sooner since then it would have worked for more than a day.  As it
> is, I could really use a final decision on which tarball will be at
> http://www.llvm.org/releases/3.2/llvm-3.2.src.tar.gz in perpetuity so I
> can change the port or not as required and avoid churn.
If you are in a hurry check with Tanya or Bill.
> 
> -- Brooks
> 
>>
>>  I am doing this as service to the LLVM community
>> but frankly I'd rather be chasing my dragons of
>> probability now.
>>
Paweł

> If you are in a hurry check with Tanya or Bill.
Tanya explicitly asked you to revert everything. I'm going to do this
for you asap.

Stay tuned

--
With best regards, Anton Korobeynikov
Faculty of Mathematics and Mechanics, Saint Petersburg State University

On Mon, Jan 14, 2013 at 01:41:06PM -0600, Pawel Wodnicki
wrote:
> On 1/14/2013 12:40 PM, Brooks Davis wrote:
> > On Sun, Jan 13, 2013 at 01:00:55PM -0600, Pawel Wodnicki wrote:
> >> Brooks,
> >>
> >>> On Fri, Jan 11, 2013 at 02:47:01PM -0600, Pawel Wodnicki
wrote:
> >>>> On 1/11/2013 2:40 PM, Brooks Davis wrote:
> >>>>> On Fri, Jan 11, 2013 at 09:33:17PM +0100, Benjamin
Kramer
> >>>>> wrote:
> >>>>>>
> >>>>>> On 11.01.2013, at 21:31, Justin Holewinski 
> >>>>>> <justin.holewinski at gmail.com> wrote:
> >>>>>>
> >>>>>>> On Fri, Jan 11, 2013 at 3:26 PM, Benjamin
Kramer
> >>>>>>> <benny.kra at gmail.com> wrote:
> >>>>>>>
> >>>>>>> On 11.01.2013, at 07:36, ????????? (Wei-Ren
Chen)
> >>>>>>> <chenwj at iis.sinica.edu.tw> wrote:
> >>>>>>>
> >>>>>>>> Hi Pawel,
> >>>>>>>>
> >>>>>>>> PTX already be replaced with NVPTX.
However, PTX
> >>>>>>>> subdirectory still sit in lib/Target in
3.2 release. Do
> >>>>>>>> you think update the release tarball is a
good idea? Also
> >>>>>>>> could you remove it from the trunk?
> >>>>>>>
> >>>>>>> Please do not, under no circumstances, change
the 3.2
> >>>>>>> release tarballs at this point. They are
mirrored around
> >>>>>>> the world now with cryptographic hashes and
signatures.
> >>>>>>> Changing them will break things for many
people, especially
> >>>>>>> for an extremely minor thing like an empty
directory.
> >>>>>>>
> >>>>>>> I'm not sure if Pawel's tarball change
should be reverted
> >>>>>>> now as it already caused uproar, so changing
it back might
> >>>>>>> only make matters worse.
> >>>>>>>
> >>>>>>> The tarballs were changed?
> >>>>>>
> >>>>>> r172208
> >>>>>
> >>>>> I finally updated the FreeBSD ports yesterday and
today a user
> >>>>>  complained about distfile changes.  IMO, this
revision should
> >>>>> be reverted or all the other BSDs will have to chase
checksums
> >>>>> as well.
> >>>>>
> >>>>> If you really want to remove the directory, ship a
3.2.1
> >>>>> tarball rather than screwing all the downstream
consumers who's
> >>>>>  infrastructure exists to detect trojan'd
tarballs.
> >>>>
> >>>> Tarball is signed, it is not trjoan. Your infrastructure
should
> >>>> be able to deal with it?
> >>>
> >>> The FreeBSD ports collection maintains a set of sha256 hashes
for
> >>> each distfile.  The system can deal with them changing, but
it's
> >>> an inconvenience to port maintainers and users.  Even if we
did
> >>> have infrastructure to verify the signatures we'd still
have to
> >>> check the contents and would not just trust the signature
since
> >>> there's always a risk of keys being compromised.
> >>>
> >>> -- Brooks
> >>>
> >>
> >>  At first your reaction has puzzled me, but despite a few fire
> >> breathing dragons getting loose and after carefully re-reading
> >> these posts over the weekend I think I understand the underlying
> >> issue.
> >>
> >>  Simply put we both take this release seriously.
> >>
> >>  The problem is the release process is not entirely
> >> defined. Which is reflected in question from David Blaikie
> >> (the other post I am quoting here).
> >>
> >>> Which release process document are you referring to that
indicates
> >>> that?
> >>>
> >>
> >>  Simple answer is the one on the llvm.org website.
> >> But the real answer is, it all depends on how you read it.
> >> Apparently, we read it differently which is not that
> >> surprising as even our well defined compilers have
> >> a bit of a problem with undefined behavior.
> >>
> >> So what can we expect from a release process document?
> >> My hope is that we have reached a sort of sequence
> >> point and the ambiguity of the release process
> >> can be resolved despite all the side effects.
> >>
> >>
> >> David also stated this:
> >>
> >>> Generally, so far as I know, this kind of post-release change
is
> >>> unprecedented.
> >>
> >>  I agree with this conclusion but even more unprecedented
> >> is to a expect timely and problem free release with zero and
> >> I'll repeat again zero involvement from the wider community
that
> >> depends on the clang+llvm!
> >>  This is how the 3.2 release has happened, just compare
> >> the list of 3.2 to 3.1 binaries and BTW not a single
> >> 3.2 binary has been produced with the help of a respective
> >> project, distribution or who ever.
> >>
> >>  I hear the cries of your wounded infrastructures
> >> and I understand that in the days of forged root certificates
> >> my signature on the tarballs may not guarantee much of a
> >> security.
> >>
> >>  But why didn't you get involved in the 3.2 release?
> >>
> >>  We could have resolved all this in the 2 months since
> >> the 3.2 release got rolling. Everybody knew about the
> >> changes for the 3.2 release, simple e-mail to the new
> >> LLVMRM would go a long way to resolve any security or
> >> other release related issues not to mention a bit
> >> of a courtesy.
> > 
> > I don't understand how my lack of involvement with this release
has
> > anything to do with your decision to fix what appears to me to be
> > a non-problem in the release tarball well after the release and in
> > the process do something (replacing release tarballs) that is quite
> > simply NOT DONE to by projects who wish to treat their downstreams
> > with respect.  This is something many projects end up learning due to
> > the painful way so it's not something you should feel bad about as
> > long as the project learns the right lesson(s).
> 
> Yessir! - lesson learned. Right one? Depends on a point of view.
> 
> Seems that downstream might also learn a thing or two about
> participating in the release, and it does not have that painful.
On the FreeBSD ports side, I certainly could have done better, but
was unable to keep LLVM interactions high enough on my priority list.
Unfortunately, the release cycle seems to be well synchronized with the
schedule of meetings that require me to focus almost exclusively on
hacking demos.  The version in the base system saw considerable testing
as it was integrated into FreeBSD at a couple points.

Thanks for all your work on the 3.2 release.  When all is said and done
I'm quite happy with it.

-- Brooks
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130114/8364d27a/attachment.sig>