On Tue, Mar 22, 2016 at 7:36 PM, Rui Ueyama <ruiu at google.com> wrote:> I have a question. If there is a ELF verifier function that walks every > part of an ELF file to verify that the file is sane, and if you can call > that before calling LLD's function, are you guys happy with that? >I'd like to get you guys opinion on this question.> On Tue, Mar 22, 2016 at 6:39 PM, Hal Finkel via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> >> ------------------------------ >> >> *From: *"David Blaikie via llvm-dev" <llvm-dev at lists.llvm.org> >> *To: *"Rafael Espíndola" <rafael.espindola at gmail.com> >> *Cc: *"llvm-dev" <llvm-dev at lists.llvm.org>, "Bruce Hoult" < >> bruce at hoult.org> >> *Sent: *Tuesday, March 22, 2016 10:18:03 AM >> *Subject: *Re: [llvm-dev] Need help with code generation >> >> >> >> On Tue, Mar 22, 2016 at 4:27 AM, Rafael Espíndola < >> llvm-dev at lists.llvm.org> wrote: >> >>> > Maybe not, but it's not impossible either - browsers manage to harden >>> themselves against malicious input and they operate in a far hostile >>> environment with many more input formats than we do. >>> >>> It is important to note how different they are. Both Firefox and >>> Chromium have people working just to try to make them more secure. >>> Compare that with LLVM: One week ago I pointed out that your patch >>> (r263521) introduces a crash. It still hasn't been reverted or even >>> acknowledge yet. >>> >>> >>> > I'm not trying to shift your personal goal, or to direct the features >>> that you choose to put your time into, but I am interested in project >>> policy. >>> >>> Why do you care about policy that is not followed? A policy saying >>> llvm should not crash on any input is as relevant as one that says >>> that clang should keep bootstrapping in under one second. >>> >> >> It's pretty different when you say, essentially, that patches to address >> these things are unlikely to be accepted. It doesn't seem surprising that >> people wouldn't try to provide those patches and would choose not to use >> the project if that's the expressed policy of the developers on the project >> and doesn't line up with the needs of other people. >> >> >> +1 >> >> -Hal >> >> >> >>> >>> So, if we stick to reality, what we have is that lld (ELF and COFF) >>> are already the most reliable parts of the toolchain. If not for Rui >>> and I being upfront about it most people would not even know that you >>> could crash it. So please, just let us keep working on the most >>> reliable part of the toolchain. >>> >>> Cheers, >>> Rafael >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> >> >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> >> >> >> -- >> Hal Finkel >> Assistant Computational Scientist >> Leadership Computing Facility >> Argonne National Laboratory >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160322/d5cca713/attachment.html>
----- Original Message -----> From: "Rui Ueyama" <ruiu at google.com> > To: "Hal Finkel" <hfinkel at anl.gov> > Cc: "David Blaikie" <dblaikie at gmail.com>, "llvm-dev" > <llvm-dev at lists.llvm.org>, "Bruce Hoult" <bruce at hoult.org> > Sent: Tuesday, March 22, 2016 2:36:34 PM > Subject: Re: [llvm-dev] Need help with code generation> On Tue, Mar 22, 2016 at 7:36 PM, Rui Ueyama < ruiu at google.com > > wrote:> > I have a question. If there is a ELF verifier function that walks > > every part of an ELF file to verify that the file is sane, and if > > you can call that before calling LLD's function, are you guys happy > > with that? > > I'd like to get you guys opinion on this question.I'll echo Rafael here. What does "sane" mean? If I define sane to mean, "will not cause lld to exhibit undefined behavior if later run over the same input", then this seems like the most efficient way of satisfying that goal (perhaps staged to avoid cache thrashing), and I'd like to know how much overhead it would add to run it in lld by default (even if we have an option to disable it for absolute speed). Thanks again, Hal> > On Tue, Mar 22, 2016 at 6:39 PM, Hal Finkel via llvm-dev < > > llvm-dev at lists.llvm.org > wrote: >> > > > From: "David Blaikie via llvm-dev" < llvm-dev at lists.llvm.org > > > > > > > > > > > To: "Rafael Espíndola" < rafael.espindola at gmail.com > > > > > > > > > > > Cc: "llvm-dev" < llvm-dev at lists.llvm.org >, "Bruce Hoult" < > > > > bruce at hoult.org > > > > > > > > > > > Sent: Tuesday, March 22, 2016 10:18:03 AM > > > > > > > > > > Subject: Re: [llvm-dev] Need help with code generation > > > > > >> > > > On Tue, Mar 22, 2016 at 4:27 AM, Rafael Espíndola < > > > > llvm-dev at lists.llvm.org > wrote: > > > > > >> > > > > > Maybe not, but it's not impossible either - browsers manage > > > > > > to > > > > > > harden themselves against malicious input and they operate > > > > > > in > > > > > > a > > > > > > far hostile environment with many more input formats than > > > > > > we > > > > > > do. > > > > > > > > > >> > > > > It is important to note how different they are. Both Firefox > > > > > and > > > > > > > > > > > > > > > Chromium have people working just to try to make them more > > > > > secure. > > > > > > > > > > > > > > > Compare that with LLVM: One week ago I pointed out that your > > > > > patch > > > > > > > > > > > > > > > (r263521) introduces a crash. It still hasn't been reverted > > > > > or > > > > > even > > > > > > > > > > > > > > > acknowledge yet. > > > > > > > > > >> > > > > > I'm not trying to shift your personal goal, or to direct > > > > > > the > > > > > > features that you choose to put your time into, but I am > > > > > > interested in project policy. > > > > > > > > > >> > > > > Why do you care about policy that is not followed? A policy > > > > > saying > > > > > > > > > > > > > > > llvm should not crash on any input is as relevant as one that > > > > > says > > > > > > > > > > > > > > > that clang should keep bootstrapping in under one second. > > > > > > > > > >> > > > It's pretty different when you say, essentially, that patches > > > > to > > > > address these things are unlikely to be accepted. It doesn't > > > > seem > > > > surprising that people wouldn't try to provide those patches > > > > and > > > > would choose not to use the project if that's the expressed > > > > policy > > > > of the developers on the project and doesn't line up with the > > > > needs > > > > of other people. > > > > > > > > > +1 > > >> > > -Hal > > >> > > > > So, if we stick to reality, what we have is that lld (ELF and > > > > > COFF) > > > > > > > > > > > > > > > are already the most reliable parts of the toolchain. If not > > > > > for > > > > > Rui > > > > > > > > > > > > > > > and I being upfront about it most people would not even know > > > > > that > > > > > you > > > > > > > > > > > > > > > could crash it. So please, just let us keep working on the > > > > > most > > > > > > > > > > > > > > > reliable part of the toolchain. > > > > > > > > > >> > > > > Cheers, > > > > > > > > > > > > > > > Rafael > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > > > > > LLVM Developers mailing list > > > > > > > > > > > > > > > llvm-dev at lists.llvm.org > > > > > > > > > > > > > > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > > > > > > > > >> > > > _______________________________________________ > > > > > > > > > > LLVM Developers mailing list > > > > > > > > > > llvm-dev at lists.llvm.org > > > > > > > > > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > > > > >> > > -- > > >> > > Hal Finkel > > > > > > Assistant Computational Scientist > > > > > > Leadership Computing Facility > > > > > > Argonne National Laboratory > > >> > > _______________________________________________ > > > > > > LLVM Developers mailing list > > > > > > llvm-dev at lists.llvm.org > > > > > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > >-- Hal Finkel Assistant Computational Scientist Leadership Computing Facility Argonne National Laboratory -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160322/a0077f93/attachment.html>
On Tue, Mar 22, 2016 at 8:40 PM, Hal Finkel <hfinkel at anl.gov> wrote:> > ------------------------------ > > *From: *"Rui Ueyama" <ruiu at google.com> > *To: *"Hal Finkel" <hfinkel at anl.gov> > *Cc: *"David Blaikie" <dblaikie at gmail.com>, "llvm-dev" < > llvm-dev at lists.llvm.org>, "Bruce Hoult" <bruce at hoult.org> > *Sent: *Tuesday, March 22, 2016 2:36:34 PM > *Subject: *Re: [llvm-dev] Need help with code generation > > On Tue, Mar 22, 2016 at 7:36 PM, Rui Ueyama <ruiu at google.com> wrote: > >> I have a question. If there is a ELF verifier function that walks every >> part of an ELF file to verify that the file is sane, and if you can call >> that before calling LLD's function, are you guys happy with that? >> > > I'd like to get you guys opinion on this question. > > > I'll echo Rafael here. What does "sane" mean? If I define sane to mean, > "will not cause lld to exhibit undefined behavior if later run over the > same input", then this seems like the most efficient way of satisfying that > goal (perhaps staged to avoid cache thrashing), and I'd like to know how > much overhead it would add to run it in lld by default (even if we have an > option to disable it for absolute speed). >ELF is a documented file format so if you are not sure if something should be considered valid, you can take a look at the spec to determine whether it is valid or not. File validity is an independent concept from LLD and I think we can determine it according to the spec. I have a different opinion about how it could be implemented. We have no idea if it'd be faster if it is "embedded" to LLD, at least. And usually separating passes leads to cleaner and more readable code. Thanks again,> Hal > > > > >> On Tue, Mar 22, 2016 at 6:39 PM, Hal Finkel via llvm-dev < >> llvm-dev at lists.llvm.org> wrote: >> >>> >>> ------------------------------ >>> >>> *From: *"David Blaikie via llvm-dev" <llvm-dev at lists.llvm.org> >>> *To: *"Rafael Espíndola" <rafael.espindola at gmail.com> >>> *Cc: *"llvm-dev" <llvm-dev at lists.llvm.org>, "Bruce Hoult" < >>> bruce at hoult.org> >>> *Sent: *Tuesday, March 22, 2016 10:18:03 AM >>> *Subject: *Re: [llvm-dev] Need help with code generation >>> >>> >>> >>> On Tue, Mar 22, 2016 at 4:27 AM, Rafael Espíndola < >>> llvm-dev at lists.llvm.org> wrote: >>> >>>> > Maybe not, but it's not impossible either - browsers manage to harden >>>> themselves against malicious input and they operate in a far hostile >>>> environment with many more input formats than we do. >>>> >>>> It is important to note how different they are. Both Firefox and >>>> Chromium have people working just to try to make them more secure. >>>> Compare that with LLVM: One week ago I pointed out that your patch >>>> (r263521) introduces a crash. It still hasn't been reverted or even >>>> acknowledge yet. >>>> >>>> >>>> > I'm not trying to shift your personal goal, or to direct the features >>>> that you choose to put your time into, but I am interested in project >>>> policy. >>>> >>>> Why do you care about policy that is not followed? A policy saying >>>> llvm should not crash on any input is as relevant as one that says >>>> that clang should keep bootstrapping in under one second. >>>> >>> >>> It's pretty different when you say, essentially, that patches to address >>> these things are unlikely to be accepted. It doesn't seem surprising that >>> people wouldn't try to provide those patches and would choose not to use >>> the project if that's the expressed policy of the developers on the project >>> and doesn't line up with the needs of other people. >>> >>> >>> +1 >>> >>> -Hal >>> >>> >>> >>>> >>>> So, if we stick to reality, what we have is that lld (ELF and COFF) >>>> are already the most reliable parts of the toolchain. If not for Rui >>>> and I being upfront about it most people would not even know that you >>>> could crash it. So please, just let us keep working on the most >>>> reliable part of the toolchain. >>>> >>>> Cheers, >>>> Rafael >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> llvm-dev at lists.llvm.org >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>> >>> >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> >>> >>> >>> >>> -- >>> Hal Finkel >>> Assistant Computational Scientist >>> Leadership Computing Facility >>> Argonne National Laboratory >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> >>> >> > > > > -- > Hal Finkel > Assistant Computational Scientist > Leadership Computing Facility > Argonne National Laboratory >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160322/83be663f/attachment.html>
David Blaikie via llvm-dev
2016-Mar-22 20:00 UTC
[llvm-dev] Need help with code generation
On Tue, Mar 22, 2016 at 12:36 PM, Rui Ueyama <ruiu at google.com> wrote:> On Tue, Mar 22, 2016 at 7:36 PM, Rui Ueyama <ruiu at google.com> wrote: > >> I have a question. If there is a ELF verifier function that walks every >> part of an ELF file to verify that the file is sane, and if you can call >> that before calling LLD's function, are you guys happy with that? >> > > I'd like to get you guys opinion on this question. >I'd still find it problematic that lld itself would consider crash-on-invalid "not a bug" to the point of not reviewing/approving patches to fix such issues. That's what I'm concerned about in this thread.> > >> On Tue, Mar 22, 2016 at 6:39 PM, Hal Finkel via llvm-dev < >> llvm-dev at lists.llvm.org> wrote: >> >>> >>> ------------------------------ >>> >>> *From: *"David Blaikie via llvm-dev" <llvm-dev at lists.llvm.org> >>> *To: *"Rafael Espíndola" <rafael.espindola at gmail.com> >>> *Cc: *"llvm-dev" <llvm-dev at lists.llvm.org>, "Bruce Hoult" < >>> bruce at hoult.org> >>> *Sent: *Tuesday, March 22, 2016 10:18:03 AM >>> *Subject: *Re: [llvm-dev] Need help with code generation >>> >>> >>> >>> On Tue, Mar 22, 2016 at 4:27 AM, Rafael Espíndola < >>> llvm-dev at lists.llvm.org> wrote: >>> >>>> > Maybe not, but it's not impossible either - browsers manage to harden >>>> themselves against malicious input and they operate in a far hostile >>>> environment with many more input formats than we do. >>>> >>>> It is important to note how different they are. Both Firefox and >>>> Chromium have people working just to try to make them more secure. >>>> Compare that with LLVM: One week ago I pointed out that your patch >>>> (r263521) introduces a crash. It still hasn't been reverted or even >>>> acknowledge yet. >>>> >>>> >>>> > I'm not trying to shift your personal goal, or to direct the features >>>> that you choose to put your time into, but I am interested in project >>>> policy. >>>> >>>> Why do you care about policy that is not followed? A policy saying >>>> llvm should not crash on any input is as relevant as one that says >>>> that clang should keep bootstrapping in under one second. >>>> >>> >>> It's pretty different when you say, essentially, that patches to address >>> these things are unlikely to be accepted. It doesn't seem surprising that >>> people wouldn't try to provide those patches and would choose not to use >>> the project if that's the expressed policy of the developers on the project >>> and doesn't line up with the needs of other people. >>> >>> >>> +1 >>> >>> -Hal >>> >>> >>> >>>> >>>> So, if we stick to reality, what we have is that lld (ELF and COFF) >>>> are already the most reliable parts of the toolchain. If not for Rui >>>> and I being upfront about it most people would not even know that you >>>> could crash it. So please, just let us keep working on the most >>>> reliable part of the toolchain. >>>> >>>> Cheers, >>>> Rafael >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> llvm-dev at lists.llvm.org >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>> >>> >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> >>> >>> >>> >>> -- >>> Hal Finkel >>> Assistant Computational Scientist >>> Leadership Computing Facility >>> Argonne National Laboratory >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> >>> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160322/8ae834ef/attachment.html>
On Tue, Mar 22, 2016 at 9:00 PM, David Blaikie <dblaikie at gmail.com> wrote:> > > On Tue, Mar 22, 2016 at 12:36 PM, Rui Ueyama <ruiu at google.com> wrote: > >> On Tue, Mar 22, 2016 at 7:36 PM, Rui Ueyama <ruiu at google.com> wrote: >> >>> I have a question. If there is a ELF verifier function that walks every >>> part of an ELF file to verify that the file is sane, and if you can call >>> that before calling LLD's function, are you guys happy with that? >>> >> >> I'd like to get you guys opinion on this question. >> > > I'd still find it problematic that lld itself would consider > crash-on-invalid "not a bug" to the point of not reviewing/approving > patches to fix such issues. That's what I'm concerned about in this thread. >That's one way to see that. The other view is it as a whole has a boolean option *IsInputTrustworthy* and works accordingly. What matters most is what we provide to the users as a guarantee. You have an opinion that that should be implemented within LLD, but that would now an internal design choice. Hypothetically we had such pass to verify inputs, and if you send a patch to "fix" crash bug of LLD, we wouldn't probably reject that but instead argue that that needs to be addressed in the verifier pass instead. This is about "how" something should be implemented and usual design choice discussion, no?>> >>> On Tue, Mar 22, 2016 at 6:39 PM, Hal Finkel via llvm-dev < >>> llvm-dev at lists.llvm.org> wrote: >>> >>>> >>>> ------------------------------ >>>> >>>> *From: *"David Blaikie via llvm-dev" <llvm-dev at lists.llvm.org> >>>> *To: *"Rafael Espíndola" <rafael.espindola at gmail.com> >>>> *Cc: *"llvm-dev" <llvm-dev at lists.llvm.org>, "Bruce Hoult" < >>>> bruce at hoult.org> >>>> *Sent: *Tuesday, March 22, 2016 10:18:03 AM >>>> *Subject: *Re: [llvm-dev] Need help with code generation >>>> >>>> >>>> >>>> On Tue, Mar 22, 2016 at 4:27 AM, Rafael Espíndola < >>>> llvm-dev at lists.llvm.org> wrote: >>>> >>>>> > Maybe not, but it's not impossible either - browsers manage to >>>>> harden themselves against malicious input and they operate in a far hostile >>>>> environment with many more input formats than we do. >>>>> >>>>> It is important to note how different they are. Both Firefox and >>>>> Chromium have people working just to try to make them more secure. >>>>> Compare that with LLVM: One week ago I pointed out that your patch >>>>> (r263521) introduces a crash. It still hasn't been reverted or even >>>>> acknowledge yet. >>>>> >>>>> >>>>> > I'm not trying to shift your personal goal, or to direct the >>>>> features that you choose to put your time into, but I am interested in >>>>> project policy. >>>>> >>>>> Why do you care about policy that is not followed? A policy saying >>>>> llvm should not crash on any input is as relevant as one that says >>>>> that clang should keep bootstrapping in under one second. >>>>> >>>> >>>> It's pretty different when you say, essentially, that patches to >>>> address these things are unlikely to be accepted. It doesn't seem >>>> surprising that people wouldn't try to provide those patches and would >>>> choose not to use the project if that's the expressed policy of the >>>> developers on the project and doesn't line up with the needs of other >>>> people. >>>> >>>> >>>> +1 >>>> >>>> -Hal >>>> >>>> >>>> >>>>> >>>>> So, if we stick to reality, what we have is that lld (ELF and COFF) >>>>> are already the most reliable parts of the toolchain. If not for Rui >>>>> and I being upfront about it most people would not even know that you >>>>> could crash it. So please, just let us keep working on the most >>>>> reliable part of the toolchain. >>>>> >>>>> Cheers, >>>>> Rafael >>>>> _______________________________________________ >>>>> LLVM Developers mailing list >>>>> llvm-dev at lists.llvm.org >>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>> >>>> >>>> >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> llvm-dev at lists.llvm.org >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>> >>>> >>>> >>>> >>>> -- >>>> Hal Finkel >>>> Assistant Computational Scientist >>>> Leadership Computing Facility >>>> Argonne National Laboratory >>>> >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> llvm-dev at lists.llvm.org >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>> >>>> >>> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160322/35f4e4ff/attachment-0001.html>