Philip Reames via llvm-dev
2015-Sep-08 16:07 UTC
[llvm-dev] Testing "normal" cross-compilers versus GPU backends
On 09/04/2015 10:25 AM, Robinson, Paul via llvm-dev wrote:> >> -----Original Message----- >> From: James Y Knight [mailto:jyknight at google.com] >> Sent: Friday, September 04, 2015 10:12 AM >> To: Mehdi Amini; Mehdi Amini via llvm-dev >> Cc: Robinson, Paul >> Subject: Re: [llvm-dev] Testing "normal" cross-compilers versus GPU >> backends >> >>>>> The Hexagon precedent is interesting; Krzysztof said they set the >> default >>>>> triple, and didn't have to xfail all that much stuff. Searching the >> tree, >>>>> I find exactly 7 individual tests marked XFAIL: hexagon, plus it >> disables >>>>> all of ExecutionEngine, and turns off the 'object-emission' feature. >>>>> >>>>> I'm curious if you would try setting the default triple to match your >>>>> target, and see what /kinds/ of tests fail. The raw number is much >> less >>>>> interesting than in the categories. >>>> Failing tests attached, let me know which ones you’d like me to >> investigate. >>> Tests: >>> <failing_tests> >>> >>> (note I forced enable the “native” feature on this run) >> So, just looking at the first one in that list for an example, running: >> llc -mtriple amdgcn test/CodeGen/Generic/2002-04-14- >> UnexpectedUnsignedType.ll >> you get an error of: >> error: unsupported call to function bar in foo >> >> ..because apparently AMDGPU simply doesn't support function calls. At all. >> >> That is a rather unusual feature to be missing, and I'd imagine is causing >> a fair number of the other nominally-generic tests to fail too... > Hmm I'm told GPUs as a class typically don't support calls (everything has > to be inlined). > > Perhaps the more appropriate direction here is to invent a "gpu" config flag > and mark some things XFAIL: gpu. > --paulrMehdi - Could you list a couple of the features that show up in generic tests which your target doesn't support by design? After reading through the discussion to this point, I don't really have a feel for that. Additionally, do you have any sense for how common that set is across GPU targets? Paul - If we introduce the type of flag your talking about, we should probably have a discussion about which classes of backends LLVM supports and try to create a *documented* list of feature expectations somewhere. p.s. Maybe I missed something, but why did simply using "XFAIL: AMDGPU" get rejected? That seems like the easiest fix. Philip
Mehdi Amini via llvm-dev
2015-Sep-08 23:04 UTC
[llvm-dev] Testing "normal" cross-compilers versus GPU backends
Hi Philip,> On Sep 8, 2015, at 9:07 AM, Philip Reames <listmail at philipreames.com> wrote: > > On 09/04/2015 10:25 AM, Robinson, Paul via llvm-dev wrote: >> >>> -----Original Message----- >>> From: James Y Knight [mailto:jyknight at google.com] >>> Sent: Friday, September 04, 2015 10:12 AM >>> To: Mehdi Amini; Mehdi Amini via llvm-dev >>> Cc: Robinson, Paul >>> Subject: Re: [llvm-dev] Testing "normal" cross-compilers versus GPU >>> backends >>> >>>>>> The Hexagon precedent is interesting; Krzysztof said they set the >>> default >>>>>> triple, and didn't have to xfail all that much stuff. Searching the >>> tree, >>>>>> I find exactly 7 individual tests marked XFAIL: hexagon, plus it >>> disables >>>>>> all of ExecutionEngine, and turns off the 'object-emission' feature. >>>>>> >>>>>> I'm curious if you would try setting the default triple to match your >>>>>> target, and see what /kinds/ of tests fail. The raw number is much >>> less >>>>>> interesting than in the categories. >>>>> Failing tests attached, let me know which ones you’d like me to >>> investigate. >>>> Tests: >>>> <failing_tests> >>>> >>>> (note I forced enable the “native” feature on this run) >>> So, just looking at the first one in that list for an example, running: >>> llc -mtriple amdgcn test/CodeGen/Generic/2002-04-14- >>> UnexpectedUnsignedType.ll >>> you get an error of: >>> error: unsupported call to function bar in foo >>> >>> ..because apparently AMDGPU simply doesn't support function calls. At all. >>> >>> That is a rather unusual feature to be missing, and I'd imagine is causing >>> a fair number of the other nominally-generic tests to fail too... >> Hmm I'm told GPUs as a class typically don't support calls (everything has >> to be inlined). >> >> Perhaps the more appropriate direction here is to invent a "gpu" config flag >> and mark some things XFAIL: gpu. >> --paulr > Mehdi - Could you list a couple of the features that show up in generic tests which your target doesn't support by design? After reading through the discussion to this point, I don't really have a feel for that. Additionally, do you have any sense for how common that set is across GPU targets?This is what I was referring to in my last email: "To be “correct”, tests should “require” (in the lit sense) features like “function call”, etc. However this is a significant amount of work with not that much added value. This is what Takumi did to disable one test that check for “debug_frame” which are not emitted on Windows.” I could try to survey all the failing tests and abstract the “features” they are using, but this does not seem like a small enough task that I can commit to an ETA now or have an estimate about when it’ll come on top of my stack. This is why I tried to offer an alternative with http://reviews.llvm.org/D12660 ; to identify the tests that rely on the default triple using a generic default_triple feature.> Paul - If we introduce the type of flag your talking about, we should probably have a discussion about which classes of backends LLVM supports and try to create a *documented* list of feature expectations somewhere. > > p.s. Maybe I missed something, but why did simply using "XFAIL: AMDGPU" get rejected? That seems like the easiest fix.You haven't missed anything IIUC. If we were to document the list of feature expectations from the backends, the “default_triple” would be applied to the tests that makes use of these features. — Mehdi
Robinson, Paul via llvm-dev
2015-Sep-08 23:56 UTC
[llvm-dev] Testing "normal" cross-compilers versus GPU backends
> > Mehdi - Could you list a couple of the features that show up in generic > tests which your target doesn't support by design? After reading through > the discussion to this point, I don't really have a feel for that. > Additionally, do you have any sense for how common that set is across GPU > targets? > > This is what I was referring to in my last email: > "To be “correct”, tests should “require” (in the lit sense) features like > “function call”, etc. However this is a significant amount of work with > not that much added value. > This is what Takumi did to disable one test that check for “debug_frame” > which are not emitted on Windows.” > > I could try to survey all the failing tests and abstract the “features” > they are using, but this does not seem like a small enough task that I can > commit to an ETA now or have an estimate about when it’ll come on top of > my stack. > > This is why I tried to offer an alternative with > http://reviews.llvm.org/D12660 ; to identify the tests that rely on the > default triple using a generic default_triple feature. > > > > Paul - If we introduce the type of flag your talking about, we should > probably have a discussion about which classes of backends LLVM supports > and try to create a *documented* list of feature expectations somewhere. > > > > p.s. Maybe I missed something, but why did simply using "XFAIL: AMDGPU" > get rejected? That seems like the easiest fix. > > You haven't missed anything IIUC. > If we were to document the list of feature expectations from the backends, > the “default_triple” would be applied to the tests that makes use of these > features. > > — > MehdiHmmm "default_triple" is certainly not an expectation or feature of any target; rather, it is an expectation of certain tools when used in certain ways (mainly: llc or llvm-mc without an explicit triple) and therefore it is an expectation of how certain tests are _written_. This has little to do with the _features_ verified by those tests. In particular, configuring a GPU-only LLVM with a default triple for that GPU will still fail all the same tests that Mehdi tripped over. Inventing "default_triple" lets us trick the test suite into skipping the tests that the GPU target would fail anyway. This is not to say "default_triple" is a bad thing; it lets us support a kind of configuration that Mehdi has persuaded me is valid, and I'm happy to have it. But it is not a target-feature-based solution. It is not impossible that if we defined a "GPU" feature set, we might find that it would exclude many of the same tests that "default_triple" excludes. But that would be essentially by chance, not because "default_triple" has any inherent meaning for targets. --paulr
Philip Reames via llvm-dev
2015-Sep-09 20:37 UTC
[llvm-dev] Testing "normal" cross-compilers versus GPU backends
On 09/08/2015 04:04 PM, Mehdi Amini wrote:> Hi Philip, > > >> On Sep 8, 2015, at 9:07 AM, Philip Reames <listmail at philipreames.com> wrote: >> >> On 09/04/2015 10:25 AM, Robinson, Paul via llvm-dev wrote: >>>> -----Original Message----- >>>> From: James Y Knight [mailto:jyknight at google.com] >>>> Sent: Friday, September 04, 2015 10:12 AM >>>> To: Mehdi Amini; Mehdi Amini via llvm-dev >>>> Cc: Robinson, Paul >>>> Subject: Re: [llvm-dev] Testing "normal" cross-compilers versus GPU >>>> backends >>>> >>>>>>> The Hexagon precedent is interesting; Krzysztof said they set the >>>> default >>>>>>> triple, and didn't have to xfail all that much stuff. Searching the >>>> tree, >>>>>>> I find exactly 7 individual tests marked XFAIL: hexagon, plus it >>>> disables >>>>>>> all of ExecutionEngine, and turns off the 'object-emission' feature. >>>>>>> >>>>>>> I'm curious if you would try setting the default triple to match your >>>>>>> target, and see what /kinds/ of tests fail. The raw number is much >>>> less >>>>>>> interesting than in the categories. >>>>>> Failing tests attached, let me know which ones you’d like me to >>>> investigate. >>>>> Tests: >>>>> <failing_tests> >>>>> >>>>> (note I forced enable the “native” feature on this run) >>>> So, just looking at the first one in that list for an example, running: >>>> llc -mtriple amdgcn test/CodeGen/Generic/2002-04-14- >>>> UnexpectedUnsignedType.ll >>>> you get an error of: >>>> error: unsupported call to function bar in foo >>>> >>>> ..because apparently AMDGPU simply doesn't support function calls. At all. >>>> >>>> That is a rather unusual feature to be missing, and I'd imagine is causing >>>> a fair number of the other nominally-generic tests to fail too... >>> Hmm I'm told GPUs as a class typically don't support calls (everything has >>> to be inlined). >>> >>> Perhaps the more appropriate direction here is to invent a "gpu" config flag >>> and mark some things XFAIL: gpu. >>> --paulr >> Mehdi - Could you list a couple of the features that show up in generic tests which your target doesn't support by design? After reading through the discussion to this point, I don't really have a feel for that. Additionally, do you have any sense for how common that set is across GPU targets? > This is what I was referring to in my last email: > "To be “correct”, tests should “require” (in the lit sense) features like “function call”, etc. However this is a significant amount of work with not that much added value. > This is what Takumi did to disable one test that check for “debug_frame” which are not emitted on Windows.” > > I could try to survey all the failing tests and abstract the “features” they are using, but this does not seem like a small enough task that I can commit to an ETA now or have an estimate about when it’ll come on top of my stack. > > This is why I tried to offer an alternative with http://reviews.llvm.org/D12660 ; to identify the tests that rely on the default triple using a generic default_triple feature.Mehdi - You're answer focuses on implementation details and a complete list. That's not what I was getting at. The *only* concrete feature mentioned so far has been function calls. All I'm looking for is this a problem specific to function calls? Or are there a couple of other known features that your backend just doesn't support? This is purely so that I can have a chance of understanding the discussion. I'm not asking for an exhaustive list or anything.> > >> Paul - If we introduce the type of flag your talking about, we should probably have a discussion about which classes of backends LLVM supports and try to create a *documented* list of feature expectations somewhere. >> >> p.s. Maybe I missed something, but why did simply using "XFAIL: AMDGPU" get rejected? That seems like the easiest fix. > You haven't missed anything IIUC. > If we were to document the list of feature expectations from the backends, the “default_triple” would be applied to the tests that makes use of these features.How many tests would you have to add XFAILs to if we went with the simple approach? Ballpark number is sufficient; I just want an idea of the magnitude.> > — > Mehdi >
Maybe Matching Threads
- Testing "normal" cross-compilers versus GPU backends
- How is target_triple/default_triple handled in tests?
- Testing "normal" cross-compilers versus GPU backends
- Testing "normal" cross-compilers versus GPU backends
- Testing "normal" cross-compilers versus GPU backends