[LLVMdev] Bug in visitSIGN_EXTEND in DAGCombiner.cpp?

Relph, Richard
Fri, 08 Mar 2013 18:53:50 -0000

visitSIGN_EXTEND() in DAGCombiner.cpp generates an ISD::SELECT even if VT is a
vector, which causes ExpandSELECT() to assert during legalization.
I think what's required is to have visitSIGN_EXTEND generate a VSELECT if VT
is a vector…

I've tried a local change that cures this particular assert, but uncovers
another assert later, so I'm a bit uncertain if I'm heading off in the
wrong direction.
The subsequent assert comes in ExpandVSELECT, which expects (for no good reason
as far as I can tell) that the "mask" (really the 'selector')
and the operands will be the same width, rather than trying to make them the
same width (in our problem cases, the mask is an i32 produced from
getSetCCResultType() and the ops are i8 or i16 - all vectors of the same length,
of course.)

I don't really want to fix that, as the generated XOR/AND/AND/OR sequence
isn't desirable at all for our target, which has an instruction that I think
does precisely VSELECT…

Richard