Nouveau - Jul 2015 - [PATCH] Add Option "DRI3" to allow to disable DRI3 under EXA.

On 07/14/2015 05:26 PM, Ilia Mirkin wrote:
> On Mon, Jul 13, 2015 at 11:43 PM, Mario Kleiner
> <mario.kleiner.de at gmail.com> wrote:
>> On 07/07/2015 09:51 PM, Ilia Mirkin wrote:
>>>
>>> Lastly, from some discussions with ajax on IRC, it appears that
DRI3
>>> is half-baked at best wrt sync between server and client. I think
we
>>> should just disable it by default for now, until issues are ironed
>>> out. (Rather than what this patch has, which is default-on for Xorg
>
>>> some version.)
>>
>>
>> What are the remaining known trouble spots wrt. sync? It seems to work
>> pretty well at least for single gpu + unredirected fullscreen windows
(=>> kms page flipping can be used for Presents. That's the use case i
usually
>> test very obsessively, as it matters very much for my type of
applications,
>> but other than that i only lightly "test" it via regular
desktop use, so
>> maybe that's were problems remain?
>
> Adam is the one who actually understands it... I was just asking
> questions and the answer was "that's broken". A user was
using DRI3
> with EXA (nouveau DDX), and was seeing render fail in the form of
> stale things on the screen. The nouveau DDX does a lot of
"implicit"
> sync stuff... it just emits commands into a pushbuf without kicking
> it. However the libdrm code has cleverness to kick out any pushbufs if
> you do a nouveau_bo_wait and that bo has been referenced.
>
> HOWEVER if you have a pixmap and you share it using the fd thing, and
> another process makes a (shared) bo out of it, and then does a
> nouveau_bo_wait, that in no way will cause the DDX to kick its
> pushbuf. Apparently there's some sort of sync thing that's supposed
to
Although then i'd expect the other process to hang in nouveau_bo_wait? 
There probably isn't a specific fdo bug for this?
> happen, but that's entirely unimplemented for DRI3. Unfortunately I
> can't provide more details than that, as my knowledge of X internals
> is quite limited. Some sort of DRI drawable or GLX drawable or ...
> something.
>
> I have no proof that this is the cause of the issue the user was
> seeing, in fact it's just as likely to be something else. However this
> seems like a pretty significant issue to me.
>
Ok, thanks for the explanation. But would this problem then be limited 
to exa + nouveau? glamor uses opengl and thereby mesa's and i think mesa 
gets sync right afaik, at least in the glx/dri3 backend. Didn't look at 
egl backend though. At least i didn't see any such corruption?

I'll change the patch to default to DRI 2 then for exa, and glamor will 
ignore the DRI parameter anyway and stick to DRI3.
>>
>> We can disable it by default on exa - intel and amd/radeon drivers also
>> disable by default. However, on gpus >= maxwell only glamor accel is
>> supported and glamor on nouveau is either dri3/present or no hw accel
at all
>> afaics.
>
> You probably saw my patches to just remove glamor from nouveau :) That
> integration doesn't support DRI2, which in turn means no core contexts
> (due to lack of GLX_ARB_create_context_profile), and a slew of other
> issues. Seemed easier to just tell people to go use modesetting, which
> gets all of these things right(er).
>
Hm, a total removal would at least make me and my users rather unhappy 
atm., as without glamor no hw accel at all on >= maxwell. modesetting 
and nouveau are not on par feature-wise atm. E.g., modesetting as of the 
next xorg 1.18 server only provides pageflipping via dri3/present and 
without pageflipping it is game over for many of my use cases on 
nouveau-kms. Also modesetting currently completely lacks ZaphodHeads 
support. Without ZaphodHeads i can't have a page-flipped fullscreen 
window on one subset of outputs and a regular desktop on another subset, 
something that is needed for neuroscience/medical/vr applications.

So i'd rather like to preserve the choice of glamor.

What makes glamor + dri2 difficult to support in nouveau vs. other drivers?
>>
>> Btw. there are also a few patches made by Chris Wilson floating on the
>> mailing list since around january, some are reviewed and tested by
myself,
>> but not included in xorg master. Might be good for people to have a
look at
>> them and maybe get them into xorg 1.18?
>
> I would not oppose the reviewing of Chris's patches ;) However I'm
in
> no position to evaluate them myself.
>
>>
>>>
>>> On Sat, Jul 4, 2015 at 3:03 PM, Emil Velikov <emil.l.velikov at
gmail.com>
>>> wrote:
>>>>
>>>> The DRI option with the intel ddx can be used to indicate the
following
>>>>    - whether dri is disabled
>>>>    - the dri "version" - dri1, dri2, dri3
>>>>    - the dri module name - doo_dri.so bar_dri.so
>>>>
>>>> I'm not sure how exactly it's supposed to work/works,
and I believe
>>>> most of that is due to legacy reasons. I'm just saying
let's not do
>>>> the whole thing - just the dri "version" would be
great (as you
>>>> suggested).
>>
>>
>> I can change that to allow selection between "2" and
"3" - at least for exa,
>> on glamor the parameter "2" would either need to get ignored
or it would
>> completely disable hw acceleration. I went for consistency with the ati
ddx
>> because i found the intel variant too confusing. I think it changed
multiple
>> times during the last year.
>
> Bleargh. The ati ddx option name is much newer. Has it seen a release
> yet? It'd be really nice to get all the DDX's to just agree on
> something, instead of having different-but-similar options which
> confuse everyone.
>
No release tagged yet, so one could still change it.

-mario
>    -ilia
>

On Tue, Jul 14, 2015 at 3:00 PM, Mario Kleiner
<mario.kleiner.de at gmail.com> wrote:
> On 07/14/2015 05:26 PM, Ilia Mirkin wrote:
>>
>> On Mon, Jul 13, 2015 at 11:43 PM, Mario Kleiner
>> <mario.kleiner.de at gmail.com> wrote:
>>>
>>> On 07/07/2015 09:51 PM, Ilia Mirkin wrote:
>>>>
>>>>
>>>> Lastly, from some discussions with ajax on IRC, it appears that
DRI3
>>>> is half-baked at best wrt sync between server and client. I
think we
>>>> should just disable it by default for now, until issues are
ironed
>>>> out. (Rather than what this patch has, which is default-on for
Xorg >
>>>> some version.)
>>>
>>>
>>>
>>> What are the remaining known trouble spots wrt. sync? It seems to
work
>>> pretty well at least for single gpu + unredirected fullscreen
windows (=>>> kms page flipping can be used for Presents. That's
the use case i usually
>>> test very obsessively, as it matters very much for my type of
>>> applications,
>>> but other than that i only lightly "test" it via regular
desktop use, so
>>> maybe that's were problems remain?
>>
>>
>> Adam is the one who actually understands it... I was just asking
>> questions and the answer was "that's broken". A user was
using DRI3
>> with EXA (nouveau DDX), and was seeing render fail in the form of
>> stale things on the screen. The nouveau DDX does a lot of
"implicit"
>> sync stuff... it just emits commands into a pushbuf without kicking
>> it. However the libdrm code has cleverness to kick out any pushbufs if
>> you do a nouveau_bo_wait and that bo has been referenced.
>>
>> HOWEVER if you have a pixmap and you share it using the fd thing, and
>> another process makes a (shared) bo out of it, and then does a
>> nouveau_bo_wait, that in no way will cause the DDX to kick its
>> pushbuf. Apparently there's some sort of sync thing that's
supposed to
>
>
> Although then i'd expect the other process to hang in nouveau_bo_wait?
There
> probably isn't a specific fdo bug for this?
No, because it's blissfully unaware of the fact that there's anything
to wait on for the bo (since it hasn't been kicked out by the process
and submitted to the kernel), and thus decides that the bo is ready to
use.
>
>> happen, but that's entirely unimplemented for DRI3. Unfortunately I
>> can't provide more details than that, as my knowledge of X
internals
>> is quite limited. Some sort of DRI drawable or GLX drawable or ...
>> something.
>>
>> I have no proof that this is the cause of the issue the user was
>> seeing, in fact it's just as likely to be something else. However
this
>> seems like a pretty significant issue to me.
>>
>
> Ok, thanks for the explanation. But would this problem then be limited to
> exa + nouveau? glamor uses opengl and thereby mesa's and i think mesa
gets
> sync right afaik, at least in the glx/dri3 backend. Didn't look at egl
> backend though. At least i didn't see any such corruption?
>
> I'll change the patch to default to DRI 2 then for exa, and glamor will
> ignore the DRI parameter anyway and stick to DRI3.
TBH I have no idea how glamor works. Presumably it may have similar
issues, but perhaps it just sync's left and right so the problems
remain unseen. There's a "block handler" or something like that
which
tends to flush things (in nouveau as well IIRC). Although that just
flushes out dirty pixmaps, not sure what causes something to get onto
the dirty pixmap list. That might still end up not flushing pending
commands out.
>
>>>
>>> We can disable it by default on exa - intel and amd/radeon drivers
also
>>> disable by default. However, on gpus >= maxwell only glamor
accel is
>>> supported and glamor on nouveau is either dri3/present or no hw
accel at
>>> all
>>> afaics.
>>
>>
>> You probably saw my patches to just remove glamor from nouveau :) That
>> integration doesn't support DRI2, which in turn means no core
contexts
>> (due to lack of GLX_ARB_create_context_profile), and a slew of other
>> issues. Seemed easier to just tell people to go use modesetting, which
>> gets all of these things right(er).
>>
>
> Hm, a total removal would at least make me and my users rather unhappy
atm.,
> as without glamor no hw accel at all on >= maxwell. modesetting and
nouveau
> are not on par feature-wise atm. E.g., modesetting as of the next xorg 1.18
> server only provides pageflipping via dri3/present and without pageflipping
> it is game over for many of my use cases on nouveau-kms. Also modesetting
> currently completely lacks ZaphodHeads support. Without ZaphodHeads i
can't
> have a page-flipped fullscreen window on one subset of outputs and a
regular
> desktop on another subset, something that is needed for
> neuroscience/medical/vr applications.
>
> So i'd rather like to preserve the choice of glamor.
>
> What makes glamor + dri2 difficult to support in nouveau vs. other drivers?
Lack of having been done, and a lack of desire by all parties involved
to do it. As-is the glamor integration is, unfortunately, quite
broken.

I have an EXA impl for maxwell in the works, although I haven't made
progress on it in months. It's missing some sort of
nouveau_scratch_data() style call to make a temp bo to put coordinates
in so that we can do draws on the 3d engine.

I highly suggest lobbying for these features in modesetting though,
I've heard of plans (that I highly disapprove of, at least for
nouveau, although my say in the matter may be limited) of killing off
DDX support for any glamor-capable hw.
>>> I can change that to allow selection between "2" and
"3" - at least for
>>> exa,
>>> on glamor the parameter "2" would either need to get
ignored or it would
>>> completely disable hw acceleration. I went for consistency with the
ati
>>> ddx
>>> because i found the intel variant too confusing. I think it changed
>>> multiple
>>> times during the last year.
>>
>>
>> Bleargh. The ati ddx option name is much newer. Has it seen a release
>> yet? It'd be really nice to get all the DDX's to just agree on
>> something, instead of having different-but-similar options which
>> confuse everyone.
>>
>
> No release tagged yet, so one could still change it.
I'd *really* like at least some attempt to reconcile these things
among the DDX's.

  -ilia

>>>
>>> We can disable it by default on exa - intel and amd/radeon drivers
also
>>> disable by default. However, on gpus >= maxwell only glamor
accel is
>>> supported and glamor on nouveau is either dri3/present or no hw
accel at
>>> all
>>> afaics.
>>
>>
>> You probably saw my patches to just remove glamor from nouveau :) That
>> integration doesn't support DRI2, which in turn means no core
contexts
>> (due to lack of GLX_ARB_create_context_profile), and a slew of other
>> issues. Seemed easier to just tell people to go use modesetting, which
>> gets all of these things right(er).
>>
>
> Hm, a total removal would at least make me and my users rather unhappy
atm.,
> as without glamor no hw accel at all on >= maxwell. modesetting and
nouveau
> are not on par feature-wise atm. E.g., modesetting as of the next xorg 1.18
> server only provides pageflipping via dri3/present and without pageflipping
> it is game over for many of my use cases on nouveau-kms. Also modesetting
> currently completely lacks ZaphodHeads support. Without ZaphodHeads i
can't
> have a page-flipped fullscreen window on one subset of outputs and a
regular
> desktop on another subset, something that is needed for
> neuroscience/medical/vr applications.
So what would you want in modesetting to be on par?

DRI2 pageflipping?
zaphod heads?

anything else? I would like for F23 to ship a modesetting which I can
replace intel/nouveau/ati with.

Dave.

On 07/18/2015 01:09 PM, Dave Airlie wrote:
>>>>
>>>> We can disable it by default on exa - intel and amd/radeon
drivers also
>>>> disable by default. However, on gpus >= maxwell only glamor
accel is
>>>> supported and glamor on nouveau is either dri3/present or no hw
accel at
>>>> all
>>>> afaics.
>>>
>>>
>>> You probably saw my patches to just remove glamor from nouveau :)
That
>>> integration doesn't support DRI2, which in turn means no core
contexts
>>> (due to lack of GLX_ARB_create_context_profile), and a slew of
other
>>> issues. Seemed easier to just tell people to go use modesetting,
which
>>> gets all of these things right(er).
>>>
>>
>> Hm, a total removal would at least make me and my users rather unhappy
atm.,
>> as without glamor no hw accel at all on >= maxwell. modesetting and
nouveau
>> are not on par feature-wise atm. E.g., modesetting as of the next xorg
1.18
>> server only provides pageflipping via dri3/present and without
pageflipping
>> it is game over for many of my use cases on nouveau-kms. Also
modesetting
>> currently completely lacks ZaphodHeads support. Without ZaphodHeads i
can't
>> have a page-flipped fullscreen window on one subset of outputs and a
regular
>> desktop on another subset, something that is needed for
>> neuroscience/medical/vr applications.
>
> So what would you want in modesetting to be on par?
>
> DRI2 pageflipping?
Probably not needed? I debugged and tested the DRI3/Present page 
flipping with modesetting a few months ago. Haven't retested the most 
recent version of the patches in x-server master, but at that time they 
seemed to work pretty ok. I'll give this some more testing soon. I also 
ran it as my desktop for 2-3 days, afair without serious issues, but 
that wasn't the kind of obsessive stress testing i do for the OpenGL 
fullscreen pageflipping stuff.

I do remember dpms activity was a crasher last time i tested. Maybe 
resolved now?
> zaphod heads?
>
Mostly Zaphodheads would be the really important one for me. Lack of it 
= show stopper for many of my users applications. Users often want to 
split their gpu into at least 2 x-screens: One for the "operator" to 
control experimental work sessions via a regular desktop GUI, the other 
for one unredirected kms-pageflipped fullscreen window on one or 
multiple attached Zaphodheads outputs for the actual presentation on the 
2nd x-screen, e.g., single display, or dual display for stereoscopic 
stuff - stereo goggles and such, or > 2 displays for more advanced VR 
stuff. The page flipping is important in my case because precise timing 
and precise timestamping is important which we only get with page flips, 
also to get out pixels as bit-exact as possible without interference 
from compositors, e.g. texture mapping artifacts etc. On DRI3/Present, 
page flips are also the only way atm. to get tear-free Presents, the way 
it is implemented.

More regular users seem to like to have a separate x-screen for TV sets 
and use ZaphodHeads for that.

The current nouveau or radeon implementation would be a good reference 
for one that seems to work well with multiple ZaphodHeads per x-screen + 
1 extra patch of mine against nouveau floating on nouveau-devel "[PATCH] 
Take shift in crtc positions for ZaphodHeads configs into account." That 
patch is needed for nouveau-ddx, or any generic driver that deals with 
nouveau-kms. Afaics the other kms drivers return a 
kencoder->possible_crtcs mask which allows any crtc to go with any 
encoder, but nouveau-kms gets that info from the VBIOS DCB and sometimes 
the mask is a bit more special and needs the extra treatment from that 
patch, or bad things will happen, e.g., on some nv-50 cards.
> anything else? I would like for F23 to ship a modesetting which I can
> replace intel/nouveau/ati with.
Not at the moment. I'll think about it some more and give the current 
modesetting driver a bit more testing to see if something shows up.

thanks,
-mario
>
> Dave.
>

On Sat, 18 Jul 2015 21:09:46 +1000
Dave Airlie <airlied at gmail.com> wrote:
> > Hm, a total removal would at least make me and my users rather unhappy
atm.,
> > as without glamor no hw accel at all on >= maxwell. modesetting and
nouveau
> > are not on par feature-wise atm. E.g., modesetting as of the next xorg
1.18
> > server only provides pageflipping via dri3/present and without
pageflipping
> > it is game over for many of my use cases on nouveau-kms. Also
modesetting
> > currently completely lacks ZaphodHeads support. Without ZaphodHeads i
can't
> > have a page-flipped fullscreen window on one subset of outputs and a
regular
> > desktop on another subset, something that is needed for
> > neuroscience/medical/vr applications.
> 
> So what would you want in modesetting to be on par?
> 
> DRI2 pageflipping?
> zaphod heads?
> 
> anything else? I would like for F23 to ship a modesetting which I can
> replace intel/nouveau/ati with.
Hi,

FYI, it seems the Oculus Rift also prefers a ZaphodHeads setup.
http://blog.emarcs.org/blog/oculus-rift-direct-mode-on-linux/


Thanks,
pq