Nouveau - Jul 2014 - [PATCH 09/17] drm/radeon: use common fence implementation for fences

Am 23.07.2014 11:44, schrieb Daniel Vetter:
> On Wed, Jul 23, 2014 at 11:39 AM, Daniel Vetter <daniel.vetter at
ffwll.ch> wrote:
>> The scheduler needs to keep track of a lot of fences, so I think
we'll
>> have to register callbacks, not a simple wait function. We must keep
>> track of all the non-i915 fences for all oustanding batches. Also, the
>> scheduler doesn't eliminate the hw queue, only keep it much slower
so
>> that we can sneak in higher priority things.
>>
>> Really, scheduler or not is orthogonal.
> Also see my other comment about interactions between wait_fence and
> the i915 reset logic. We can't actually use it from within the
> scheduler code since that would deadlock.
Yeah, I see. You would need some way to abort the waiting on other 
devices fences in case of a lockup.

What about an userspace thread to offload waiting and command submission to?

Just playing with ideas right now,
Christian.
> -Daniel

On Wed, Jul 23, 2014 at 11:47 AM, Christian K?nig
<deathsimple at vodafone.de> wrote:
> Am 23.07.2014 11:44, schrieb Daniel Vetter:
>> On Wed, Jul 23, 2014 at 11:39 AM, Daniel Vetter <daniel.vetter at
ffwll.ch>
>> wrote:
>>>
>>> The scheduler needs to keep track of a lot of fences, so I think
we'll
>>> have to register callbacks, not a simple wait function. We must
keep
>>> track of all the non-i915 fences for all oustanding batches. Also,
the
>>> scheduler doesn't eliminate the hw queue, only keep it much
slower so
>>> that we can sneak in higher priority things.
>>>
>>> Really, scheduler or not is orthogonal.
>>
>> Also see my other comment about interactions between wait_fence and
>> the i915 reset logic. We can't actually use it from within the
>> scheduler code since that would deadlock.
>
>
> Yeah, I see. You would need some way to abort the waiting on other devices
> fences in case of a lockup.
>
> What about an userspace thread to offload waiting and command submission
to?
That's what your android guys currently do. They hate it. And google
explicitly created their syncpts stuff to move all that into the
kernel. That one does explicit fencing, but the end result is still
that you have fences as deps between different drivers.

The other problem is that dri/prime is running under an implicitly
sync'ed model, so there's no clear point/responsibility for who would
actually do the waiting. You'll end up with synchronous behaviour
since the render sooner or later needs to perfectly align with
client/compositor ipc.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

op 23-07-14 11:47, Christian K?nig schreef:
> Am 23.07.2014 11:44, schrieb Daniel Vetter:
>> On Wed, Jul 23, 2014 at 11:39 AM, Daniel Vetter <daniel.vetter at
ffwll.ch> wrote:
>>> The scheduler needs to keep track of a lot of fences, so I think
we'll
>>> have to register callbacks, not a simple wait function. We must
keep
>>> track of all the non-i915 fences for all oustanding batches. Also,
the
>>> scheduler doesn't eliminate the hw queue, only keep it much
slower so
>>> that we can sneak in higher priority things.
>>>
>>> Really, scheduler or not is orthogonal.
>> Also see my other comment about interactions between wait_fence and
>> the i915 reset logic. We can't actually use it from within the
>> scheduler code since that would deadlock.
>
> Yeah, I see. You would need some way to abort the waiting on other devices
fences in case of a lockup.
>
> What about an userspace thread to offload waiting and command submission
to?
You would still need enable_signaling, else polling on the dma-buf wouldn't
work. ;-)
Can't wait synchronously with multiple shared fences, need to poll for that.
And the dma-buf would still have fences belonging to both drivers, and it would
still call from outside the driver.

~Maarten

Am 23.07.2014 11:55, schrieb Maarten Lankhorst:
> op 23-07-14 11:47, Christian K?nig schreef:
>> Am 23.07.2014 11:44, schrieb Daniel Vetter:
>>> On Wed, Jul 23, 2014 at 11:39 AM, Daniel Vetter <daniel.vetter
at ffwll.ch> wrote:
>>>> The scheduler needs to keep track of a lot of fences, so I
think we'll
>>>> have to register callbacks, not a simple wait function. We must
keep
>>>> track of all the non-i915 fences for all oustanding batches.
Also, the
>>>> scheduler doesn't eliminate the hw queue, only keep it much
slower so
>>>> that we can sneak in higher priority things.
>>>>
>>>> Really, scheduler or not is orthogonal.
>>> Also see my other comment about interactions between wait_fence and
>>> the i915 reset logic. We can't actually use it from within the
>>> scheduler code since that would deadlock.
>> Yeah, I see. You would need some way to abort the waiting on other
devices fences in case of a lockup.
>>
>> What about an userspace thread to offload waiting and command
submission to?
> You would still need enable_signaling, else polling on the dma-buf
wouldn't work. ;-)
> Can't wait synchronously with multiple shared fences, need to poll for
that.
No you don't. Just make a list of fences you need to wait for and wait 
for each after another. But having an thread for each command submission 
context doesn't sounds like the best solution anyway.
> And the dma-buf would still have fences belonging to both drivers, and it
would still call from outside the driver.
Calling from outside the driver is fine as long as the driver can do 
everything necessary to complete it's work and isn't forced into any 
ugly hacks and things that are not 100% reliable.

So I don't see much other approach as integrating recovery code for not 
firing interrupts and some kind of lockup handling into the fence code 
as well.

Christian.
>
> ~Maarten
>