freebsd stable - Aug 2016 - unionfs bugs, a partial patch and some comments [Was: Re: 1-BETA3 Panic: __lockmgr_args: downgrade a recursed lockmgr nfs @ /usr/local/share/deploy-tools/RELENG_11/src/sys/fs/unionfs/union_vnops.c:1905]

Bez?glich Mark Johnston's Nachricht vom 09.08.2016 08:02 (localtime):
?
>>
>> Just for anybody else needing unionfs:
>>
https://people.freebsd.org/~attilio/unionfs_missing_insmntque_lock.patch
>>
>> This patch still applies and I'm successfully using this
(unmodified) up
>> to FreeBSD-10.3 and never had any panic in all these years.
> 
> Having spent some time looking at unionfs, I'm a bit skeptical that
this
> patch will address the panic you reported earlier, though I'd be
> interested to know if it does. 
Thanks for your attention.
I can confirm that it has prevented panics for more than 4 years
(9.0-10.3) and it seems to be still "good enough" to also prevent
panics
in 11-BETA4.
I updated my build host (stable/11, this time with the
unionfs_missing_insmntque_lock.patch), where the recent panics happened
and unionfs gets much more utilized than usually in my setups: No panic
with that patch anymore.
Just one message like "prevented resource deadlock" occured.
> Reading the code, I think it will just
> address an INVARIANTS-only assertion in insmntque1().
> 
> Unfortunately, unionfs is quite difficult to fix within the current
> constraints of FreeBSD's VFS. unionfs_readdir() is a particularly good
> demonstration of this fact: some callers of VOP_READDIR expect the
> cookies returned by the FS to be monotonically increasing, but unionfs
> has no straightforward way to make this guarantee.
I'm sorry, I can't provide help here. My skills would require a huge
ammount of lerning-time to get into that matter. I'd love to do that,
but I can't afford :-(

Thanks,

-Harry

Harry Schmalzbauer wrote:
Bez?glich Mark Johnston's Nachricht vom 09.08.2016 08:02 (localtime):
?
>>
>> Just for anybody else needing unionfs:
>>
https://people.freebsd.org/~attilio/unionfs_missing_insmntque_lock.patch
>>
>> This patch still applies and I'm successfully using this
(unmodified) up
>> to FreeBSD-10.3 and never had any panic in all these years.
>
> Having spent some time looking at unionfs, I'm a bit skeptical that
this
> patch will address the panic you reported earlier, though I'd be
> interested to know if it does.
[stuff snipped for brevity]
I took a look at this. (I know nothing about unionfs, but a little w.r.t. the
VFS).
I can confirm that this function (unionfs_nodeget()) is weird and appears to
be broken to me.

The function calls insmntque() before it initializes the vnode, which seems
racey, especially if it isn't LK_EXCLUSIVE locked.
Also, line#s 278-281:
    if (uppervp != NULLVP)
         vp->v_vnlock = uppervp->v_vnlock;
    else
         vp->v_vnlock = lowervp->v_vnlock;
so your patch isn't locking the vnode lock that it actually uses.
I think the vp argument to insmntque() is required to be LK_EXCLUSIVE
locked mostly so other threads won't fiddle with the vnode until this
function is done with it, but I am not sure?

I think a more correct version of this (not saying it would be correct[?],
would call insmntque() later in the function, after it has been initialized.
(This means that the cleanup if it fails is more involved, but...)

I've attached a patch (untested) that does this. Maybe you could try it?

rick
ps: I've cc'd Kostik, in case he has some insight w.r.t. how this should
be handled?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: unionfs-newvnode.patch
Type: application/octet-stream
Size: 1549 bytes
Desc: unionfs-newvnode.patch
URL:
<http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20160811/8686d731/attachment.obj>