samba - Feb 2023 - Missing Files/Missing Folders from an NFS Share

I think it has a lot to do with telldir on an NFS share returning a cookie
that is supposed to be unique rather than an actual offset. Problem is,
that cookie is stored as a 31bit hash which can then end up as a negative
signed offset in Samba which is not handled correctly. The cookies are not
necessarily consecutive as you move through a directory.

On Mon, 27 Feb 2023 at 19:00, Jeremy Allison <jra at samba.org> wrote:
> On Mon, Feb 27, 2023 at 01:18:20PM +0100, Conor Armstrong via samba wrote:
> >Just by way of update on this, it looks like the problem exists with
the
> >ReadDir, TellDir and SeekDir implementation on NFS shares.  In my case
it
> >was an AWS EFS share that uses NFS.
>
> Can you elaborate exactly the issue with the AWS NFS server here ?
> It would be really helpful to understand what it's doing here.
>
> >The fix in the samba code isn't straightforward.  As a workaround
on my
> >case I discovered that if I mount the NFS share at say /mnt/mountpoint1
> and
> >the use sshfs to connect back into the same machine and mount at
> >/mnt/mountpoint2 the /mnt/mountpoint1 directory I can then share the
> >/mnt/mountpoint2 share with Samba and the sshfs layer that sits between
> >Samba and the NFS mount seems to correct the issues that are present if
t
> >the mounted NFS share were shared with Samba directly.
> >
> >So instead of
> >
> >NFS -> /mnt/mountpoint1 and /mnt/mountpoint1 -> Samba ->
Windows Client
> >
> >I use
> >
> >NFS -> /mnt/mountpoint1 and /mnt/mountpoint1 -> /mnt/mountpoint2
(via
> >sshfs) and /mnt/mountpoint2 -> Samba -> Windows Client
> >
> >There is a performance hit in doing this, but it is something I can
live
> >with.
>
> I'm glad you found a fix, but if we can understand what is wrong
> with AWS NFS here we might be able to do something more performant
> in Samba.
>

On Mon, Feb 27, 2023 at 08:01:40PM +0100, Conor Armstrong
wrote:
>   I think it has a lot to do with telldir on an NFS share returning a
cookie
>   that is supposed to be unique rather than an actual offset. Problem is,
>   that cookie is stored as a 31bit hash which can then end up as a negative
>   signed offset in Samba which is not handled correctly. The cookies are
not
>   necessarily consecutive as you move through a directory. ?
We only store the returned cookie as a 'long' from telldir(), and then
use
it to get to the same position again via seekdir(). There isn't any
interpretation of the actual value inside Samba other than 3 'magic'
values we use #defined here:

/* "Special" directory offsets. */
#define END_OF_DIRECTORY_OFFSET ((long)-1)
#define START_OF_DIRECTORY_OFFSET ((long)0)
#define DOT_DOT_DIRECTORY_OFFSET ((long)0x80000000)

So long as telldir() doesn't return any of these,
we should work fine. We don't expect the cookies
to be consecutive or in any order for that matter.

So I would take issue with your statement:

"can then end up as a negative signed offset in Samba which is not handled
correctly"

:-).

By the way, there is nothing special about an AWS EFS share as far as I can
see. It is just basically an NFS share.

The answer some others were saying about Samba sharing the folder out from
the NFS server isn?t possible. In AWS there is no access to the server
behind EFS other than the NFS share itself. EFS has to be mounted as NFS.
No other protocols are supported (eg CIFS or AFS).

Anyway the sshfs intermediate mount worked for me but it would be good to
have this issue addressed in Samba.

On Mon, 27 Feb 2023 at 20:01, Conor Armstrong <conorarmstrong at
gmail.com>
wrote:
> I think it has a lot to do with telldir on an NFS share returning a cookie
> that is supposed to be unique rather than an actual offset. Problem is,
> that cookie is stored as a 31bit hash which can then end up as a negative
> signed offset in Samba which is not handled correctly. The cookies are not
> necessarily consecutive as you move through a directory.
>
> On Mon, 27 Feb 2023 at 19:00, Jeremy Allison <jra at samba.org>
wrote:
>
>> On Mon, Feb 27, 2023 at 01:18:20PM +0100, Conor Armstrong via samba
wrote:
>> >Just by way of update on this, it looks like the problem exists
with the
>> >ReadDir, TellDir and SeekDir implementation on NFS shares.  In my
case it
>> >was an AWS EFS share that uses NFS.
>>
>> Can you elaborate exactly the issue with the AWS NFS server here ?
>> It would be really helpful to understand what it's doing here.
>>
>> >The fix in the samba code isn't straightforward.  As a
workaround on my
>> >case I discovered that if I mount the NFS share at say
/mnt/mountpoint1
>> and
>> >the use sshfs to connect back into the same machine and mount at
>> >/mnt/mountpoint2 the /mnt/mountpoint1 directory I can then share
the
>> >/mnt/mountpoint2 share with Samba and the sshfs layer that sits
between
>> >Samba and the NFS mount seems to correct the issues that are
present if t
>> >the mounted NFS share were shared with Samba directly.
>> >
>> >So instead of
>> >
>> >NFS -> /mnt/mountpoint1 and /mnt/mountpoint1 -> Samba ->
Windows Client
>> >
>> >I use
>> >
>> >NFS -> /mnt/mountpoint1 and /mnt/mountpoint1 ->
/mnt/mountpoint2 (via
>> >sshfs) and /mnt/mountpoint2 -> Samba -> Windows Client
>> >
>> >There is a performance hit in doing this, but it is something I can
live
>> >with.
>>
>> I'm glad you found a fix, but if we can understand what is wrong
>> with AWS NFS here we might be able to do something more performant
>> in Samba.
>>
>