samba - Dec 2023 - Accessing files though samba and efficient copy

A while back I tried to come with a solution to a problem - updating an
application
(consisting of a large number of files) which is accessed though a samba server.

The problem was that when updating a file which is open / in use in windows,
Bad Things (tm) happen.  Unix allows such update "behind windows's
back", and
when windows tries to open this file again, it discovers this is a different
file already, and fails badly.

The solution to this was to use samba to update files too, - and sure thing,
this way we immediately know if a file is in use, - because the update just
fails without a way around it.

So there are 2 bad cases and no good: either windows fails to open updated
files, or there's no way to update at all.  So far so good.

I worked around this in a different way: by creating a reparse point for the
whole application directory (msdfs root = true) and in case we have to update
a file which is in use, we just create a "new version" of the whole
thing
and update the referral, so with time all clients switches to the new version.

Now the next question is: how to copy files effectively?  We always used
rsync for the copy, running on the samba server.  It's fast to check a few
1000s of unchanged files and sync just the ones which changed, just the actual
differences.  But if we do it though samba in order to determine if any of
the new files has changed, we have to copy using regular windows copy mechanism,
which is insanely slow in our case (since files are large).

It seems to be a never-ending story...

/mjt

Robocopy would do that on Windows...

Could using rsync with a CIFS-mounted shared directory achieve the same?
I don't know how well it performs though, but at least figuring out
which files _not_ to copy could be fast. Just and idea...

Btw, good idea to use changing DFS / reparse points to change content :)

regards,
Norbert

On 17.12.2023 19:33, Michael Tokarev via samba wrote:
> A while back I tried to come with a solution to a problem - updating
> an application
> (consisting of a large number of files) which is accessed though a
> samba server.
>
> The problem was that when updating a file which is open / in use in
> windows,
> Bad Things (tm) happen.? Unix allows such update "behind windows's
> back", and
> when windows tries to open this file again, it discovers this is a
> different
> file already, and fails badly.
>
> The solution to this was to use samba to update files too, - and sure
> thing,
> this way we immediately know if a file is in use, - because the update
> just
> fails without a way around it.
>
> So there are 2 bad cases and no good: either windows fails to open
> updated
> files, or there's no way to update at all.? So far so good.
>
> I worked around this in a different way: by creating a reparse point
> for the
> whole application directory (msdfs root = true) and in case we have to
> update
> a file which is in use, we just create a "new version" of the
whole thing
> and update the referral, so with time all clients switches to the new
> version.
>
> Now the next question is: how to copy files effectively?? We always used
> rsync for the copy, running on the samba server.? It's fast to check a
> few
> 1000s of unchanged files and sync just the ones which changed, just
> the actual
> differences.? But if we do it though samba in order to determine if
> any of
> the new files has changed, we have to copy using regular windows copy
> mechanism,
> which is insanely slow in our case (since files are large).
>
> It seems to be a never-ending story...
>
> /mjt
>