samba - Oct 2016 - Samba and BTRFS server-side copy

On Thu, 27 Oct 2016 16:27:05 -0700, Jeremy Allison <jra at samba.org>
wrote :
> On Fri, Oct 28, 2016 at 01:00:24AM +0200, Saint Germain via samba
> wrote:
> > 
> > On Thu, 27 Oct 2016 15:39:27 -0700, Jeremy Allison <jra at
samba.org>
> > wrote :
> > 
> > > On Fri, Oct 28, 2016 at 12:08:50AM +0200, Saint Germain via samba
> > > wrote:
> > > > On Thu, 27 Oct 2016 14:50:48 -0700, Jeremy Allison
> > > > <jra at samba.org> wrote :
> > > > > 
> > > > > Server-side copy can be requested using smbclient,
using
> > > > > the "scopy" command.
> > > > > 
> > > > > It's a simple:
> > > > > 
> > > > > scopy source dest
> > > > 
> > > > Thanks but I just tested and it doesn't trigger
> > > > BTRFS_IOC_CLONE_RANGE. The server-side copy is working all
> > > > right, but the file is not copied through BTRFS reflink (cp
> > > > --reflink=auto). It is quite noticeable on large file, where
it
> > > > should be nearly instantaneous.
> > > > 
> > > > My configuration:
> > > >  - client: Debian Stable + smbclient 4.4.6 (coming from
> > > > Testing) + kernel 4.7 (amd64)
> > > >  - server: Debian Stable + samba 4.2.10 + kernel 4.7 (amd64)
> > > 
> > > OK, follow though using debug level 10 and see why it isn't
> > > selecting the reflink code.
> > > 
> > > Are you using SMB2 ?
> > 
> > No I was using SMB1: my mistake.
> > Now with SMB2, it works, thanks !
> > 
> > If I can ask, what is this restrictions with "Btrfs filesystem
> > aligned" and "FSCTL_SRV_COPYCHUNK wire request byte offsets
and
> > lengths align to the Btrfs sector-size" ?
> > Is it a restriction on the server or the client ?
> 
> Server.
> 
> > How can I check that I don't have this restriction ?
> 
> Ensure clone range request sizes are integer
> divisible by 4096.
> 
> If the btrfs clone range request fails smbd
> will fall back to copychunk copies.
> 
> Debug level 5 has:
> 
>                 /*
>                  * BTRFS_IOC_CLONE_RANGE only supports 'sectorsize'
> aligned
>                  * cloning. Which is 4096 by default, therefore fall
> back to
>                  * manual read/write on failure.
>                  */
>                 DEBUG(5, ("BTRFS_IOC_CLONE_RANGE failed: %s, length
> %llu, " "src fd: %lld off: %llu, dest fd: %d off: %llu\n",
>                           strerror(errno),
>                           (unsigned long long)cr_args.src_length,
>                           (long long)cr_args.src_fd,
>                           (unsigned long long)cr_args.src_offset,
>                           dest_fsp->fh->fd,
>                           (unsigned long long)cr_args.dest_offset));
> 
> if you see any of these then it happened.
Ok I understand how to check that it happened, but how can I make sure
that it doesn't happen ?
How can I ensure that clone range request sizes are integer divisible
by 4096 on my server ?
Should I change something on the BTRFS filesystem, or shift my
partition or something ?

Thanks !

On Fri, Oct 28, 2016 at 01:42:35AM +0200, Saint Germain via samba
wrote:
> On Thu, 27 Oct 2016 16:27:05 -0700, Jeremy Allison <jra at samba.org>
> > if you see any of these then it happened.
> 
> Ok I understand how to check that it happened, but how can I make sure
> that it doesn't happen ?
> How can I ensure that clone range request sizes are integer divisible
> by 4096 on my server ?
> Should I change something on the BTRFS filesystem, or shift my
> partition or something ?
Nope. It's not a critical error. In smbclient, scopy
requests a copy of the entire filesize. The libsmbclient
layer splits this into copychunk requests of default
size 1mb - which is certainly integer divisible by
4096.

The only time you'll get the server copy is if the
last chunk isn't divisible by 4096. It's not a problem.

On Thu, 27 Oct 2016 16:57:13 -0700, Jeremy Allison <jra at samba.org>
wrote :
> On Fri, Oct 28, 2016 at 01:42:35AM +0200, Saint Germain via samba
> wrote:
> > On Thu, 27 Oct 2016 16:27:05 -0700, Jeremy Allison <jra at
samba.org>
> > > if you see any of these then it happened.
> > 
> > Ok I understand how to check that it happened, but how can I make
> > sure that it doesn't happen ?
> > How can I ensure that clone range request sizes are integer
> > divisible by 4096 on my server ?
> > Should I change something on the BTRFS filesystem, or shift my
> > partition or something ?
> 
> Nope. It's not a critical error. In smbclient, scopy
> requests a copy of the entire filesize. The libsmbclient
> layer splits this into copychunk requests of default
> size 1mb - which is certainly integer divisible by
> 4096.
> 
> The only time you'll get the server copy is if the
> last chunk isn't divisible by 4096. It's not a problem.
Thank you very much for all the help and explanations.
It's much clearer now.

So if I understand correctly, as long as the client send the
FSCTL_SRV_COPYCHUNK, it should be ok.

The FSCTL_SRV_COPYCHUNK instructions seems to be included in SMB2 in
smbclient starting from version 4.3.

On linux, most current utilities like Nautilus (gvfs) still use SMB1 to
connect, so there is no chance to trigger a server-side copy.

On Mac, it seems to be using SMB3, so it should be ok.

On Windows, I'll have to check.

Thanks for your help !