thr3ads.net - samba - [Samba] (DeviceIoControl, FSCTL_SET

If this information is useful, please help other people find it:
Share via:

noid at wolfdogg.org

2017-Feb-14 03:16 UTC

[Samba] (DeviceIoControl, FSCTL_SET_SPARSE)

Im having an issue with what I believe might be pinned down to a need for a
proper samba configuration. 

 

The problem: sparse writes (not sure which, either write sparse files to a
system that doesn't support them, or vice versa)

 

The system:

Zfs-on-linux, samba share

 

Accessing system which is erroring:

Win7x64_ult, freefilesync

 

Error message

 

NOTHING I have done to it is fixing, including 

strict allocate = yes

strict allocate = no

share:fake_fscaps = 0

#share:fake_fscaps = 0

 

Current samba config:

[global]

    #security = user

    #passdb backend = tdbsam

 

                workgroup = MYGROUP

                #strict allocate = yes

                

                #kill printing

                printing = bsd

                printcap name = /dev/null

                load printers = no

    #spoolss

    disable spoolss = yes

                

    #aim to fix windows freefilesync copy symlinks not working

                #follow symlinks = yes

    #wide links = yes

 

    #another zpool implemention style testing with, didnt help

    #usershare path = /san/vault

    #max shares = 0 is disabled

    #usershare max shares = 100

    #usershare allow guest = no

    #share:fake_fscaps = 0

 

[homes]

                comment = Home Directories

                valid users = %S, %D%w%S

                browseable = No

                read only = No

                inherit acls = Yes

 

[san]

    comment = ZFS 'san' pool Backup File Server

    path = /san/vault

                #share:fake_fscaps = 0

    #ironically doesnt make it readonly, however, worked in favor of
writeabe, valid users, and browseable yes, on arch

    #read only = yes

    write list = @sys1

 

Writing is done on windows machine to this path, non mapped

//hostname/san/backup/..

 

Reference urls

http://www.freefilesync.org/forum/viewtopic.php?f=2
<http://www.freefilesync.org/forum/viewtopic.php?f=2&t=1650&p=13722#p13722>
&t=1650&p=13722#p13722

https://github.com/zfsonlinux/zfs

 

same problem, now on rhel7.3, followed me from an archx64 system, same pool
https://aur.archlinux.org/packages/zfs-linux-git/ 

 

Note, I tried last night to create instead a sparse vol, and mount it after
mkfs an ext3 filesystem on there, I didn't map to it separately using samba,
but I did a test write to that dir, and still, same problem.  I didn't know
how feasible it was to actually think that a sparse vol, would solve a
sparse file write issue, but it was worth a try to me. 

 

Any ideas, ideals a proper samba config that will fix this in one shot, are
warmly welcomes, I have TB's of data hanging on a thread, getting a bit
scattered, and causing a lot of other drives to have to suffer high head
writes because of it.  One failure and im on a downhill slide.  Time is of
the essence.  I have already persued a ZFS driver issue / bug path, a
FreefileSync issue / bug path, and now im out of options I think, unless its
just the way I mounted or created my pool, OR what I suspected a bad
Property on ZFS, or the samba config.  



Regards,

Brian

David Disseldorp

2017-Feb-14 10:54 UTC

head link

[Samba] (DeviceIoControl, FSCTL_SET_SPARSE)

Hi Brian,

On Mon, 13 Feb 2017 19:16:13 -0800, brian lamb via samba wrote:
>  
> 
> Im having an issue with what I believe might be pinned down to a need for a
> proper samba configuration. 
Which version of Samba are you using?
> The problem: sparse writes (not sure which, either write sparse files to a
> system that doesn't support them, or vice versa)
What doesn't support sparse writes? How do you know that the problem is
related to sparse files?
> The system:
> 
> Zfs-on-linux, samba share
ZFS on Linux + Samba supports sparse files...
> Accessing system which is erroring:
> 
> Win7x64_ult, freefilesync
> 
>  
> 
> Error message
What's the error message? Has this application ever worked successfully
against Samba, or only against a Windows share?
> NOTHING I have done to it is fixing, including 
> 
> strict allocate = yes
> 
> strict allocate = no
> 
> share:fake_fscaps = 0
> 
> #share:fake_fscaps = 0
Windows clients request sparse file semantics explicitly, by calling
FSCTL_SET_SPARSE. Samba then retains this flag in the "user.DOSATTRIB"
extended attribute for the given file.
One way to modify this behaviour would be to set "store dos
attributes"
to "no" in smb.conf - this would see Samba respond to any
FSCTL_SET_SPARSE request with NT_STATUS_INVALID_DEVICE_REQUEST. That
said, I have no idea whether or not this will fix the application error
that you're seeing.

Cheers, David

noid at wolfdogg.org

2017-Feb-15 07:04 UTC

head link

[Samba] (DeviceIoControl, FSCTL_SET_SPARSE)

-----Original Message-----
Subject: Re: [Samba] (DeviceIoControl, FSCTL_SET_SPARSE)

I wanted to also mention, that during this sparse write, I have captured the
strace, which shows the file attributes specifically that it is trying to pass.

I see a difference, for e.g. I see that its st_size is 0 for the file that’s
causing the failure, which this is a tmp sparse file so the application can add
a placeholder for the file, so that it can do a fail safe copy I presume.  (also
turning off the fail safe file copy doesn’t help).

Strace below for one of several PID's that were running. Not sure if the
results will be the same on all pids, or why I even see multiple PID's for
smb when I run ps -ax.  Note, the trace I copied the portion that goes all the
way from the previous file's filename portion, to the following filename, so
that you can place yourself in the log.  Also note, the first file copied in
successfully.  Also note, the program is designed now to delete file if found,
then add new.  Also note, the file does exist, and if the other files exist, it
only complains when they are sparse, and ONLY these .tmp files are the sparse
files, not sure really why they are there to be honest with you, if fail-safe
copy mode is off....
------------------------
newfstatat(8, "MSIAfterburnerSetup.zip", {st_mode=S_IFREG|0744,
st_size=40376862, ...}, 0) = 0
fcntl(21, F_SETLKW, {type=F_RDLCK, whence=SEEK_SET, start=35036, len=1}) = 0
fcntl(21, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=35036, len=1}) = 0
newfstatat(8, "MPC-HCPortable_1.7.10.paf.exe", {st_mode=S_IFREG|0744,
st_size=12670432, ...}, 0) = 0
fcntl(21, F_SETLKW, {type=F_RDLCK, whence=SEEK_SET, start=38448, len=1}) = 0
fcntl(21, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=38448, len=1}) = 0
newfstatat(8, "esetsmartinstaller_enu.exe", {st_mode=S_IFREG|0744,
st_size=2870984, ...}, 0) = 0
fcntl(21, F_SETLKW, {type=F_RDLCK, whence=SEEK_SET, start=21484, len=1}) = 0
fcntl(21, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=21484, len=1}) = 0
newfstatat(8, "My Family Tree.zip.ffs_tmp", {st_mode=S_IFREG|0544,
st_size=0, ...}, 0) = 0
fcntl(21, F_SETLKW, {type=F_RDLCK, whence=SEEK_SET, start=11552, len=1}) = 0
fcntl(21, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=11552, len=1}) = 0
newfstatat(8, "2006-2008", {st_mode=S_IFDIR|0755, st_size=9, ...}, 0)
= 0
fcntl(21, F_SETLKW, {type=F_RDLCK, whence=SEEK_SET, start=6300, len=1}) = 0
fcntl(21, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=6300, len=1}) = 0
newfstatat(8, "kodi-16.1-Jarvis.exe", {st_mode=S_IFREG|0744,
st_size=87266194, ...}, 0) = 0
fcntl(21, F_SETLKW, {type=F_RDLCK, whence=SEEK_SET, start=34432, len=1}) = 0
fcntl(21, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=34432, len=1}) = 0
newfstatat(8, "expressBuilder.tar.gz", {st_mode=S_IFREG|0744,
st_size=1055704, ...}, 0) = 0

noid at wolfdogg.org

2017-Feb-15 07:16 UTC

head link

[Samba] (DeviceIoControl, FSCTL_SET_SPARSE)

To whomever reads this, 
This post I accidentally sent to david himself, I am now sending it to the
group.  It will be out of order, and includes the strace.

Regards
Brian

My apologies for any inconvenience. 

--------------

Hi David,

Thanks for your response. 

The Error message is for e.g.:

Cannot write file attributes of "\\MYHOST\backup\sparse\My Family Tree
6.0.zip.ffs_tmp".
Error Code 1: Incorrect function. (DeviceIoControl, FSCTL_SET_SPARSE)

Samba Version:
Samba version 4.4.4
PID     Username     Group        Machine                                  
Protocol Version  Encryption           Signing
----------------------------------------------------------------------------------------------------------------------------------------
21582   backupadmin sys1       myhost (ipv4:192.168.1.99:55068)         SMB2_10 
-                    -

I just tried the option   " store dos attributes = no"
Im getting the same error as above still unfortunately, but im doing some more
thorough testing on that.  WOud you happen to know if I want the other two
options set, and to what, at all? (> strict allocate & share:fake_fscaps
= 0)

The application (freefilesync) has worked successfully against Samba before I
rebuilt the array to a raidz2, from a zfs linear span (no raid parity disk,
still other features worked, scrub, crc checking, etc..) and I had no problem in
the 6TB 3 disk span.  Now I added three disks, , its now ~8TB with two parity
disks.    The only thing I can imagine different is the freefilesync program,
since there was a gap in time when I switched out the raid array, it didn’t work
from the get go(the sparse part, same errors) but this was about a month or two,
and a new version of freefilesync I believe too.  I noted tht  freefilesync has
supported sparse for a long time, and I posted the url for the freefilesync
forum, where the author of the app himself suspected it was possibly samba, see
that post.

Much Appreciated David!

Regards Brian

-----Original Message-----
From: samba [mailto:samba-bounces at lists.samba.org] On Behalf Of David
Disseldorp via samba
Sent: Tuesday, February 14, 2017 2:55 AM
To: brian lamb via samba <samba at lists.samba.org>
Subject: Re: [Samba] (DeviceIoControl, FSCTL_SET_SPARSE)

Hi Brian,

On Mon, 13 Feb 2017 19:16:13 -0800, brian lamb via samba wrote:
>  
> 
> Im having an issue with what I believe might be pinned down to a need 
> for a proper samba configuration.
Which version of Samba are you using?
> The problem: sparse writes (not sure which, either write sparse files 
> to a system that doesn't support them, or vice versa)
What doesn't support sparse writes? How do you know that the problem is
related to sparse files?
> The system:
> 
> Zfs-on-linux, samba share
ZFS on Linux + Samba supports sparse files...
> Accessing system which is erroring:
> 
> Win7x64_ult, freefilesync
> 
>  
> 
> Error message
What's the error message? Has this application ever worked successfully
against Samba, or only against a Windows share?
> NOTHING I have done to it is fixing, including
> 
> strict allocate = yes
> 
> strict allocate = no
> 
> share:fake_fscaps = 0
> 
> #share:fake_fscaps = 0
Windows clients request sparse file semantics explicitly, by calling
FSCTL_SET_SPARSE. Samba then retains this flag in the "user.DOSATTRIB"
extended attribute for the given file.
One way to modify this behaviour would be to set "store dos
attributes"
to "no" in smb.conf - this would see Samba respond to any
FSCTL_SET_SPARSE request with NT_STATUS_INVALID_DEVICE_REQUEST. That said, I
have no idea whether or not this will fix the application error that you're
seeing.

Cheers, David

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba

Reasonably Related Threads

Search for more reasonably related threads

samba - Feb 2017 - (DeviceIoControl, FSCTL_SET_SPARSE)

[Samba] (DeviceIoControl, FSCTL_SET_SPARSE)

[Samba] (DeviceIoControl, FSCTL_SET_SPARSE)

[Samba] (DeviceIoControl, FSCTL_SET_SPARSE)

[Samba] (DeviceIoControl, FSCTL_SET_SPARSE)

Reasonably Related Threads