samba - Oct 2000 - Locking propigation probelm samba to netatalk & netatalk to samba

Take a look at this output from lsof:
smbd    11490        0    8u   REG    3,3 55296 1136647 Maintenanc.fp5
afpd    31010        0    2u   REG    3,3  3207 1136655 auction - envelope.rtf
afpd    31010        0    5u   REG    3,3  4156 1136656 Word Work File L 1
afpd    31010        0    8u   REG    3,3 55296 1136647 Maintenanc.fp5
smbd    31386        0    9u   REG    3,3  4156 1136656 Word Work File L 1

This is showing a rtf file open in word on mac and windows, both writable.  I am
able to save on one while the other has the same file open.  The information
isn't transferring between samba and netatalk.  Locking works within samba
and
within netatalk, meaning mac-mac locking conflicts are reported and smb-smb
locking conflicts are too.  But if you have the same file open in windows and
mac, there isn't any warning!

I've turned off oplocks, level 2 oplocks, kernel oplocks and turned on
strict
locking.  Same difference.

This is a Debian 2.2 box running 2.2.17  I am attaching my netatalk and samba
configs.

Please help!

I'm getting corrupted files because of this. :(
-- 

Mike Fedyk                   "They that can give up essential liberty
Information Systems           to obtain a little temporary safety
Match Mail Productions Inc.   deserve neither liberty nor safety."
mfedyk@matchmail.com                                   Ben Franklin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: config.tar.gz
Type: application/x-gzip
Size: 5363 bytes
Desc: not available
Url :
http://lists.samba.org/archive/samba/attachments/20001018/d727ad82/config.tar-0002.bin

Mike Fedyk wrote:
> 
> Take a look at this output from lsof:
> smbd    11490        0    8u   REG    3,3 55296 1136647 Maintenanc.fp5
> afpd    31010        0    2u   REG    3,3  3207 1136655 auction -
envelope.rtf
> afpd    31010        0    5u   REG    3,3  4156 1136656 Word Work File L 1
> afpd    31010        0    8u   REG    3,3 55296 1136647 Maintenanc.fp5
> smbd    31386        0    9u   REG    3,3  4156 1136656 Word Work File L 1
> 
> This is showing a rtf file open in word on mac and windows, both writable. 
I am
> able to save on one while the other has the same file open.  The
information
> isn't transferring between samba and netatalk.  Locking works within
samba and
> within netatalk, meaning mac-mac locking conflicts are reported and smb-smb
> locking conflicts are too.  But if you have the same file open in windows
and
> mac, there isn't any warning!
As I understand it, Samba 2.0.7 implements oplocks for windows clients,
but those locks do not propagate to the underlying Unix system. This can
cause locking problems with Unix apps and windows apps accessing the
same files, which is essentially what you are seeing. Neither Netatalk
or Samba have any idea that the file is locked by the other.

One of the new features of Samba 2.2 is the mapping of oplocks to POSIX
locks. From the release notes:-

=======================================================Rewritten internal
locking semantics for more robustness.
This alpha supports full 64 bit locking semantics on all
(even 32 bit) platforms. SMB locks are mapped onto POSIX
locks (32 bit or 64 bit) as the underlying system allows.
=======================================================
This should mean that, assuming your undelying system supports POSIX
locks, then Samba will have done its bit with regard to this. You still
need Netatalk to honour the POSIX locks, and also to map Mac locks to
POSIX locks to be able to use this sort of setup with impunity. I have
no idea what the state of this sort of thing is within Netatalk - maybe
someone on the Netatalk list will know this...

I'd suggest that you make Mac and Windows users use different shares for
write access. Allow both groups read-only access to the others shares,
and if they need to edit a file, they can make a copy. Otherwise, you're
probably in for large amounts of pain....

HTH

Mike.


-- 
Unsubscribe?  mail -s unsubscribe debian-user-request@lists.debian.org <
/dev/null

Mike Brodbelt wrote:
> 
> As I understand it, Samba 2.0.7 implements oplocks for windows clients,
> but those locks do not propagate to the underlying Unix system. This can
> cause locking problems with Unix apps and windows apps accessing the
> same files, which is essentially what you are seeing. Neither Netatalk
> or Samba have any idea that the file is locked by the other.
> 
> One of the new features of Samba 2.2 is the mapping of oplocks to POSIX
> locks. From the release notes:-
I have disabled oplocks, and kernel oplocks.  Oplocks aren't the problem,
and
they work perfectly, even level 2 oplocks in a smb only setup.
> 
> =======================================================> Rewritten
internal locking semantics for more robustness.
> This alpha supports full 64 bit locking semantics on all
> (even 32 bit) platforms. SMB locks are mapped onto POSIX
> locks (32 bit or 64 bit) as the underlying system allows.
> =======================================================> 
> This should mean that, assuming your undelying system supports POSIX
> locks, then Samba will have done its bit with regard to this. You still
> need Netatalk to honour the POSIX locks, and also to map Mac locks to
> POSIX locks to be able to use this sort of setup with impunity. I have
> no idea what the state of this sort of thing is within Netatalk - maybe
> someone on the Netatalk list will know this...
It looks like older versions of netatalk used flock, but newer versions may be
using fcntl.  This makes me think that the underlying unix will see the lock.

I've also read that samba needs fcntl locks to work properly.  Why don't
they
see each other's locks if both are using fcntl?
> 
> I'd suggest that you make Mac and Windows users use different shares
for
> write access. Allow both groups read-only access to the others shares,
> and if they need to edit a file, they can make a copy. Otherwise,
you're
> probably in for large amounts of pain....
> 
This would be a major pain because we have two departments that are slowly
replacing macs with  NT and they need to work on the same sets of files with
working locks.  

If this can't happen, I'll look into smb from the mac, and as a last
resort,
I've tested this under NT and the locking works between smb and afp,
although
it'll be slower, that's better than data corruption.  That would be bad,
because
I'm trying to move all of our file serving from mac (completed) and NT over
to
linux.
> HTH
> 
> Mike.

-- 

Mike Fedyk                   "They that can give up essential liberty
Information Systems           to obtain a little temporary safety
Match Mail Productions Inc.   deserve neither liberty nor safety."
mfedyk@matchmail.com                                   Ben Franklin

This is the type of information many of us have been looking for with
respect to Netatalk and Samba coexisting in a "friendly" matter. Many
of
the replies I received when originally asking whether the two behaved
together basically stated that the biggest problem was a Windows user and
a Netatalk user accessing the same file at the same time.

Some of the other comments regarding Samba and Netatalk being used on the
same share...


Sorry about the long post, but this is pretty important to a lot of
people.

-----

I've been doing this for awhile.  I have a stable netatalk/samba server
at netatalk 1.4b2+asun.  No trouble; unless my users aren't telling me.
Read the mailing lists - there a --with-netatalk config time option (not
in the build docs that I could see, but it was covered on the mailing
list IIRC).  --with-netatalk is currently an experimental hack, if I
remember, it forces samba to move the .AppleDouble entries if a PC user
moves files around from the PC (so as not to pooch appleshare side
stuff).  Don't know how reliable it is, I only included that option on
the most recent server I config'd and it hasn't been tested thoroughly
yet.

-----

I've been researching this for a while, and the biggest problem I've
seen
is incompatible locking.  Simultaneously opening a datafile from both a
Samba connected PC and a Netatalk connected Mac can cause strange things
to
happen if they both try to modify the file.  There are no errors, and the
file can easily be corrupted.

AFAIK the --with-netatalk in Samba does nothing.  Unless something has
been
changed recently, it was a feature that was never finished.

----

The two seem to co-exist together pretty well.  Rich made a
few valid points with .AppleDouble files.

        However, there is one thing that you may want to bring up in
regards to Windows and Linux users.  You need to let them know that
they can't create files greater than 32 characters or the Mac users
won't be a able to see them.  And you will get holy hell from your
Mac users when they call you and day "Bob said that he put the file
on the server, but I can't see it, is the server broken?"
        So, you may need to do some social engineering with your
Windows and Linux users to make sure that they understand the
limitation.  They will forget about it anyway, so make sure you have
them sign something.  :-)  Can you tell I've been an Admin for way
too long?

----

There is some sort of enable-netatalk-compatibility (or something similar
to that) option in Samba which is supposed to, I repeat *supposed to* make
it easier for netatalk to understand the file types.  But I've never tried
it and I've heard some say that it doesn't work and they can't
figure out
what that feature is supposed to be for.  Might be worth investigating,
though.

----

On the other hand I had the oportunity to look pretty deep into samba's
mechanisms and I never heard or read about any  "netatalk
compatibility"
option though. So personnaly I would say go for it, but make sure of two
things to avoid mac users coming at you complaining about generic icons
and
apps not launching:

- first have your .AppleVolumes.system up to date so files coming from a
PC
(and without resources fork) are seen properly on macs. The
.AppleVolume.system does that by "telling" files to assume such or
such
mac-identity according to their file extension.

-second, you will have to somehow get the mac users into the habit of
naming
their files with an extension, so that they are seen properly on windows
stations. And this is not easy, let me tell you that!

----

Make sure that you're running a version of netatalk that has working
codepage support (the current snapshots are broken in this regard, AFAIK)
if you want your users to use German umlauts and such in their
filenames. Then you need to configure both Samba and Netatalk to use the
same codepage. Current versions also have a mswindows options which
disallows the creation of filenames containing the characters

/\:*?"<>|

which the PC can't deal with (my PC actually crashes when trying to
access a file containing ':' that was created on the Mac).

I'm still looking for a version of Netatalk that has _both_ working
codepage support and a working mswindows option . . . which would really
resolve all the problems we've had.

The locking issue is another matter (see below in this thread).
Interestingly, we haven't come across a single incident of this being
a problem in about 6 months of running Netatalk/Samba in a Network with 10
Macs and 5 PCs.

-----

Albeit better than the NON-EXISTANCE of shared locking between
Samba/Netatalk (although the new Samba has a locking API, so
theoretically any program can use its locking API to "stay safe" with
Samba), NT/2k still do not lock well between the two platforms. There is
no way to lock parts of a file,  and write-locks don't exist, so both
plats can simultaneously write to the same file and possibly corrupt it
horribly.

----

To make Samba and Netatalk work together, it is a good plan to veto
those files which are automagically created by Netatalk on the Samba
side. My veto string looks like this:

/.AppleDesktop/.AppleDouble/Network Trash
Folder/TheVolumeSettingsFolder/TheFindByContentFolder/Temporary
Items/lost+found/Folder Settings/.finf/

What I have not been able to figure out is how to veto Icon^M files.
Does anybody know?

----

I wonder if collaboration between Netatalk folks and Samba folks could
occur?!

God, it would be sweet to have both coexist, eh!

On Wed, 18 Oct 2000, Mike Brodbelt wrote:
>
> Mike Fedyk wrote:
> >
> > Take a look at this output from lsof:
> > smbd    11490        0    8u   REG    3,3 55296 1136647 Maintenanc.fp5
> > afpd    31010        0    2u   REG    3,3  3207 1136655 auction -
envelope.rtf
> > afpd    31010        0    5u   REG    3,3  4156 1136656 Word Work File
L 1
> > afpd    31010        0    8u   REG    3,3 55296 1136647 Maintenanc.fp5
> > smbd    31386        0    9u   REG    3,3  4156 1136656 Word Work File
L 1
> >
> > This is showing a rtf file open in word on mac and windows, both
writable.  I am
> > able to save on one while the other has the same file open.  The
information
> > isn't transferring between samba and netatalk.  Locking works
within samba and
> > within netatalk, meaning mac-mac locking conflicts are reported and
smb-smb
> > locking conflicts are too.  But if you have the same file open in
windows and
> > mac, there isn't any warning!
>
> As I understand it, Samba 2.0.7 implements oplocks for windows clients,
> but those locks do not propagate to the underlying Unix system. This can
> cause locking problems with Unix apps and windows apps accessing the
> same files, which is essentially what you are seeing. Neither Netatalk
> or Samba have any idea that the file is locked by the other.
>
> One of the new features of Samba 2.2 is the mapping of oplocks to POSIX
> locks. From the release notes:-
>
> =======================================================> Rewritten
internal locking semantics for more robustness.
> This alpha supports full 64 bit locking semantics on all
> (even 32 bit) platforms. SMB locks are mapped onto POSIX
> locks (32 bit or 64 bit) as the underlying system allows.
> =======================================================>
> This should mean that, assuming your undelying system supports POSIX
> locks, then Samba will have done its bit with regard to this. You still
> need Netatalk to honour the POSIX locks, and also to map Mac locks to
> POSIX locks to be able to use this sort of setup with impunity. I have
> no idea what the state of this sort of thing is within Netatalk - maybe
> someone on the Netatalk list will know this...
>
> I'd suggest that you make Mac and Windows users use different shares
for
> write access. Allow both groups read-only access to the others shares,
> and if they need to edit a file, they can make a copy. Otherwise,
you're
> probably in for large amounts of pain....
>
> HTH
>
> Mike.
>
>
>