samba - Jul 2016 - Winbind process stuck at 100% after changing use_mmap to no

On Thu, Jul 14, 2016 at 02:39:48PM +0100, Alex Crow
wrote:
> >The main hint I would like to give to the MooseFS developers is to get
> >ping_pong -rw working. If you need any assistance in setting that up
> >and getting arguments on why this is important, let us know!
> >
> >With best regards,
> >
> >Volker
> 
> Any arguments that I can pass on would be much appreciated.
The main argument is that Windows applications tend to coordinate access
to files via byte-range locks. They expect data coherency under those
byte range locks, meaning that if a value is written under a brlock,
the brlock is released, and another process reads the value under the
brlock, the value written must be visible. If this is not guaranteed,
then data corruption will likely happen. The main question to the
MooseFS developers would be under which circumstances a value written
via pwrite on node A is visible via pread on node B. ping_pong checks
this sequentiality with fcntl byte range locks, which are the Posix
way of serializing access to ranges of a file. If MooseFS gives other
guarantees, we would like to know about them.
> I've also had a try with RozoFS, which did not work as expected as
> it only supports block-level locks (4096 bytes) so the first lock
> prevent further locks in that block range, which naturally causes
> ping_pong to stall. However I am told by their devs that this should
> not be an issue for CTDB's recovery lock or Samba as they should not
> need to lock ranges of less than a 4k, and they in fact integrate
> CTDB for their customers.
That's right. For the recovery lock this does not matter. But for plain
file access it will. On RozoFS you should set "posix locking = no",
but then you depend on external serialization via ctdb: With "strict
locking = yes" (which is default), Samba does check byte range locks and
this way serializes access to user files on RozoFS. This serialization
however is not coordinated with the cluster file system in question,
so the RozoFS developers might have to tell us about their approach to
data coherency too.

With best regards,

Volker Lendecke

On 14/07/16 14:51, Volker Lendecke wrote:
> On Thu, Jul 14, 2016 at 02:39:48PM +0100, Alex Crow wrote:
>>> The main hint I would like to give to the MooseFS developers is to
get
>>> ping_pong -rw working. If you need any assistance in setting that
up
>>> and getting arguments on why this is important, let us know!
>>>
>>> With best regards,
>>>
>>> Volker
>> Any arguments that I can pass on would be much appreciated.
> The main argument is that Windows applications tend to coordinate access
> to files via byte-range locks. They expect data coherency under those
> byte range locks, meaning that if a value is written under a brlock,
> the brlock is released, and another process reads the value under the
> brlock, the value written must be visible. If this is not guaranteed,
> then data corruption will likely happen. The main question to the
> MooseFS developers would be under which circumstances a value written
> via pwrite on node A is visible via pread on node B. ping_pong checks
> this sequentiality with fcntl byte range locks, which are the Posix
> way of serializing access to ranges of a file. If MooseFS gives other
> guarantees, we would like to know about them.
>
>> I've also had a try with RozoFS, which did not work as expected as
>> it only supports block-level locks (4096 bytes) so the first lock
>> prevent further locks in that block range, which naturally causes
>> ping_pong to stall. However I am told by their devs that this should
>> not be an issue for CTDB's recovery lock or Samba as they should
not
>> need to lock ranges of less than a 4k, and they in fact integrate
>> CTDB for their customers.
> That's right. For the recovery lock this does not matter. But for plain
> file access it will. On RozoFS you should set "posix locking =
no",
> but then you depend on external serialization via ctdb: With "strict
> locking = yes" (which is default), Samba does check byte range locks
and
> this way serializes access to user files on RozoFS. This serialization
> however is not coordinated with the cluster file system in question,
> so the RozoFS developers might have to tell us about their approach to
> data coherency too.
So perhaps posix locking=no might be worth trying on MooseFS too?

I will pass on your comments in my support ticket with the MooseFS staff...

Thanks again and best regards

Alex

--
This message is intended only for the addressee and may contain
confidential information. Unless you are that person, you may not
disclose its contents or use it in any way and are requested to delete
the message along with any attachments and notify us immediately.
This email is not intended to, nor should it be taken to, constitute advice.
The information provided is correct to our knowledge & belief and must not
be used as a substitute for obtaining tax, regulatory, investment, legal or
any other appropriate advice.

"Transact" is operated by Integrated Financial Arrangements Ltd.
29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608
5300.
(Registered office: as above; Registered in England and Wales under
number: 3727592). Authorised and regulated by the Financial Conduct
Authority (entered on the Financial Services Register; no. 190856).

On Thu, Jul 14, 2016 at 03:01:38PM +0100, Alex Crow
wrote:
> So perhaps posix locking=no might be worth trying on MooseFS too?
Maybe. But we need a statement by the MooseFS developers on which
events they make sure the data is consistent.

Regards, Volker