Alex Crow
2016-Jul-05 19:12 UTC
[Samba] Winbind process stuck at 100% after changing use_mmap to no
On 05/07/16 19:45, Volker Lendecke wrote:> On Tue, Jul 05, 2016 at 07:21:16PM +0100, Alex Crow wrote: >> I've set up the "DR" side of my cluster to "use mmap = no" and with >> "private dir" removed from the smb.conf. > Why do you set "use mmap = no"? > >> I have the MooseFS guys on the case as well. Should I put them in touch >> with you or vice/versa? > MooseFS should not have any business at all in tdb files. If you still > find tdb files used on MooseFS, you should investigate your > configuration and modify it such that tdb files only exist on local > file systems. > > VolkerHi Volker, There are *no* more tdb files on MooseFS, that's what I'm trying to test. From the ping_pong wiki page it's instructed to use "use_mmap = no" if the ping_pong test does not work correctly on "ping_ping -m -rw test.file", which it did, completely. Text here: " Testing mmap coherence If you add the -m switch to ping_pong along with -rw then it will do the I/O coherence test via mmap. It isn't absolutely essential that a cluster filesystem supports coherent mmap for CTDB/Samba, but it's nice for bragging points over other cluster filesystems. If your cluster filesystem doesn't pass this test then just use the "use mmap = no" option in smb.conf. Even if it does pass this test that option may be a good idea on most cluster filesystems." So is this documentation wrong as well? It seems to suggest setting it anyway, which is clearly not working for us (and now for sure there is no involvement with TDB files as I've deleted the "private dir" in smb.conf. The only reason I was asking if you'd like to talk to the MooseFS chaps is because of the failure of the -rw test, which as far as the documentation goes, seems to refer to the shared FS rather than any places where tdb files live. I hope you can see what I'm getting at here! Many thanks Alex -- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. This email is not intended to, nor should it be taken to, constitute advice. The information provided is correct to our knowledge & belief and must not be used as a substitute for obtaining tax, regulatory, investment, legal or any other appropriate advice. "Transact" is operated by Integrated Financial Arrangements Ltd. 29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608 5300. (Registered office: as above; Registered in England and Wales under number: 3727592). Authorised and regulated by the Financial Conduct Authority (entered on the Financial Services Register; no. 190856).
Volker Lendecke
2016-Jul-05 19:45 UTC
[Samba] Winbind process stuck at 100% after changing use_mmap to no
On Tue, Jul 05, 2016 at 08:12:59PM +0100, Alex Crow wrote:> > > On 05/07/16 19:45, Volker Lendecke wrote: > > On Tue, Jul 05, 2016 at 07:21:16PM +0100, Alex Crow wrote: > >> I've set up the "DR" side of my cluster to "use mmap = no" and with > >> "private dir" removed from the smb.conf. > > Why do you set "use mmap = no"? > > > >> I have the MooseFS guys on the case as well. Should I put them in touch > >> with you or vice/versa? > > MooseFS should not have any business at all in tdb files. If you still > > find tdb files used on MooseFS, you should investigate your > > configuration and modify it such that tdb files only exist on local > > file systems. > > > > Volker > > There are *no* more tdb files on MooseFS, that's what I'm trying to test.Why are you still using "use mmap = no" then?> From the ping_pong wiki page it's instructed to use "use_mmap = no" ifAs I sad in an earlier mail: The wiki page is wrong. If you don't trust one of the primary developers of Samba and to a lesser extent ctdb (myself, sorry for bragging) and trust the wiki more, then I am a bit lost, sorry.> Text here: > > " > > > Testing mmap coherence > > If you add the -m switch to ping_pong along with -rw then it will do the > I/O coherence test via mmap. It isn't absolutely essential that a^^^^^^^^^^^^^^^^^^^^^^^^^^ This is the key piece. It is not essential.> cluster filesystem supports coherent mmap for CTDB/Samba, but it's nice > for bragging points over other cluster filesystems. If your cluster > filesystem doesn't pass this test then just use the "use mmap = no" > option in smb.conf. Even if it does pass this test that option may be a > good idea on most cluster filesystems."This piece is wrong. The wiki should give you hints who wrote this. Please contact the author of those lines, refer to the mail thread here and ask these lines to be changed or even better -- change these lines yourself.> So is this documentation wrong as well? It seems to suggest setting itYes, the documentation is wrong.> anyway, which is clearly not working for us (and now for sure there is > no involvement with TDB files as I've deleted the "private dir" in smb.conf. > > > The only reason I was asking if you'd like to talk to the MooseFS chaps > is because of the failure of the -rw test, which as far as the > documentation goes, seems to refer to the shared FS rather than any > places where tdb files live.If MooseFS does not even survive -rw without asking for mmap coherence, just don't use it for Samba. This will eat your data for breakfast. Gluster was there loooong ago, and they fixed it before becoming a viable storage solution for Samba. Volker
Alex Crow
2016-Jul-05 19:49 UTC
[Samba] Winbind process stuck at 100% after changing use_mmap to no
FYI, by "it did, completely" I meant it failed completely. Even if the only file we have to have on the cluster FS (which now seems to be down solely to the ctdb lock file), I'm still worried what these failures mean: 1) does the -rw (without -m) test suggest any problems with the MooseFS FS I'm serving files to users from? 2) does the mmap test failure mean I should not even have my CTDB lock file on MooseFS? 3) Do the inconsistent reports from a ping_pong -rw test show there is a major problem with MooseFS as the underlying FS for Samba exports and something needs to be done with the underlying FS? I'm a paying licensee of MooseFS Pro, so I hope you can understand my wishes to get this problem sorted. If contracting some of the Sernet chaps is required, we've done it before and might be prepared to do so in this case. Best regards Alex On 05/07/16 20:12, Alex Crow wrote:> > > On 05/07/16 19:45, Volker Lendecke wrote: >> On Tue, Jul 05, 2016 at 07:21:16PM +0100, Alex Crow wrote: >>> I've set up the "DR" side of my cluster to "use mmap = no" and with >>> "private dir" removed from the smb.conf. >> Why do you set "use mmap = no"? >> >>> I have the MooseFS guys on the case as well. Should I put them in touch >>> with you or vice/versa? >> MooseFS should not have any business at all in tdb files. If you still >> find tdb files used on MooseFS, you should investigate your >> configuration and modify it such that tdb files only exist on local >> file systems. >> >> Volker > > Hi Volker, > > There are *no* more tdb files on MooseFS, that's what I'm trying to test. > > From the ping_pong wiki page it's instructed to use "use_mmap = no" if > the ping_pong test does not work correctly on "ping_ping -m -rw > test.file", which it did, completely. > > Text here: > > " > > > Testing mmap coherence > > If you add the -m switch to ping_pong along with -rw then it will do > the I/O coherence test via mmap. It isn't absolutely essential that a > cluster filesystem supports coherent mmap for CTDB/Samba, but it's > nice for bragging points over other cluster filesystems. If your > cluster filesystem doesn't pass this test then just use the "use mmap > = no" option in smb.conf. Even if it does pass this test that option > may be a good idea on most cluster filesystems." > > > So is this documentation wrong as well? It seems to suggest setting it > anyway, which is clearly not working for us (and now for sure there is > no involvement with TDB files as I've deleted the "private dir" in > smb.conf. > > > The only reason I was asking if you'd like to talk to the MooseFS > chaps is because of the failure of the -rw test, which as far as the > documentation goes, seems to refer to the shared FS rather than any > places where tdb files live. > > I hope you can see what I'm getting at here! > > Many thanks > > Alex > > >-- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. This email is not intended to, nor should it be taken to, constitute advice. The information provided is correct to our knowledge & belief and must not be used as a substitute for obtaining tax, regulatory, investment, legal or any other appropriate advice. "Transact" is operated by Integrated Financial Arrangements Ltd. 29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608 5300. (Registered office: as above; Registered in England and Wales under number: 3727592). Authorised and regulated by the Financial Conduct Authority (entered on the Financial Services Register; no. 190856).
Rowland penny
2016-Jul-05 20:00 UTC
[Samba] Winbind process stuck at 100% after changing use_mmap to no
On 05/07/16 20:49, Alex Crow wrote:> FYI, by "it did, completely" I meant it failed completely. > > Even if the only file we have to have on the cluster FS (which now > seems to be down solely to the ctdb lock file), I'm still worried what > these failures mean: > > 1) does the -rw (without -m) test suggest any problems with the > MooseFS FS I'm serving files to users from? > 2) does the mmap test failure mean I should not even have my CTDB lock > file on MooseFS? > 3) Do the inconsistent reports from a ping_pong -rw test show there is > a major problem with MooseFS as the underlying FS for Samba exports > and something needs to be done with the underlying FS? > > I'm a paying licensee of MooseFS Pro, so I hope you can understand my > wishes to get this problem sorted. If contracting some of the Sernet > chaps is required, we've done it before and might be prepared to do so > in this case. > >ROFL ROFL Do you know who Volker is ??? Please listen to him, do what he advises and don't bother contacting Sernet, because you already have. Rowland
Alex Crow
2016-Jul-05 20:06 UTC
[Samba] Winbind process stuck at 100% after changing use_mmap to no
Hi Volker, I apologise if I came across as an a***hole here. Responses below: On 05/07/16 20:45, Volker Lendecke wrote:> On Tue, Jul 05, 2016 at 08:12:59PM +0100, Alex Crow wrote: >> >> On 05/07/16 19:45, Volker Lendecke wrote: >>> On Tue, Jul 05, 2016 at 07:21:16PM +0100, Alex Crow wrote: >>>> I've set up the "DR" side of my cluster to "use mmap = no" and with >>>> "private dir" removed from the smb.conf. >>> Why do you set "use mmap = no"? >>> >>>> I have the MooseFS guys on the case as well. Should I put them in touch >>>> with you or vice/versa? >>> MooseFS should not have any business at all in tdb files. If you still >>> find tdb files used on MooseFS, you should investigate your >>> configuration and modify it such that tdb files only exist on local >>> file systems. >>> >>> Volker >> There are *no* more tdb files on MooseFS, that's what I'm trying to test. > Why are you still using "use mmap = no" then?Because the wiki article does not explicitly state that mmap coherence is only needed for areas that store tdb files. I'd perhaps incorrectly deduced that as you had explicitly said that "private dir" on the cluster FS was a bad idea, that disabling that would stop all tdb files being written to the MooseFS FS, and that would be the solution. My understanding was the "use mmap" affected general FS operations from clients, not just tdb files.> >> From the ping_pong wiki page it's instructed to use "use_mmap = no" if > As I sad in an earlier mail: The wiki page is wrong. If you don't trust > one of the primary developers of Samba and to a lesser extent ctdb > (myself, sorry for bragging) and trust the wiki more, then I am a bit > lost, sorry.I am completely aware of who you and Jeremy are and I have the utmost respect and thanks for the amazing software you have provided. We're planning our migration to Samba 4 AD (from classic) right now.> >> Text here: >> >> " >> >> >> Testing mmap coherence >> >> If you add the -m switch to ping_pong along with -rw then it will do the >> I/O coherence test via mmap. It isn't absolutely essential that a > ^^^^^^^^^^^^^^^^^^^^^^^^^^ > This is the key piece. It is not essential.Again, I was not clear if it meant that "it's not essential, so if it fails, you *must* disable mmap, if it doesn't fail, no worries but perhaps do it just in case". However there is no mention anywhere, including in the smb.conf man page, that it may lead to huge perfomance impact.> >> cluster filesystem supports coherent mmap for CTDB/Samba, but it's nice >> for bragging points over other cluster filesystems. If your cluster >> filesystem doesn't pass this test then just use the "use mmap = no" >> option in smb.conf. Even if it does pass this test that option may be a >> good idea on most cluster filesystems." > This piece is wrong. The wiki should give you hints who wrote this. > Please contact the author of those lines, refer to the mail thread > here and ask these lines to be changed or even better -- change these > lines yourself.I see what you mean here, but I'd not want to edit a wiki when I have less idea of the facts that the person that wrote the original text! I'd not want to update it without knowing the precise facts, hence my reaching out to the Samba list and yourself.> >> So is this documentation wrong as well? It seems to suggest setting it > Yes, the documentation is wrong. > >> anyway, which is clearly not working for us (and now for sure there is >> no involvement with TDB files as I've deleted the "private dir" in smb.conf. >> >> >> The only reason I was asking if you'd like to talk to the MooseFS chaps >> is because of the failure of the -rw test, which as far as the >> documentation goes, seems to refer to the shared FS rather than any >> places where tdb files live. > If MooseFS does not even survive -rw without asking for mmap coherence, > just don't use it for Samba. This will eat your data for breakfast. > Gluster was there loooong ago, and they fixed it before becoming a > viable storage solution for Samba. > > VolkerMooseFS does not ask for mmap coherence. It has inconsistent results on a plain -rw test. I think the chaps over there are interested in gaining some input from your considerable experience. If they have to pay fees via Sernet or we do, I'm not bothered. I already have them going over my setup and checking everything, so I'm hoping it's a misconfiguration on my part. The other choice was GlusterFS but its small-file performance was almost an order of magnitude below MooseFS. Many thanks Alex -- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. This email is not intended to, nor should it be taken to, constitute advice. The information provided is correct to our knowledge & belief and must not be used as a substitute for obtaining tax, regulatory, investment, legal or any other appropriate advice. "Transact" is operated by Integrated Financial Arrangements Ltd. 29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608 5300. (Registered office: as above; Registered in England and Wales under number: 3727592). Authorised and regulated by the Financial Conduct Authority (entered on the Financial Services Register; no. 190856).
Apparently Analagous Threads
- Winbind process stuck at 100% after changing use_mmap to no
- Winbind process stuck at 100% after changing use_mmap to no
- Winbind process stuck at 100% after changing use_mmap to no
- Winbind process stuck at 100% after changing use_mmap to no
- Winbind process stuck at 100% after changing use_mmap to no