Hi there,I run samba as a PDC and tried to make this PDC high available with redhat cluster suite and gfs. I experienced the following problem while doing this: If I set the option locking = no in smb.conf it takes about 4 minutes to copy a file of 1GB size. If I set locking = yes it takes about 1 hour. Im not sure if locking = no sets locking off for all locking options. At least I need locking for some of my shares. Are there some useful options for gfs or recommendations? Thanks, markus
On Feb 9, 2008 8:49 AM, markus neis <markus.neis@googlemail.com> wrote:> Hi there,I run samba as a PDC and tried to make this PDC high available > with > redhat cluster suite and gfs. I experienced the following problem while > doing this: > If I set the option locking = no in smb.conf it takes about 4 minutes to > copy a file of 1GB size. If I set locking = yes it takes about 1 hour. Im > not sure if locking = no sets locking off for all locking options. At > least > I need locking for some of my shares. Are there some useful options for > gfs > or recommendations? > > Thanks, markusJust out of curiosity, what do you have set for oplocks, and do you have blocking locks turned off? -- Peace and Blessings, -Scott. "Of course, that's just my opinion; I could be wrong" -Dennis Miller
i set oplocks = yes , kernel oplocks = yes and as I said locking = yes, but this slows down everything _____________________________________________________________________ Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! http://smartsurfer.web.de/?mc=100071&distributionid=000000000066
On Feb 9, 2008 3:01 PM, markus neis <markus.neis@googlemail.com> wrote:> i set oplocks = yes , kernel oplocks = yes and as I said locking = yes, > but this slows down everything > >OK, from what I gather (which very well could be inaccurate), it looks like you might be stuck on a spinlock timeout on a blocking call. Also, if I understand the documentation correctly, when you set locking yes, you pass the call for a lock to the next interface layer, which will traverse a good number of calls, and query the file system somewhere along the way. Samba will queue the lock with blocking locks, and check every now and then, not continuing along with the I/O until it hears back on the lock status. Meanwhile the call is passed to GFS for the lock which will then query the other node, which must make the same series of calls and send its response back across the wire (this may not happen depending on caches, I'm not sure), all the way back to samba who then continues if it can have the lock. If samba can't have the lock, this process starts all over. That is, once again, as I understand it. If you set locking = no, it never gets passed to the first interface, immediately returning a success, without ever having done the lock. So you ask for the lock and samba says, "You have the lock.", then I come along asking for the same range lock and samba once again says "You have the lock.". from man 5 smb.conf: blocking locks (S) This parameter controls the behavior of smbd(8) when given a request by a client to obtain a byte range lock on a region of an open file, and the request has a time limit associated with it. If this parameter is set and the lock range requested cannot be immediately satisfied, samba will internally queue the lock request, and periodically attempt to obtain the lock until the timeout period expires. If this parameter is set to no, then samba will behave as previous versions of Samba would and will fail the lock request immedi- ately if the lock range cannot be obtained. Default: blocking locks = yes locking (S) This controls whether or not locking will be performed by the server in response to lock requests from the client. If locking = no, all lock and unlock requests will appear to succeed and all lock queries will report that the file in ques- tion is available for locking. If locking = yes, real locking will be performed by the server. This option may be useful for read-only filesystems which may not need locking (such as CDROM drives), although setting this parameter of no is not really recommended even in this case. Be careful about disabling locking either globally or in a specific service, as lack of locking may result in data corruption. You should never need to set this parameter. No default lock spin time (G) The time in microseconds that smbd should keep waiting to see if a failed lock request can be granted. This parameter has changed in default value from Samba 3.0.23 from 10 to 200. The associated lock spin count parameter is no longer used in Samba 3.0.24. You should not need to change the value of this parameter. Default: lock spin time = 200 It would seem that you could get a bit of performance tuning GFS, but I'm thinking that you'll also have to tune layers that GFS depends on to see much of a difference (lower throughput for faster response from network, CPU and memory). There is a "gfs_tool gettune" command that will get the gfs tunable parameters which can be set via "gfs_tool settune". This thing seems to cross so many layers that you'd have to tune each layer along its path, IMHO. I hope this was a bit helpful. Can anyone with more low level knowledge confirm or refute this at all? -- Peace and Blessings, -Scott. "Of course, that's just my opinion; I could be wrong" -Dennis Miller
Damn! this doesn't sound good. I hope somebody else can refute what you say ;-) gfs shouldn't be that slow. I'm really confused. _______________________________________ GRATIS: Movie-FLAT. Jetzt freischalten! http://freemail.web.de/club/maxdome.htm
ok, but i heard people say that ctdb isn't for productional use. can you confirm that volker? Is it also a good idea to save the locking information on the gfs filesystem? ________________________________________________________ Bis 50 MB Dateianh?nge? Kein Problem! http://www.digitaledienste.web.de/freemail/club/lp/?lp=7
Ok I understand. People that earn more money than me made this decision ;-) But I wonder why Redhat offers the possibility in their cluster suite to setup samba services on top of gfs. This should be a known problem then. However, as I understand you its not a good idea to use samba with gfs unless we would use ctdb? _________________________________________________________________________ In 5 Schritten zur eigenen Homepage. Jetzt Domain sichern und gestalten! Nur 3,99 EUR/Monat! http://www.maildomain.web.de/?mc=021114
I put the tdb files out of gfs only some data files are now on gfs, but it's the same behavior. ;-( ________________________________________________________ Bis 50 MB Dateianh?nge? Kein Problem! http://www.digitaledienste.web.de/freemail/club/lp/?lp=7
On Sun, Feb 10, 2008 at 12:51:24AM +0100, Markus Neis wrote:> Ok I understand. People that earn more money than me made > this decision ;-) But I wonder why Redhat offers the > possibility in their cluster suite to setup samba services > on top of gfs. This should be a known problem then. > However, as I understand you its not a good idea to use > samba with gfs unless we would use ctdb?You can perfectly fine use samba on top of gfs, as long as you only share your data from a single node or (more precisely) make sure that every directory is only shared via a single node. Different directories can be shared via different nodes. Volker -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://lists.samba.org/archive/samba/attachments/20080210/c296edbf/attachment.bin
here's my smb.conf if you would be so nice ... ;-) [global] dos charset = 850 unix charset = CP850 display charset = LOCALE workgroup = FILESRV realm = netbios name = FILESRV netbios aliases = austauschsrv, fachaustsrv, grplwsrv netbios scope = server string = filesrv-pdc93 interfaces = bind interfaces only = No security = USER auth methods = encrypt passwords = Yes update encrypted = No client schannel = Auto server schannel = Auto allow trusted domains = Yes map to guest = Never null passwords = No obey pam restrictions = No password server = * smb passwd file = /etc/samba/smbpasswd private dir = /etc/samba passdb backend = ldapsam:"ldap://10.50.100.110:389 ldap://10.50.100.111:389 ldap://localhost:389" algorithmic rid base = 1000 root directory = guest account = nobody enable privileges = Yes pam password change = No passwd program = /usr/sbin/smbldap-passwd -u %u passwd chat = "Changing password for*\nNew password*" %n\n "*Retype new password*" %n\n" passwd chat debug = No passwd chat timeout = 2 check password script = username map = /etc/samba/smbusers password level = 0 username level = 0 unix password sync = No restrict anonymous = 0 lanman auth = No ntlm auth = Yes client NTLMv2 auth = No client lanman auth = Yes client plaintext auth = No preload modules = use kerberos keytab = No log level = 0 syslog = 0 syslog only = No log file = /var/log/samba/%m.log max log size = 100000 debug timestamp = Yes debug prefix timestamp = No debug hires timestamp = No debug pid = No debug uid = No enable core files = Yes smb ports = 445 139 large readwrite = Yes max protocol = NT1 min protocol = CORE read bmpx = No read raw = Yes write raw = Yes disable netbios = No reset on zero vc = No acl compatibility = auto defer sharing violations = Yes nt pipe support = Yes nt status support = Yes announce version = 4.9 announce as = NT max mux = 50 max xmit = 16644 name resolve order = lmhosts wins host bcast max ttl = 259200 max wins ttl = 518400 min wins ttl = 21600 time server = No unix extensions = No use spnego = Yes client signing = auto server signing = No client use spnego = Yes enable asu support = No svcctl list = deadtime = 10 getwd cache = Yes keepalive = 300 lpq cache time = 30 max smbd processes = 0 paranoid server security = Yes max disk size = 0 max open files = 10000 open files database hash size = 10007 socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192 use mmap = Yes hostname lookups = No name cache timeout = 660 load printers = Yes printcap cache time = 750 printcap name = cups cups server = iprint server = disable spoolss = No addport command = enumports command = addprinter command = deleteprinter command = show add printer wizard = Yes os2 driver map = mangling method = hash2 mangle prefix = 1 max stat cache size = 1024 stat cache = Yes machine password timeout = 604800 add user script = rename user script = delete user script = add group script = delete group script = add user to group script = delete user from group script = set primary group script = add machine script = /usr/sbin/smbldap-useradd -w "%u" shutdown script = abort shutdown script = username map script = logon script = logon path = \\%N\%U\profile logon drive = H: logon home = \\%N\%U domain logons = Yes os level = 254 lm announce = Auto lm interval = 60 preferred master = Yes local master = Yes domain master = Yes browse list = Yes enhanced browsing = Yes dns proxy = Yes wins proxy = No wins server = wins support = Yes wins hook = kernel oplocks = Yes lock spin time = 200 oplock break wait time = 0 ldap admin dn = cn=ldapadmin,dc=fmber93,dc=de ldap delete dn = Yes ldap group suffix = ou=Groups ldap idmap suffix = ou=Users ldap machine suffix = ou=Computers ldap passwd sync = Yes ldap replication sleep = 1000 ldap suffix = dc=fmber93,dc=de ldap ssl = no ldap timeout = 15 ldap page size = 1024 ldap user suffix = ou=Users add share command = change share command = delete share command = eventlog list = config file = preload = lock directory = /var/cache/samba pid directory = /var/run utmp directory = wtmp directory = utmp = No default service = message command = get quota command = set quota command = remote announce = remote browse sync = socket address = 0.0.0.0 homedir map = auto.home afs username map = afs token lifetime = 604800 log nt token command = time offset = 0 NIS homedir = No usershare allow guests = No usershare max shares = 0 usershare owner only = Yes usershare path = /var/cache/samba/usershares usershare prefix allow list = usershare prefix deny list = usershare template share = panic action = host msdfs = Yes passdb expand explicit = No idmap domains = idmap backend = idmap alloc backend = idmap cache time = 900 idmap negative cache time = 120 idmap uid = 16777216-33554431 idmap gid = 16777216-33554431 template homedir = /home/%D/%U template shell = /bin/false winbind separator = \ winbind cache time = 300 winbind enum users = No winbind enum groups = No winbind use default domain = No winbind trusted domains only = No winbind nested groups = Yes winbind nss info = template winbind refresh tickets = No winbind offline logon = No winbind normalize names = No comment = path = username = invalid users = valid users = admin users = read list = write list = printer admin = force user = force group = read only = Yes acl check permissions = Yes acl group control = No acl map full control = Yes create mask = 0744 force create mode = 00 security mask = 0777 force security mode = 00 directory mask = 0775 force directory mode = 00 directory security mask = 0777 force directory security mode = 00 force unknown acl user = No inherit permissions = No inherit acls = No inherit owner = No guest only = No guest ok = No only user = No hosts allow = hosts deny = allocation roundup size = 1048576 aio read size = 0 aio write size = 0 aio write behind = ea support = No nt acl support = Yes profile acls = No map acl inherit = No afs share = No block size = 1024 change notify = Yes kernel change notify = Yes max connections = 0 min print space = 0 strict allocate = No strict sync = No sync always = No use sendfile = No write cache size = 0 max reported print jobs = 0 max print jobs = 1000 printable = No printing = cups cups options = print command = lpq command = %p lprm command = lppause command = lpresume command = queuepause command = queueresume command = printer name = use client driver = No default devmode = Yes force printername = No printjob username = %U default case = lower case sensitive = No preserve case = Yes short preserve case = Yes mangling char = ~ hide dot files = Yes hide special files = No hide unreadable = No hide unwriteable files = No delete veto files = No veto files = hide files = veto oplock files = /*.?db/*.exe/*.doc/*.ppt/ map archive = Yes map hidden = No map system = No map readonly = yes mangled names = Yes mangled map = store dos attributes = No dmapi support = No browseable = Yes blocking locks = Yes csc policy = manual fake oplocks = No locking = Yes oplocks = Yes level2 oplocks = Yes oplock contention limit = 2 posix locking = no strict locking = No share modes = Yes dfree cache time = 0 dfree command = copy = include = preexec = preexec close = No postexec = root preexec = root preexec close = No root postexec = available = Yes volume = fstype = NTFS set directory = No wide links = Yes follow symlinks = Yes dont descend = /proc,/dev,/etc,/lib,/lost+found,/initrd magic script = magic output = delete readonly = No dos filemode = Yes dos filetimes = Yes dos filetime resolution = Yes fake directory create times = No vfs objects = msdfs root = No msdfs proxy = [netlogon] path = /mnt/cluconf/netlogon browseable = No [profiles] path = /mnt/services/profiles valid users = %U, "Domain Admins" force user = %U read only = No create mask = 0754 guest ok = Yes profile acls = Yes browseable = No csc policy = disable [homes] comment = repertoire de %U, %u path = /mnt/services/HomeLw/%U valid users = %S read only = No create mask = 0754 hide files = /*~/ [net$] path = /mnt/services/netlogon valid users = mantowsk, keilen force user = root read only = No browseable = No [nagcon] comment = Monitoring Freigabe fuer Nagios path = /mnt/services/HomeLw/nagcon [lizscan$] comment = path = /mnt/services/Austausch/lizscan read only = No create mask = 0777 force create mode = 0777 directory mask = 0777 hide files = /*~/ [Groups] comment = path = /mnt/cluconf/tmplink/Groups/%U read only = No create mask = 0770 force create mode = 0770 force security mode = 0700 directory mask = 0770 force directory mode = 0770 hide files = /*~/ root preexec = /usr/local/bin/mklinks %U Gruppen Gruppen [awadm$] comment = path = /mnt/cluconf/tmplink/awadm/%U valid users = dorn, falabu, reimer force user = root read only = No create mask = 0770 force create mode = 0770 force security mode = 0777 directory mask = 0770 hide files = /*~/ root preexec = /usr/local/bin/mklinks %U Gruppen Berawadm [fb$] comment = path = /mnt/services/Groups/fb valid users = frach, jonen force group = ba_210_1 read only = No create mask = 0777 force create mode = 0777 directory mask = 0777 hide files = /*~/ [exch] comment = path = /mnt/cluconf/tmplink/exchange/%U read only = No create mask = 0770 force create mode = 0770 force security mode = 0700 directory mask = 0770 hide files = /*~/ root preexec = /usr/local/bin/mklinks %U Abteilung FachAust Bereich FachAust [betrieb_Auftraege_Statistik$] comment = betrieb_Auftr????ge_Statistik path = /mnt/services/Abteilung/betrieb/betrieb_Auftraege_Statistik force group = betrieb read only = No create mask = 0770 force create mode = 0770 force security mode = 0777 directory mask = 0770 hide files = /*~/ _______________________________________ GRATIS: Movie-FLAT. Jetzt freischalten! http://freemail.web.de/club/maxdome.htm
On Sun, Feb 10, 2008 at 01:08:04AM +0100, Volker Lendecke wrote:> > You can perfectly fine use samba on top of gfs, as long as > you only share your data from a single node or (more > precisely) make sure that every directory is only shared via > a single node. Different directories can be shared via > different nodes.Setting that up is what MS-DFS is for :-). Jeremy.
That's true, but I still wonder why there are no performance problems when I set locking = no ! _____________________________________________________________________ Unbegrenzter Speicherplatz f?r Ihr E-Mail Postfach? Jetzt aktivieren! http://www.digitaledienste.web.de/freemail/club/lp/?lp=7
Volker Lendecke wrote:> On Sun, Feb 10, 2008 at 12:51:24AM +0100, Markus Neis wrote: > > Ok I understand. People that earn more money than me made > > this decision ;-) But I wonder why Redhat offers the > > possibility in their cluster suite to setup samba services > > on top of gfs. This should be a known problem then. > > However, as I understand you its not a good idea to use > > samba with gfs unless we would use ctdb? > > You can perfectly fine use samba on top of gfs, as long as > you only share your data from a single node or (more > precisely) make sure that every directory is only shared via > a single node. Different directories can be shared via > different nodes.If samba+gfs is really only feasible for single node access then why even use GFS? Why not just use ext3 and CLVM and have the logical volumes fail-over from node-to-node? It definitely would be a whole lot simpler to deploy. -Ross ______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
On Sun, Feb 10, 2008 at 10:56:30AM -0500, Ross S. W. Walker wrote:> > You can perfectly fine use samba on top of gfs, as long as > > you only share your data from a single node or (more > > precisely) make sure that every directory is only shared via > > a single node. Different directories can be shared via > > different nodes. > > If samba+gfs is really only feasible for single node access then > why even use GFS? Why not just use ext3 and CLVM and have the > logical volumes fail-over from node-to-node? It definitely > would be a whole lot simpler to deploy.Two points: ctdb is made to work around this limitation. So a cluster-enabled Samba can share the same file space via different nodes. Having the storage online on multiple nodes should give you much quicker failover times. No fsck. Volker -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://lists.samba.org/archive/samba/attachments/20080210/72264c2c/attachment.bin