We upgraded to 2.2.6rc2 yesterday to continue our resolving of the printer driver problems. (Thanks Jerry & Rohin) For unrelated reasons I wanted to log into the machine late last night and found Putty couldn't connect. When I get in this morning the server was completely hooped with Out of Memory errors on the screen with SMBD as the offending process. Trying to log into any console resulted in an instant lockup with the same error so I was forced to hard reboot. Anyone had any similar experience? Some relevant info: The server was not rebooted after upgrading to 2.2.6rc2 from Pre2, just the Samba services where re-started. HP's WebJetAdmin Daemon was running with over 20 instances. (Was 2 before). We've taken it off for now as we're not big enough to warrant that level of control. Greg
On Fri, Oct 11, 2002 at 09:41:44AM -0700, gsmith@ahbl.ca wrote:> We upgraded to 2.2.6rc2 yesterday to continue our resolving of the printer > driver problems. (Thanks Jerry & Rohin) For unrelated reasons I wanted to > log into the machine late last night and found Putty couldn't connect. > > When I get in this morning the server was completely hooped with Out of > Memory errors on the screen with SMBD as the offending process. Trying to > log into any console resulted in an instant lockup with the same error so I > was forced to hard reboot.Damn - we could have done with some feedback... Sorry for the problem. Can you watch the machine today and alert us if the problem starts to re-assert itself. Jeremy.
It's winbindd. We had been running ok then noticed that winbindd wasn't running. We started it up then sometime later noticed the server slowing down. Meminfo showed free had dropped from 80+MB to 4. Restaring Samba didn't change things much (Free went to 7MB), restarting winbindd put it back up to 80MB. Has anything changed in winbindd between pre2 and rc2? We were running with pre2 for a couple of days without problems. Greg -----Original Message----- From: jra@dp.samba.org [mailto:jra@dp.samba.org] Sent: Friday, October 11, 2002 10:49 AM To: gsmith@ahbl.ca Subject: Re: [Samba] Memory Leak in 2.2.6rc2?? On Fri, Oct 11, 2002 at 10:45:42AM -0700, gsmith@ahbl.ca wrote:> So far so good. Uptime shows load levels at 0.00, smbd doesn't appear in > top. > > Anything else you recommend I check?I'd recommend something that periodicly (every 30 seconds or so) looks at the output from /proc/meminfo and alerts you if the free area drops low. Also looking at the current RSS size from of all smbd's using ps is a good idea. Jeremy.
Its at it again. Does rc3 address this? The only thing being done on this server right now is printing by one user. Screen capture of top: PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 3826 root 25 0 3096 2832 1540 R 88.0 0.8 9:05 winbindd 3134 root 15 0 3272 2816 2000 S 3.1 0.8 0:01 smbd 3905 root 15 0 2732 2312 1456 S 2.9 0.7 0:01 smbd 3841 root 15 0 24440 23M 1456 S 2.7 7.4 0:43 smbd 3868 root 15 0 14628 13M 1456 S 2.5 4.4 0:24 smbd 3908 root 15 0 1036 1036 848 R 0.3 0.3 0:00 top -----Original Message----- From: jra@dp.samba.org [mailto:jra@dp.samba.org] Sent: Friday, October 11, 2002 1:54 PM To: gsmith@ahbl.ca Cc: samba@lists.samba.org Subject: Re: [Samba] Memory Leak in 2.2.6rc2?? On Fri, Oct 11, 2002 at 12:53:52PM -0700, gsmith@ahbl.ca wrote:> It's winbindd. > > We had been running ok then noticed that winbindd wasn't running. Westarted> it up then sometime later noticed the server slowing down. > > Meminfo showed free had dropped from 80+MB to 4. Restaring Samba didn't > change things much (Free went to 7MB), restarting winbindd put it back upto> 80MB. > > Has anything changed in winbindd between pre2 and rc2? We were runningwith> pre2 for a couple of days without problems.Hmmmm. Can you do a ps on the winbindd and let me know what the SIZE and RSS figures are ? Also, can you let me know more about your domain environment (how many trusted domains etc.). Jeremy.
Same thing with rc3. Browse to a share on the Samba server and winbindd and smbd go nuts. Winbindd takes 90% cpu and an smb daemon take upwards of 12MB. We'll backtrack to pre2 and see how things go. greg -----Original Message----- From: Gerald (Jerry) Carter [mailto:jerry@samba.org] Sent: Friday, October 11, 2002 3:11 PM To: gsmith@ahbl.ca Cc: samba@lists.samba.org Subject: RE: [Samba] Memory Leak in 2.2.6rc2?? On Fri, 11 Oct 2002 gsmith@ahbl.ca wrote:> Has anything changed in winbindd between pre2 and rc2? We were running > with pre2 for a couple of days without problems.Jeremy merged a lot of winbind code from APP_HEAD. I'll have to look at a diff to be sure what the exact changes were. Is anyone else seeing this? cheers, jerry --------------------------------------------------------------------- Hewlett-Packard http://www.hp.com SAMBA Team http://www.samba.org -- http://www.plainjoe.org "SAMS Teach Yourself Samba in 24 Hours" 2ed. ISBN 0-672-32269-2 --"I never saved anything for the swim back." Ethan Hawk in Gattaca--
On Fri, Oct 11, 2002 at 04:35:52PM -0700, gsmith@ahbl.ca wrote:> Same thing with rc3. Browse to a share on the Samba server and winbindd and > smbd go nuts. > > Winbindd takes 90% cpu and an smb daemon take upwards of 12MB.Can you get me a debug level 10 log of this occurring please ? Jeremy.
On Fri, 11 Oct 2002 gsmith@ahbl.ca wrote:> Its at it again. Does rc3 address this? The only thing being done on this > server right now is printing by one user. > > Screen capture of top: > > > PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND > 3826 root 25 0 3096 2832 1540 R 88.0 0.8 9:05 winbindd > 3134 root 15 0 3272 2816 2000 S 3.1 0.8 0:01 smbd > 3905 root 15 0 2732 2312 1456 S 2.9 0.7 0:01 smbd > 3841 root 15 0 24440 23M 1456 S 2.7 7.4 0:43 smbd > 3868 root 15 0 14628 13M 1456 S 2.5 4.4 0:24 smbd > 3908 root 15 0 1036 1036 848 R 0.3 0.3 0:00 topHmmm... can you run "getent group" and see if it keeps looping over the same domain groups? Make sure that "winbind enum [user|group]" are both enable (are by default). Also, are you in a Windows 2000 native mode domain? cheers, jerry --------------------------------------------------------------------- Hewlett-Packard http://www.hp.com SAMBA Team http://www.samba.org -- http://www.plainjoe.org "SAMS Teach Yourself Samba in 24 Hours" 2ed. ISBN 0-672-32269-2 --"I never saved anything for the swim back." Ethan Hawk in Gattaca--
On Sat, 12 Oct 2002, Gerald (Jerry) Carter wrote:> Hmmm... can you run "getent group" and see if it keeps looping > over the same domain groups? Make sure that "winbind enum [user|group]" > are both enable (are by default). > > Also, are you in a Windows 2000 native mode domain?Also still need * smb.conf * build options * server platform & kernel (seem to have forgotten if you mentioned it before) * debug level 10 I think i might have recreated it here so any information you can provide to confirm will be a big help. cheers, jerry --------------------------------------------------------------------- Hewlett-Packard http://www.hp.com SAMBA Team http://www.samba.org -- http://www.plainjoe.org "SAMS Teach Yourself Samba in 24 Hours" 2ed. ISBN 0-672-32269-2 --"I never saved anything for the swim back." Ethan Hawk in Gattaca--
Hi Jerry, i just pulled an all-nighter doing a re-org of all our servers which gave me an opportunity to clean things up. Thi ngs went well with once expection; A clean install of RedHat 7.3 + Samba 2.2.6 was an absolute fscking nightmare. I could join the domain, wbinfo showed groups and users properly, pam.d/samba, nsswitch.conf and smb.conf were configured identically with another samba server that runs great. What I ran into was: - getent would show only local groups - Win2k workstations connecting would only come in as nobody - shares would show up but double-clicking on a share would result in a login dialog. I started with pre2 and worked up to rc3. At some point things went really haywire and I got the looping getent you describe. In answer to your other questions: The domain is mixed mode. Winbind enum users and groups are explicitly specified Server in question is dual PIII No build options - straight from rpm. the one thing I do notice is a noticeable pause when loggin in or for any process requiring authentication (e.g. scp) SMB.CONF: # Global parameters [global] workgroup = AHBLNT1 netbios name = MILTON server string = MILTON File/Print/Web Server security = DOMAIN encrypt passwords = Yes password server = iman log file = /var/log/samba/%m.log max log size = 0 announce version = 4.0 socket options = TCP_NODELAY IPTOS_THROUGHPUT SO_RCVBUF=8192 SO_SNDBUF=8192 domain admin group = @DomainAdmins os level = 1 preferred master = False local master = No domain master = False dns proxy = No wins server = 10.0.10.8 winbind uid = 10000-20000 winbind gid = 10000-20000 winbind enum users = yes winbind enum groups = yes template shell = /bin/false winbind cache time = 10 winbind use default domain = Yes admin users = @DomainAdmins hosts allow = 10. 127. 142.173. max connections = 5000 printer admin = @DomainAdmins,root load printers = yes printing = cups print command = lpr.cups -r -P%p -o raw %s lpq command = lpstat -o %p lprm command = cancel.cups %p-%j large readwrite = yes name resolve order = wins bcast host utmp = yes [ers] comment = ERS Data path = /shares/ers admin users = @DomainUsers write list = @DomainUsers create mask = 0775 directory mask = 0775 force group =DomainUsers [printers] comment = All Printers path = /var/spool/samba printer admin = @DomainAdmins, root, repro printable = Yes browseable = No guest ok = yes [print$] comment = Win2K/NT Printer Drivers path = /etc/samba/drivers write list = @DomainAdmins, root, repro admin users = @DomainAdmins browseable = no read only = no [apps] comment = Standard AHB&L Applications path = /shares/apps force group = DomainUsers read only = No create mask = 0777 directory mask = 0777 admin users = @DomainUsers write list = @DomainUsers nt acl support = yes [data] comment = AHB&L Data path = /shares/data write list = @DomainUsers create mask = 0777 force group = DomainUsers directory mask = 0777 oplocks = false level2 oplocks = false [dexco] comment = Dexco force group = DomainUsers read only = No create mask = 0777 directory mask = 0777 admin users = @DomainUsers write list = @DomainUsers oplocks = false level2 oplocks = false path = /shares/dexco [imanage] comment = production documents browseable=no path = /shares/imanage write list = @DomainAdmins imanage -----Original Message----- From: Gerald (Jerry) Carter [mailto:jerry@samba.org] Sent: Saturday, October 12, 2002 8:25 AM To: gsmith@ahbl.ca Cc: jra@dp.samba.org; samba@lists.samba.org Subject: RE: [Samba] Memory Leak in 2.2.6rc2?? On Sat, 12 Oct 2002, Gerald (Jerry) Carter wrote:> Hmmm... can you run "getent group" and see if it keeps looping > over the same domain groups? Make sure that "winbind enum [user|group]" > are both enable (are by default). > > Also, are you in a Windows 2000 native mode domain?Also still need * smb.conf * build options * server platform & kernel (seem to have forgotten if you mentioned it before) * debug level 10 I think i might have recreated it here so any information you can provide to confirm will be a big help. cheers, jerry --------------------------------------------------------------------- Hewlett-Packard http://www.hp.com SAMBA Team http://www.samba.org -- http://www.plainjoe.org "SAMS Teach Yourself Samba in 24 Hours" 2ed. ISBN 0-672-32269-2 --"I never saved anything for the swim back." Ethan Hawk in Gattaca--
On Sun, 13 Oct 2002 gsmith@ahbl.ca wrote:> What I ran into was: > > - getent would show only local groupsWhat about domain users ?> - Win2k workstations connecting would only come in as nobody > - shares would show up but double-clicking on a share would result in a > login dialog. > > I started with pre2 and worked up to rc3. At some point things went really > haywire and I got the looping getent you describe.Could you grab logs for any of thse failures?> The domain is mixed mode.Just to clarify, this is the RedHat 2.2.6[pre|rc]X rpms from samba.org right?> the one thing I do notice is a noticeable pause when loggin in or for > any process requiring authentication (e.g. scp)This could be solved by setting "winbind enum [user|group]" to no. I get looping in "getent group" regardless.> winbind use default domain = YesThis should be disabled. Not a very well tested option. Everything else your smb.conf looks in fair order. cheers, jerry