Hello,
I'm running Samba 3.0.6 PDC with OpenLDAP 2.1.25 backend on a Linux
machine with RedHat 3.0 ES installed. This is a large installation
with separate Samba BDC and 2 file servers. The BDC server uses a
replica LDAP server, working as slave for the master LDAP server
installed at PDC. The number of domain accounts is about 1850 and
at the moment about 500 machines are added to the Samba domain. The
number of machines increased slowly since April and for the last few
weeks we observed large delays during the domain logons.
The logon process for some Windows machines takes as much as 10-20
minutes (!) For most of the users these times are of course
unacceptable.
Most of the users start their work and logon to the domain between
7:30-8:30 AM. Within these hours the load of the PDC server sometimes
exceeds 100-120. About 90% of the CPU time is utilized by slapd.
The PDC/BDC machines are HP DL-380 server with single Xeon CPU 2.80GHz,
2,5 GB of RAM, no swap and with Gigabit Ethernet interface.
When I turned on the high debug level for both Samba and OpenLDAP
daemons and the problem is that during the processing of the logon
script Samba orders the LDAP backend to perform multiple searches for
all the domain users and repeats it 3 or 4 times. This gives about 8-9
_thousand_ of full LDAP directory searches for single logon session!
The small part of slapd debug file follows:
Sep 27 15:01:09 umwsap11 slapd[16930]: conn=458 op=65 SRCH base="dc=XX
Company,dc=pl" scope=2
filter="(&(uid=umwadd01)(objectClass=sambaSamAccount))"
Sep 27 15:01:09 umwsap11 slapd[16930]: conn=458 op=65 SEARCH RESULT
tag=101 err=0 nentries=1 textSep 27 15:01:09 umwsap11 slapd[16930]: conn=458
op=66 SRCH base="dc=XX
Company,dc=pl" scope=2
filter="(&(uid=umwadd02)(objectClass=sambaSamAccount))"
Sep 27 15:01:09 umwsap11 slapd[16930]: conn=458 op=66 SEARCH RESULT
tag=101 err=0 nentries=1 textSep 27 15:01:09 umwsap11 slapd[16930]: conn=458
op=67 SRCH base="dc=XX
Company,dc=pl" scope=2
filter="(&(uid=umwadd03)(objectClass=sambaSamAccount))"
Sep 27 15:01:09 umwsap11 slapd[16930]: conn=458 op=67 SEARCH RESULT
tag=101 err=0 nentries=1 text
... and so on, for some reason every user must be found in LDAP several
times. All these searches are performed during the logon script
processing. Since many of our users are still using Win98 workstations,
the system "hangs" for them for several minutes with empty screen and
only a logon script window open.
What's more confusing, for some of the domain users only about 60
LDAP searches are performed and they are able to log on to the domain
in a few seconds. I tried to compare their exported ldif data with
users which experience the delays, but there's nothing exceptional,
only their names, UIDs and SIDs are different.
The problem does not depend on the operating system of the workstation
- we've tested Win98, NT, W2000 and XP systems. It seems to be rather
user-centric.
I tried to increase OpenLDAP and nscd performance by setting the thread
number up to 256 and increasing the cache size, but this gives only a
small improvement. The indexes in slapd.conf are defined as
described in the Samba docs:
index default sub
index objectClass eq
index uidNumber,gidNumber eq
index memberUid eq
index cn,sn,uid,displayName pres,sub,eq
index mail,givenname eq,subinitial
index nisMapName,nisMapEntry eq,pres,sub
index homeDirectory,sambaLogonScript eq
index sambaSID eq
index sambaPrimaryGroupSID eq
index sambaDomainName eq
sizelimit -1
cachesize 100000
dbcachesize 15000000
threads 256
We have BDC server configured as the second logon server, but for some
reason only small number of workstation chooses this server as logon
server. Perhaps I should increase the "os level" for the BDC from 33
to 255, as it is configured for the PDC?
The smb.conf of the PDC server follows:
[global]
workgroup = XXCOMP
security = user
server string = XX Company - PDC
passdb backend = ldapsam:ldap://127.0.0.1
idmap backend = ldap:ldap://127.0.0.1
idmap uid = 40000-50000
idmap gid = 40000-50000
log level = 1
log file = /var/log/samba/log.%m
max log size = 500
time server = Yes
socket options = TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE
SO_RCVBUF=8192 SO_SNDBUF=8192
logon path logon drive = K:
logon home = \\fileserv02\homes\%U
#logon script = %U.bat
domain logons = Yes
os level = 255
local master = Yes
preferred master = Yes
domain master = Yes
wins proxy = Yes
wins support = Yes
ldap suffix = dc=XX Company,dc=pl
ldap group suffix = ou=groups
ldap user suffix = ou=people
ldap idmap suffix = ou=idmap,dc=XX Company,dc=pl
ldap machine suffix = ou=machines
ldap admin dn = cn=Manager,dc=XX Company,dc=pl
ldap ssl = no
ldap passwd sync = Yes
remote browse sync = 10.255.255.255 130.130.255.255
printing = cups
hide unreadable = Yes
nt acl support = Yes
admin users = "Domain Admins"
name resolve order = lmhosts wins hosts bcast
ldap timeout = 15
I tried to use the idmap feature od Samba, but for some reason after
creating the "ou=idmap,dc=XX Company,dc=pl" container in LDAP, Samba
does not populate it with SID-GID mappings. Perhaps this is the root
cause of our problem.
The whole Samba domain worked properly without these logon delays for
several months. When the number of users and workstations was small, no
performance problems occcured. Now we have serious problem with about
1/3 of the intended number of domain workstations (500 of ~1500).
Unless I find the solution, our management will probably decide to
migrate from Samba to Active Directory...
Thanks in advance,
Tomasz Finke
> Sep 27 15:01:09 umwsap11 slapd[16930]: conn=458 op=65 SRCH base="dc=XX > Company,dc=pl" scope=2 > filter="(&(uid=umwadd01)(objectClass=sambaSamAccount))" >> ldap suffix = dc=XX Company,dc=pl > ldap group suffix = ou=groups > ldap user suffix = ou=people > ldap idmap suffix = ou=idmap,dc=XX Company,dc=pl > ldap machine suffix = ou=machinesThese entries make me think you could probably speed things up a bit with a tighter search scope. It looks like you're searching the whole DIT every time since you've got your machine and user accounts split up. I'm assuming you also have nss configured to search dc=XX Company,dc=pl?sub. I'd suggest either merging the user and machine OUs or perhaps putting both of them in a container OU you can search in, rather than doing the whole LDAP tree. True, that won't solve the multiple searches problem, but it should help along the speed of the searches that it does do. -- Paul Gienger Office: 701-281-1884 Applied Engineering Inc. Information Systems Consultant Fax: 701-281-1322 URL: www.ae-solutions.com mailto: pgienger@ae-solutions.com
Marcel de Riedmatten
2004-Oct-08 08:24 UTC
[Samba] Samba 3.0.6 and OpenLDAP performance problem
Le jeu 07/10/2004 ? 23:05, Tomasz Finke a ?crit :> Marcel de Riedmatten wrote: > > > You aren't running winbind aren't you ? > > No, I'm not, just slapd, nscd and Samba.I have looked in my log and i see something similar as you, except this append after the logon script is closed. I see at least three time the same enumeration request: Oct 8 09:56:08 sarge slapd[25437]: conn=266107 op=401 SRCH base="dc=nofida,dc=ch" scope=2 filter="(&(uid=*)(objectClass=sambaSamAccount))" Oct 8 09:56:08 sarge slapd[25437]: conn=266107 op=401 SRCH attr=uid uidNumber gidNumber homeDirectory sambaPwdLastSet sambaPwdCanChange sambaPwdMustChange sambaLogonTime sambaLogoffTime sambaKickoffTime cn displayName sambaHomeDrive sambaHomePath sambaLogonScript sambaProfilePath description sambaUserWorkstations sambaSID sambaPrimaryGroupSID sambaLMPassword sambaNTPassword sambaDomainName objectClass sambaAcctFlags sambaMungedDial sambaBadPasswordCount sambaBadPasswordTime Oct 8 09:56:08 sarge slapd[25437]: conn=266107 op=401 SEARCH RESULT tag=101 err=0 nentries=37 text With 37 entries i don't see performance issue. This is with samba 3.05 (debian). I'all check later if this is realy has to do with the logon process. Another question: have you replicated your ldap server ? -- Marcel -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Ceci est une partie de message =?ISO-8859-1?Q?num=E9riquement?= =?ISO-8859-1?Q?_sign=E9e=2E?Url : http://lists.samba.org/archive/samba/attachments/20041008/d16b39a3/attachment.bin
Marcel de Riedmatten
2004-Oct-08 09:37 UTC
[Samba] Samba 3.0.6 and OpenLDAP performance problem
Le ven 08/10/2004 ? 11:01, Tomasz Finke a ?crit :> Marcel de Riedmatten wrote: > > > Another question: have you replicated your ldap server ? > > Yes, I have BDC server with Samba and slave slapd installed. But > more than 90% of users choose PDC as their logon server. The > "os level" at PDC is set to 255 and on the BDC to 33. Perhaps > I should set equal values for both servers?I made some search on that one. You should probably use a 2 PDC setup like the one discribed at http://www.unav.es/cti/ldap-smb/smb-ldap-3-howto.html toward the end of the page. Both PDC have domain master = yes local master = yes preferred master = yes os level = 255 What change is the netbios name and the ldapsam entry doesn't have the same order. -- Marcel de Riedmatten -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Ceci est une partie de message =?ISO-8859-1?Q?num=E9riquement?= =?ISO-8859-1?Q?_sign=E9e=2E?Url : http://lists.samba.org/archive/samba/attachments/20041008/02c758d7/attachment.bin
Andrew Bartlett
2004-Oct-08 13:02 UTC
[Samba] Samba 3.0.6 and OpenLDAP performance problem
On Wed, 2004-10-06 at 21:06, Tomasz Finke wrote:> Hello, > > I'm running Samba 3.0.6 PDC with OpenLDAP 2.1.25 backend on a Linux > machine with RedHat 3.0 ES installed. This is a large installation > with separate Samba BDC and 2 file servers. The BDC server uses a > replica LDAP server, working as slave for the master LDAP server > installed at PDC. The number of domain accounts is about 1850 and > at the moment about 500 machines are added to the Samba domain. The > number of machines increased slowly since April and for the last few > weeks we observed large delays during the domain logons. > > The logon process for some Windows machines takes as much as 10-20 > minutes (!) For most of the users these times are of course > unacceptable.I looked at deploying Samba 3.0.6 at my site, and found that I could not upgrade past the particular Samba 3.0.3 pre-release that we had at the time. I found that certain windows clients would want to know who was in certain groups, and if there were a lot of people in those groups, then all hell broke loose. On the samba-technical list, we have been looking at one potential solution, but I think the patch needs more work to make it robust. Part of the problem is that it looked for 'primary' group members, by scanning the entire password database. This, and possibly the gid->sid lookups, cause the performance issues. At one point I thought that get_sid_list_of_group() in groupdb/mapping.c was the problem, but it's unused now, so you could try current SVN. Really, we need to look at the incoming SMB requests, and what LDAP traffic it produces. With that data, we should be able to pin down what's killing things at your site (which may very well be different to what my problems were). Andrew Bartlett -- Andrew Bartlett abartlet@samba.org Authentication Developer, Samba Team http://samba.org Student Network Administrator, Hawker College abartlet@hawkerc.net -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.samba.org/archive/samba/attachments/20041008/87b80e83/attachment.bin