thr3ads.net - samba - [Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ? [Mar 2017]

If this information is useful, please help other people find it:
Share via:

Andrew Bartlett

2017-Mar-27 09:06 UTC

[Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?

On Mon, 2017-03-27 at 10:43 +0200, Gaetan SLONGO via samba
wrote:> Zarafa is not on the same server as Samba 
> 
> We only have 2 AD/DC Samba 4.5 (CentOS 7) and we put required indexes
> on LDAP . 
> 
> Arround 1000 mailboxes but not all are simultaneously in use (approx
> 1/3 in use). 
> MTA is postfix (and is still connected to Samba AD, this one is not
> causing the issue). 
> As a workarround, we currently deployed a synchronization connector
> from AD to OpenLDAP. It solves the performance issue during the
> investigation because Zarafa was totally unusable at all when
> connected to Samba... But We plan to move to Zimbra by the end of the
> year so I think the work arround can stay in place until the
> migration. However this performance issue could be a bottleneck in
> other applications, in the future... 
> 
> I did not found any config setting allowing tu enable multi-threading 
> on Samba LDAP backend (maybe an hidden one ?).. I think it could help
> a lot 
Given these discussions, I'm keen to add it.  I was going to add this
for Samba 4.6.0, but the initial approach I used was slower in some
cases (the connect/bind/disconnect case).  Sadly at the time there
wasn't this level of concern regarding the LDAP performance, so we
focussed on what we could achieve, which was making NETLOGON multi-
process.

This remains on my radar, along with any other approaches we find along
the way to make search-heavy operation practical. 

I'm sorry this is causing so much trouble, and I look forward to
helping improve this area.  

In the meantime, adding the indexes that your client tools need will
help a lot.

Andrew Bartlett
> ----- Mail original -----
> 
> De: "L.P.H. van Belle via samba" <samba at
lists.samba.org> 
> À: samba at lists.samba.org 
> Envoyé: Lundi 27 Mars 2017 10:26:22 
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> Can you tell more about your setup? 
> 
> Is zarafa and samba on the same server for example. 
> 
> Which MTA are you using postfix/exim? 
> 
> 
> 
> My top was about 150 users, and all my printers are connected also so
> about 200 devices do ldap searches. 
> 
> but my setup is split over 10+ servers ( 2 are AD DC ) 
> 
> 
> 
> So best is to tell what you can about your setup, anonimize if
> needed. 
> 
> 
> 
> Greetz, 
> 
> 
> 
> Louis 
> 
> 
> 
> 
> 
> 
> 
> 
> Van: Gaetan SLONGO [mailto:gslongo at it-optics.com] 
> Verzonden: maandag 27 maart 2017 10:12 
> Aan: L.P.H. van Belle 
> CC: samba at lists.samba.org 
> Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> 
> 
> 
> What we found is Zarafa makes a very big amount of queries, which
> makes Samba run at 100% CPU (one process, LDAP does not seems to be
> multi-threaded..?)... but we have hundreds of users... 
> 
> What do you think could be wrong in the current database/setup ? We
> verified all the setup and everything seems OK 
> 
> 
> De: "L.P.H. van Belle via samba" <samba at
lists.samba.org> 
> À: samba at lists.samba.org 
> Envoyé: Lundi 27 Mars 2017 09:58:55 
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> No, you have to do that manualy, or look the the samba4 ADS script
> for kopano ( or zarafa ) 
> 
> But I mostly follow the documentation. 
> 
> 
> 
> And when i run : 
> 
> time ldbsearch -H /var/lib/samba/private/sam.ldb -s base -b
> @INDEXLIST 
> 
> .... 
> 
> real 0m0.230s 
> 
> user 0m0.184s 
> 
> sys 0m0.044s 
> 
> 
> 
> so if yours take more that 20 sec there is something very wrong. 
> 
> I suggest check you samba AD database and samba4 ADDC setup, 
> 
> i dont think this is zarafa related. 
> 
> 
> 
> 
> 
> Greetz, 
> 
> 
> 
> Louis 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Van: Gaetan SLONGO [mailto:gslongo at it-optics.com] 
> Verzonden: maandag 27 maart 2017 8:46 
> Aan: L.P.H. van Belle 
> CC: samba at lists.samba.org 
> Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> 
> 
> 
> Hi ! 
> 
> Thanks for answer. Yes we use zarafaAccount in search filter. 
> There is an installer provided for Samba4 to install new schemas ? 
> 
> Thanks ! 
> 
> 
> De: "L.P.H. van Belle via samba" <samba at
lists.samba.org> 
> À: samba at lists.samba.org 
> Envoyé: Jeudi 23 Mars 2017 11:54:50 
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> 
> Are use using zarafaAccount=1 withing the search filters? 
> I use this things like this : 
> 
> (&(objectClass=person)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s)))
>  
> Or for groups. 
> (&(objectclass=group)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s))) 
> 
> That helps a lot. 
> 
> ! If you switch to kopano beware to change the SCHEMA and filters 
> zarafaAccount changed to kopanoAccount 
> 
> 
> Greetz. 
> 
> Louis 
> 
> 
> > -----Oorspronkelijk bericht----- 
> > Van: samba [mailto:samba-bounces at lists.samba.org] Namens Gaetan
> > SLONGO via 
> > samba 
> > Verzonden: donderdag 23 maart 2017 11:12 
> > Aan: samba at lists.samba.org 
> > Onderwerp: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> > unusable), 
> > performance tunning ? 
> > Urgentie: Hoog 
> > 
> > 
> > Dear users, 
> > 
> > We are facing to a big latency issue regarding the LDAP Server
> > (both 
> > encrypted & plain). 
> > 
> > We have a Zarafa mail server which makes a lot of queries and puts
> > a samba 
> > process to 100% usage. This latency makes the mail server
> > unusable.. The 
> > mail server was previously on OpenLDAP and there was not
> > performance 
> > issues. 
> > 
> > A simple LDAP query can take up to 25 sec to perform !! 
> > 
> > We have added some indexes : 
> > 
> > [root at califix ~]# ldbsearch -H /var/lib/samba/private/sam.ldb -s
> > base -b 
> > @INDEXLIST 
> > # record 1 
> > dn: @INDEXLIST 
> > @IDXONE: 1 
> > @IDXVERSION: 2 
> > @IDXATTR: objectClass 
> > @IDXATTR: msDS-Cached-Membership-Time-Stamp 
> > @IDXATTR: userPrincipalName 
> > @IDXATTR: rpcNsInterfaceID 
> > @IDXATTR: fileExtPriority 
> > @IDXATTR: dnsRoot 
> > @IDXATTR: mSMQLabelEx 
> > @IDXATTR: dNSTombstoned 
> > @IDXATTR: msDS-PhoneticCompanyName 
> > @IDXATTR: msSFU30Domains 
> > @IDXATTR: dhcpType 
> > @IDXATTR: ou 
> > @IDXATTR: gidNumber 
> > @IDXATTR: msFVE-VolumeGuid 
> > @IDXATTR: msTSManagingLS2 
> > @IDXATTR: implementedCategories 
> > @IDXATTR: oMTIndxGuid 
> > @IDXATTR: cOMClassID 
> > @IDXATTR: volTableIdxGUID 
> > @IDXATTR: l 
> > @IDXATTR: mSMQDigests 
> > @IDXATTR: msTSExpireDate4 
> > @IDXATTR: flatName 
> > @IDXATTR: msSFU30YpServers 
> > @IDXATTR: packageFlags 
> > @IDXATTR: mSMQOwnerID 
> > @IDXATTR: objectCategory 
> > @IDXATTR: msSFU30IsValidContainer 
> > @IDXATTR: msTSProperty02 
> > @IDXATTR: mS-DS-CreatorSID 
> > @IDXATTR: proxyAddresses 
> > @IDXATTR: msPKI-Cert-Template-OID 
> > @IDXATTR: uNCName 
> > @IDXATTR: mS-SQL-Name 
> > @IDXATTR: fSMORoleOwner 
> > @IDXATTR: msSFU30NisDomain 
> > @IDXATTR: otherMailbox 
> > @IDXATTR: location 
> > @IDXATTR: msSFU30NetgroupHostAtDomain 
> > @IDXATTR: uSNChanged 
> > @IDXATTR: sIDHistory 
> > @IDXATTR: birthLocation 
> > @IDXATTR: msDS-SecondaryKrbTgtNumber 
> > @IDXATTR: msTSProperty01 
> > @IDXATTR: msTSManagingLS4 
> > @IDXATTR: msSFU30OrderNumber 
> > @IDXATTR: msDS-HABSeniorityIndex 
> > @IDXATTR: primaryGroupID 
> > @IDXATTR: mSMQQueueType 
> > @IDXATTR: msDFSR-ReplicationGroupGuid 
> > @IDXATTR: msDS-PhoneticDepartment 
> > @IDXATTR: mail 
> > @IDXATTR: msSFU30Name 
> > @IDXATTR: msSFU30NetgroupUserAtDomain 
> > @IDXATTR: fromServer 
> > @IDXATTR: displayName 
> > @IDXATTR: msTSLicenseVersion2 
> > @IDXATTR: groupType 
> > @IDXATTR: msTSLicenseVersion3 
> > @IDXATTR: msTSLicenseVersion4 
> > @IDXATTR: userAccountControl 
> > @IDXATTR: physicalLocationObject 
> > @IDXATTR: servicePrincipalName 
> > @IDXATTR: msTSExpireDate 
> > @IDXATTR: serviceClassName 
> > @IDXATTR: lDAPDisplayName 
> > @IDXATTR: zarafaAccount 
> > @IDXATTR: terminalServer 
> > @IDXATTR: givenName 
> > @IDXATTR: msTSManagingLS3 
> > @IDXATTR: msSFU30MaxUidNumber 
> > @IDXATTR: msDS-Entry-Time-To-Die 
> > @IDXATTR: msTSLSProperty01 
> > @IDXATTR: msDS-PhoneticFirstName 
> > @IDXATTR: trustPartner 
> > @IDXATTR: msTSLSProperty02 
> > @IDXATTR: msTSExpireDate3 
> > @IDXATTR: objectGUID 
> > @IDXATTR: showInAdvancedViewOnly 
> > @IDXATTR: rpcNsTransferSyntax 
> > @IDXATTR: sAMAccountName 
> > @IDXATTR: mS-SQL-Version 
> > @IDXATTR: msDS-Site-Affinity 
> > @IDXATTR: sn 
> > @IDXATTR: name 
> > @IDXATTR: nETBIOSName 
> > @IDXATTR: sAMAccountType 
> > @IDXATTR: msTSManagingLS 
> > @IDXATTR: msDFSR-DfsPath 
> > @IDXATTR: altSecurityIdentities 
> > @IDXATTR: USNIntersite 
> > @IDXATTR: msSFU30MasterServerName 
> > @IDXATTR: msDS-PhoneticLastName 
> > @IDXATTR: cn 
> > @IDXATTR: netbootGUID 
> > @IDXATTR: lastLogonTimestamp 
> > @IDXATTR: legacyExchangeDN 
> > @IDXATTR: mSMQLabel 
> > @IDXATTR: uSNCreated 
> > @IDXATTR: mS-SQL-Database 
> > @IDXATTR: msDS-PhoneticDisplayName 
> > @IDXATTR: msSFU30MaxGidNumber 
> > @IDXATTR: rpcNsObjectID 
> > @IDXATTR: timeVolChange 
> > @IDXATTR: msTSExpireDate2 
> > @IDXATTR: groupAttributes 
> > @IDXATTR: physicalDeliveryOfficeName 
> > @IDXATTR: msFVE-RecoveryGuid 
> > @IDXATTR: msDS-AdditionalSamAccountName 
> > @IDXATTR: objectSid 
> > @IDXATTR: keywords 
> > @IDXATTR: mS-SQL-Alias 
> > @IDXATTR: invocationId 
> > @IDXATTR: msTSLicenseVersion 
> > @IDXATTR: requiredCategories 
> > @IDXATTR: msDS-AzObjectGuid 
> > distinguishedName: @INDEXLIST 
> > 
> > There is any way to improve LDAP responses times ? It seems there
> > is only 
> > one process which is managing LDAP queries (no forks/threads?) 
> > 
> > Thank you in advance for your help !! 
> > 
> > -- 
> > To unsubscribe from this list go to the following URL and read the 
> > instructions: https://lists.samba.org/mailman/options/samba 
> 
> 
> 
> -- 
> To unsubscribe from this list go to the following URL and read the 
> instructions: https://lists.samba.org/mailman/options/samba 
> 
> 
> 
> 
> 
> -- 
> 
> 
> 
> www.it-optics.com 
> 
> Gaëtan SLONGO | Head of Infrastructure Department 
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> 
> Company : 
> 
> +32 (0)65 84 23 85 
> 
> Direct : 
> 
> +32 (0)65 32 85 88 
> 
> Fax : 
> 
> +32 (0)65 84 66 76 
> 
> Skype ID : 
> 
> gslongo.pro 
> 
> GPG Key : 
> 
> gslongo-gpg_key.asc 
> 
> 
> 
> 
> 
> 
> 
> - Please consider your environmental responsibility before printing
> this e-mail - 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> To unsubscribe from this list go to the following URL and read the 
> instructions: https://lists.samba.org/mailman/options/samba 
> 
> 
> 
> 
> 
> -- 
> 
> 
> 
> www.it-optics.com 
> 
> Gaëtan SLONGO | Head of Infrastructure Department 
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> 
> Company : 
> 
> +32 (0)65 84 23 85 
> 
> Direct : 
> 
> +32 (0)65 32 85 88 
> 
> Fax : 
> 
> +32 (0)65 84 66 76 
> 
> Skype ID : 
> 
> gslongo.pro 
> 
> GPG Key : 
> 
> gslongo-gpg_key.asc 
> 
> 
> 
> 
> 
> 
> 
> - Please consider your environmental responsibility before printing
> this e-mail - 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> To unsubscribe from this list go to the following URL and read the 
> instructions: https://lists.samba.org/mailman/options/samba 
> 
> 
> 
> -- 
> 
> 
> 
> 
> www.it-optics.com 
> 	
> Gaëtan SLONGO | Head of Infrastructure Department 
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> Company : 	+32 (0)65 84 23 85 
> Direct : 	+32 (0)65 32 85 88 
> Fax : 	+32 (0)65 84 66 76 
> Skype ID : 	gslongo.pro 
> GPG Key : 	gslongo-gpg_key.asc 
> 	
> 
> - Please consider your environmental responsibility before printing
> this e-mail - 
> 
> 
> 
> 
> 
> 
> 
> -- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba

Gaetan SLONGO

2017-Mar-27 09:11 UTC

head link

[Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?

Thank you for your message Andrew. Then .. Waiting for this improvement... :-) 

Indexes seem correct to me. But I'm not sure at 100% because I cannot find
clear explanation regarding the searchFlags attribute value .. 1, 2, ... ? Maybe
you have one ?

Thanks ! 

----- Mail original -----

De: "Andrew Bartlett" <abartlet at samba.org> 
À: "Gaetan SLONGO" <gslongo at it-optics.com>, "L.P.H. van
Belle" <belle at bazuin.nl>
Cc: samba at lists.samba.org 
Envoyé: Lundi 27 Mars 2017 11:06:56 
Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable),
performance tunning ?

On Mon, 2017-03-27 at 10:43 +0200, Gaetan SLONGO via samba wrote:
> Zarafa is not on the same server as Samba 
> 
> We only have 2 AD/DC Samba 4.5 (CentOS 7) and we put required indexes 
> on LDAP . 
> 
> Arround 1000 mailboxes but not all are simultaneously in use (approx 
> 1/3 in use). 
> MTA is postfix (and is still connected to Samba AD, this one is not 
> causing the issue). 
> As a workarround, we currently deployed a synchronization connector 
> from AD to OpenLDAP. It solves the performance issue during the 
> investigation because Zarafa was totally unusable at all when 
> connected to Samba... But We plan to move to Zimbra by the end of the 
> year so I think the work arround can stay in place until the 
> migration. However this performance issue could be a bottleneck in 
> other applications, in the future... 
> 
> I did not found any config setting allowing tu enable multi-threading 
> on Samba LDAP backend (maybe an hidden one ?).. I think it could help 
> a lot 
Given these discussions, I'm keen to add it. I was going to add this 
for Samba 4.6.0, but the initial approach I used was slower in some 
cases (the connect/bind/disconnect case). Sadly at the time there 
wasn't this level of concern regarding the LDAP performance, so we 
focussed on what we could achieve, which was making NETLOGON multi- 
process. 

This remains on my radar, along with any other approaches we find along 
the way to make search-heavy operation practical. 

I'm sorry this is causing so much trouble, and I look forward to 
helping improve this area. 

In the meantime, adding the indexes that your client tools need will 
help a lot. 

Andrew Bartlett 
> ----- Mail original ----- 
> 
> De: "L.P.H. van Belle via samba" <samba at lists.samba.org>
> À: samba at lists.samba.org 
> Envoyé: Lundi 27 Mars 2017 10:26:22 
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> unusable), performance tunning ? 
> 
> Can you tell more about your setup? 
> 
> Is zarafa and samba on the same server for example. 
> 
> Which MTA are you using postfix/exim? 
> 
> 
> 
> My top was about 150 users, and all my printers are connected also so 
> about 200 devices do ldap searches. 
> 
> but my setup is split over 10+ servers ( 2 are AD DC ) 
> 
> 
> 
> So best is to tell what you can about your setup, anonimize if 
> needed. 
> 
> 
> 
> Greetz, 
> 
> 
> 
> Louis 
> 
> 
> 
> 
> 
> 
> 
> 
> Van: Gaetan SLONGO [mailto:gslongo at it-optics.com] 
> Verzonden: maandag 27 maart 2017 10:12 
> Aan: L.P.H. van Belle 
> CC: samba at lists.samba.org 
> Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> unusable), performance tunning ? 
> 
> 
> 
> 
> What we found is Zarafa makes a very big amount of queries, which 
> makes Samba run at 100% CPU (one process, LDAP does not seems to be 
> multi-threaded..?)... but we have hundreds of users... 
> 
> What do you think could be wrong in the current database/setup ? We 
> verified all the setup and everything seems OK 
> 
> 
> De: "L.P.H. van Belle via samba" <samba at lists.samba.org>
> À: samba at lists.samba.org 
> Envoyé: Lundi 27 Mars 2017 09:58:55 
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> unusable), performance tunning ? 
> 
> No, you have to do that manualy, or look the the samba4 ADS script 
> for kopano ( or zarafa ) 
> 
> But I mostly follow the documentation. 
> 
> 
> 
> And when i run : 
> 
> time ldbsearch -H /var/lib/samba/private/sam.ldb -s base -b 
> @INDEXLIST 
> 
> .... 
> 
> real 0m0.230s 
> 
> user 0m0.184s 
> 
> sys 0m0.044s 
> 
> 
> 
> so if yours take more that 20 sec there is something very wrong. 
> 
> I suggest check you samba AD database and samba4 ADDC setup, 
> 
> i dont think this is zarafa related. 
> 
> 
> 
> 
> 
> Greetz, 
> 
> 
> 
> Louis 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Van: Gaetan SLONGO [mailto:gslongo at it-optics.com] 
> Verzonden: maandag 27 maart 2017 8:46 
> Aan: L.P.H. van Belle 
> CC: samba at lists.samba.org 
> Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> unusable), performance tunning ? 
> 
> 
> 
> 
> Hi ! 
> 
> Thanks for answer. Yes we use zarafaAccount in search filter. 
> There is an installer provided for Samba4 to install new schemas ? 
> 
> Thanks ! 
> 
> 
> De: "L.P.H. van Belle via samba" <samba at lists.samba.org>
> À: samba at lists.samba.org 
> Envoyé: Jeudi 23 Mars 2017 11:54:50 
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> unusable), performance tunning ? 
> 
> 
> Are use using zarafaAccount=1 withing the search filters? 
> I use this things like this : 
> 
> (&(objectClass=person)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s))) 
> 
> Or for groups. 
> (&(objectclass=group)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s))) 
> 
> That helps a lot. 
> 
> ! If you switch to kopano beware to change the SCHEMA and filters 
> zarafaAccount changed to kopanoAccount 
> 
> 
> Greetz. 
> 
> Louis 
> 
> 
> > -----Oorspronkelijk bericht----- 
> > Van: samba [mailto:samba-bounces at lists.samba.org] Namens Gaetan 
> > SLONGO via 
> > samba 
> > Verzonden: donderdag 23 maart 2017 11:12 
> > Aan: samba at lists.samba.org 
> > Onderwerp: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> > unusable), 
> > performance tunning ? 
> > Urgentie: Hoog 
> > 
> > 
> > Dear users, 
> > 
> > We are facing to a big latency issue regarding the LDAP Server 
> > (both 
> > encrypted & plain). 
> > 
> > We have a Zarafa mail server which makes a lot of queries and puts 
> > a samba 
> > process to 100% usage. This latency makes the mail server 
> > unusable.. The 
> > mail server was previously on OpenLDAP and there was not 
> > performance 
> > issues. 
> > 
> > A simple LDAP query can take up to 25 sec to perform !! 
> > 
> > We have added some indexes : 
> > 
> > [root at califix ~]# ldbsearch -H /var/lib/samba/private/sam.ldb -s 
> > base -b 
> > @INDEXLIST 
> > # record 1 
> > dn: @INDEXLIST 
> > @IDXONE: 1 
> > @IDXVERSION: 2 
> > @IDXATTR: objectClass 
> > @IDXATTR: msDS-Cached-Membership-Time-Stamp 
> > @IDXATTR: userPrincipalName 
> > @IDXATTR: rpcNsInterfaceID 
> > @IDXATTR: fileExtPriority 
> > @IDXATTR: dnsRoot 
> > @IDXATTR: mSMQLabelEx 
> > @IDXATTR: dNSTombstoned 
> > @IDXATTR: msDS-PhoneticCompanyName 
> > @IDXATTR: msSFU30Domains 
> > @IDXATTR: dhcpType 
> > @IDXATTR: ou 
> > @IDXATTR: gidNumber 
> > @IDXATTR: msFVE-VolumeGuid 
> > @IDXATTR: msTSManagingLS2 
> > @IDXATTR: implementedCategories 
> > @IDXATTR: oMTIndxGuid 
> > @IDXATTR: cOMClassID 
> > @IDXATTR: volTableIdxGUID 
> > @IDXATTR: l 
> > @IDXATTR: mSMQDigests 
> > @IDXATTR: msTSExpireDate4 
> > @IDXATTR: flatName 
> > @IDXATTR: msSFU30YpServers 
> > @IDXATTR: packageFlags 
> > @IDXATTR: mSMQOwnerID 
> > @IDXATTR: objectCategory 
> > @IDXATTR: msSFU30IsValidContainer 
> > @IDXATTR: msTSProperty02 
> > @IDXATTR: mS-DS-CreatorSID 
> > @IDXATTR: proxyAddresses 
> > @IDXATTR: msPKI-Cert-Template-OID 
> > @IDXATTR: uNCName 
> > @IDXATTR: mS-SQL-Name 
> > @IDXATTR: fSMORoleOwner 
> > @IDXATTR: msSFU30NisDomain 
> > @IDXATTR: otherMailbox 
> > @IDXATTR: location 
> > @IDXATTR: msSFU30NetgroupHostAtDomain 
> > @IDXATTR: uSNChanged 
> > @IDXATTR: sIDHistory 
> > @IDXATTR: birthLocation 
> > @IDXATTR: msDS-SecondaryKrbTgtNumber 
> > @IDXATTR: msTSProperty01 
> > @IDXATTR: msTSManagingLS4 
> > @IDXATTR: msSFU30OrderNumber 
> > @IDXATTR: msDS-HABSeniorityIndex 
> > @IDXATTR: primaryGroupID 
> > @IDXATTR: mSMQQueueType 
> > @IDXATTR: msDFSR-ReplicationGroupGuid 
> > @IDXATTR: msDS-PhoneticDepartment 
> > @IDXATTR: mail 
> > @IDXATTR: msSFU30Name 
> > @IDXATTR: msSFU30NetgroupUserAtDomain 
> > @IDXATTR: fromServer 
> > @IDXATTR: displayName 
> > @IDXATTR: msTSLicenseVersion2 
> > @IDXATTR: groupType 
> > @IDXATTR: msTSLicenseVersion3 
> > @IDXATTR: msTSLicenseVersion4 
> > @IDXATTR: userAccountControl 
> > @IDXATTR: physicalLocationObject 
> > @IDXATTR: servicePrincipalName 
> > @IDXATTR: msTSExpireDate 
> > @IDXATTR: serviceClassName 
> > @IDXATTR: lDAPDisplayName 
> > @IDXATTR: zarafaAccount 
> > @IDXATTR: terminalServer 
> > @IDXATTR: givenName 
> > @IDXATTR: msTSManagingLS3 
> > @IDXATTR: msSFU30MaxUidNumber 
> > @IDXATTR: msDS-Entry-Time-To-Die 
> > @IDXATTR: msTSLSProperty01 
> > @IDXATTR: msDS-PhoneticFirstName 
> > @IDXATTR: trustPartner 
> > @IDXATTR: msTSLSProperty02 
> > @IDXATTR: msTSExpireDate3 
> > @IDXATTR: objectGUID 
> > @IDXATTR: showInAdvancedViewOnly 
> > @IDXATTR: rpcNsTransferSyntax 
> > @IDXATTR: sAMAccountName 
> > @IDXATTR: mS-SQL-Version 
> > @IDXATTR: msDS-Site-Affinity 
> > @IDXATTR: sn 
> > @IDXATTR: name 
> > @IDXATTR: nETBIOSName 
> > @IDXATTR: sAMAccountType 
> > @IDXATTR: msTSManagingLS 
> > @IDXATTR: msDFSR-DfsPath 
> > @IDXATTR: altSecurityIdentities 
> > @IDXATTR: USNIntersite 
> > @IDXATTR: msSFU30MasterServerName 
> > @IDXATTR: msDS-PhoneticLastName 
> > @IDXATTR: cn 
> > @IDXATTR: netbootGUID 
> > @IDXATTR: lastLogonTimestamp 
> > @IDXATTR: legacyExchangeDN 
> > @IDXATTR: mSMQLabel 
> > @IDXATTR: uSNCreated 
> > @IDXATTR: mS-SQL-Database 
> > @IDXATTR: msDS-PhoneticDisplayName 
> > @IDXATTR: msSFU30MaxGidNumber 
> > @IDXATTR: rpcNsObjectID 
> > @IDXATTR: timeVolChange 
> > @IDXATTR: msTSExpireDate2 
> > @IDXATTR: groupAttributes 
> > @IDXATTR: physicalDeliveryOfficeName 
> > @IDXATTR: msFVE-RecoveryGuid 
> > @IDXATTR: msDS-AdditionalSamAccountName 
> > @IDXATTR: objectSid 
> > @IDXATTR: keywords 
> > @IDXATTR: mS-SQL-Alias 
> > @IDXATTR: invocationId 
> > @IDXATTR: msTSLicenseVersion 
> > @IDXATTR: requiredCategories 
> > @IDXATTR: msDS-AzObjectGuid 
> > distinguishedName: @INDEXLIST 
> > 
> > There is any way to improve LDAP responses times ? It seems there 
> > is only 
> > one process which is managing LDAP queries (no forks/threads?) 
> > 
> > Thank you in advance for your help !! 
> > 
> > -- 
> > To unsubscribe from this list go to the following URL and read the 
> > instructions: https://lists.samba.org/mailman/options/samba 
> 
> 
> 
> -- 
> To unsubscribe from this list go to the following URL and read the 
> instructions: https://lists.samba.org/mailman/options/samba 
> 
> 
> 
> 
> 
> -- 
> 
> 
> 
> www.it-optics.com 
> 
> Gaëtan SLONGO | Head of Infrastructure Department 
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> 
> Company : 
> 
> +32 (0)65 84 23 85 
> 
> Direct : 
> 
> +32 (0)65 32 85 88 
> 
> Fax : 
> 
> +32 (0)65 84 66 76 
> 
> Skype ID : 
> 
> gslongo.pro 
> 
> GPG Key : 
> 
> gslongo-gpg_key.asc 
> 
> 
> 
> 
> 
> 
> 
> - Please consider your environmental responsibility before printing 
> this e-mail - 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> To unsubscribe from this list go to the following URL and read the 
> instructions: https://lists.samba.org/mailman/options/samba 
> 
> 
> 
> 
> 
> -- 
> 
> 
> 
> www.it-optics.com 
> 
> Gaëtan SLONGO | Head of Infrastructure Department 
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> 
> Company : 
> 
> +32 (0)65 84 23 85 
> 
> Direct : 
> 
> +32 (0)65 32 85 88 
> 
> Fax : 
> 
> +32 (0)65 84 66 76 
> 
> Skype ID : 
> 
> gslongo.pro 
> 
> GPG Key : 
> 
> gslongo-gpg_key.asc 
> 
> 
> 
> 
> 
> 
> 
> - Please consider your environmental responsibility before printing 
> this e-mail - 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> To unsubscribe from this list go to the following URL and read the 
> instructions: https://lists.samba.org/mailman/options/samba 
> 
> 
> 
> -- 
> 
> 
> 
> 
> www.it-optics.com 
> 
> Gaëtan SLONGO | Head of Infrastructure Department 
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> Company : +32 (0)65 84 23 85 
> Direct : +32 (0)65 32 85 88 
> Fax : +32 (0)65 84 66 76 
> Skype ID : gslongo.pro 
> GPG Key : gslongo-gpg_key.asc 
> 
> 
> - Please consider your environmental responsibility before printing 
> this e-mail - 
> 
> 
> 
> 
> 
> 
> 
> -- 
Andrew Bartlett http://samba.org/~abartlet/ 
Authentication Developer, Samba Team http://samba.org 
Samba Developer, Catalyst IT http://catalyst.net.nz/services/samba 




-- 




www.it-optics.com 
	
Gaëtan SLONGO | Head of Infrastructure Department 
Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
Company : 	+32 (0)65 84 23 85 
Direct : 	+32 (0)65 32 85 88 
Fax : 	+32 (0)65 84 66 76 
Skype ID : 	gslongo.pro 
GPG Key : 	gslongo-gpg_key.asc 
	

- Please consider your environmental responsibility before printing this e-mail
-

L.P.H. van Belle

2017-Mar-27 10:05 UTC

head link

[Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?

I reviewed you indexing list again against mine. 

 

A diff between your indexing list and mine shows just a small difference.

 
> @IDXATTR: otherMailbox
109d109

< @IDXATTR: uidNumber

116a117
> @IDXATTR: zarafaAccount
 

Left (<) me, right (>) yours.

 

You did you install the zarafa schema in the AD? 

I did use : zarafa_schema_add.sh 

 

Beside that i dont see much. 

 

 

Greetz, 

 

Louis

 

 

 

 


Van: Gaetan SLONGO [mailto:gslongo at it-optics.com] 
Verzonden: maandag 27 maart 2017 11:12
Aan: Andrew Bartlett
CC: samba at lists.samba.org; L.P.H. van Belle
Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable),
performance tunning ?


 

Thank you for your message Andrew. Then .. Waiting for this improvement... :-)

Indexes seem correct to me. But I'm not sure at 100% because I cannot find
clear explanation regarding the searchFlags attribute value .. 1, 2, ... ? Maybe
you have one ?

Thanks !


De: "Andrew Bartlett" <abartlet at samba.org>
À: "Gaetan SLONGO" <gslongo at it-optics.com>, "L.P.H. van
Belle" <belle at bazuin.nl>
Cc: samba at lists.samba.org
Envoyé: Lundi 27 Mars 2017 11:06:56
Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable),
performance tunning ?

On Mon, 2017-03-27 at 10:43 +0200, Gaetan SLONGO via samba
wrote:> Zarafa is not on the same server as Samba 
> 
> We only have 2 AD/DC Samba 4.5 (CentOS 7) and we put required indexes
> on LDAP . 
> 
> Arround 1000 mailboxes but not all are simultaneously in use (approx
> 1/3 in use). 
> MTA is postfix (and is still connected to Samba AD, this one is not
> causing the issue). 
> As a workarround, we currently deployed a synchronization connector
> from AD to OpenLDAP. It solves the performance issue during the
> investigation because Zarafa was totally unusable at all when
> connected to Samba... But We plan to move to Zimbra by the end of the
> year so I think the work arround can stay in place until the
> migration. However this performance issue could be a bottleneck in
> other applications, in the future... 
> 
> I did not found any config setting allowing tu enable multi-threading 
> on Samba LDAP backend (maybe an hidden one ?).. I think it could help
> a lot 
Given these discussions, I'm keen to add it.  I was going to add this
for Samba 4.6.0, but the initial approach I used was slower in some
cases (the connect/bind/disconnect case).  Sadly at the time there
wasn't this level of concern regarding the LDAP performance, so we
focussed on what we could achieve, which was making NETLOGON multi-
process.

This remains on my radar, along with any other approaches we find along
the way to make search-heavy operation practical. 

I'm sorry this is causing so much trouble, and I look forward to
helping improve this area.  

In the meantime, adding the indexes that your client tools need will
help a lot.

Andrew Bartlett
> ----- Mail original -----
> 
> De: "L.P.H. van Belle via samba" <samba at
lists.samba.org> 
> À: samba at lists.samba.org 
> Envoyé: Lundi 27 Mars 2017 10:26:22 
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> Can you tell more about your setup? 
> 
> Is zarafa and samba on the same server for example. 
> 
> Which MTA are you using postfix/exim? 
> 
> 
> 
> My top was about 150 users, and all my printers are connected also so
> about 200 devices do ldap searches. 
> 
> but my setup is split over 10+ servers ( 2 are AD DC ) 
> 
> 
> 
> So best is to tell what you can about your setup, anonimize if
> needed. 
> 
> 
> 
> Greetz, 
> 
> 
> 
> Louis 
> 
> 
> 
> 
> 
> 
> 
> 
> Van: Gaetan SLONGO [mailto:gslongo at it-optics.com] 
> Verzonden: maandag 27 maart 2017 10:12 
> Aan: L.P.H. van Belle 
> CC: samba at lists.samba.org 
> Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> 
> 
> 
> What we found is Zarafa makes a very big amount of queries, which
> makes Samba run at 100% CPU (one process, LDAP does not seems to be
> multi-threaded..?)... but we have hundreds of users... 
> 
> What do you think could be wrong in the current database/setup ? We
> verified all the setup and everything seems OK 
> 
> 
> De: "L.P.H. van Belle via samba" <samba at
lists.samba.org> 
> À: samba at lists.samba.org 
> Envoyé: Lundi 27 Mars 2017 09:58:55 
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> No, you have to do that manualy, or look the the samba4 ADS script
> for kopano ( or zarafa ) 
> 
> But I mostly follow the documentation. 
> 
> 
> 
> And when i run : 
> 
> time ldbsearch -H /var/lib/samba/private/sam.ldb -s base -b
> @INDEXLIST 
> 
> .... 
> 
> real 0m0.230s 
> 
> user 0m0.184s 
> 
> sys 0m0.044s 
> 
> 
> 
> so if yours take more that 20 sec there is something very wrong. 
> 
> I suggest check you samba AD database and samba4 ADDC setup, 
> 
> i dont think this is zarafa related. 
> 
> 
> 
> 
> 
> Greetz, 
> 
> 
> 
> Louis 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Van: Gaetan SLONGO [mailto:gslongo at it-optics.com] 
> Verzonden: maandag 27 maart 2017 8:46 
> Aan: L.P.H. van Belle 
> CC: samba at lists.samba.org 
> Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> 
> 
> 
> Hi ! 
> 
> Thanks for answer. Yes we use zarafaAccount in search filter. 
> There is an installer provided for Samba4 to install new schemas ? 
> 
> Thanks ! 
> 
> 
> De: "L.P.H. van Belle via samba" <samba at
lists.samba.org> 
> À: samba at lists.samba.org 
> Envoyé: Jeudi 23 Mars 2017 11:54:50 
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> 
> Are use using zarafaAccount=1 withing the search filters? 
> I use this things like this : 
> 
> (&(objectClass=person)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s)))
>  
> Or for groups. 
> (&(objectclass=group)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s))) 
> 
> That helps a lot. 
> 
> ! If you switch to kopano beware to change the SCHEMA and filters 
> zarafaAccount changed to kopanoAccount 
> 
> 
> Greetz. 
> 
> Louis 
> 
> 
> > -----Oorspronkelijk bericht----- 
> > Van: samba [mailto:samba-bounces at lists.samba.org] Namens Gaetan
> > SLONGO via 
> > samba 
> > Verzonden: donderdag 23 maart 2017 11:12 
> > Aan: samba at lists.samba.org 
> > Onderwerp: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> > unusable), 
> > performance tunning ? 
> > Urgentie: Hoog 
> > 
> > 
> > Dear users, 
> > 
> > We are facing to a big latency issue regarding the LDAP Server
> > (both 
> > encrypted & plain). 
> > 
> > We have a Zarafa mail server which makes a lot of queries and puts
> > a samba 
> > process to 100% usage. This latency makes the mail server
> > unusable.. The 
> > mail server was previously on OpenLDAP and there was not
> > performance 
> > issues. 
> > 
> > A simple LDAP query can take up to 25 sec to perform !! 
> > 
> > We have added some indexes : 
> > 
> > [root at califix ~]# ldbsearch -H /var/lib/samba/private/sam.ldb -s
> > base -b 
> > @INDEXLIST 
> > # record 1 
> > dn: @INDEXLIST 
> > @IDXONE: 1 
> > @IDXVERSION: 2 
> > @IDXATTR: objectClass 
> > @IDXATTR: msDS-Cached-Membership-Time-Stamp 
> > @IDXATTR: userPrincipalName 
> > @IDXATTR: rpcNsInterfaceID 
> > @IDXATTR: fileExtPriority 
> > @IDXATTR: dnsRoot 
> > @IDXATTR: mSMQLabelEx 
> > @IDXATTR: dNSTombstoned 
> > @IDXATTR: msDS-PhoneticCompanyName 
> > @IDXATTR: msSFU30Domains 
> > @IDXATTR: dhcpType 
> > @IDXATTR: ou 
> > @IDXATTR: gidNumber 
> > @IDXATTR: msFVE-VolumeGuid 
> > @IDXATTR: msTSManagingLS2 
> > @IDXATTR: implementedCategories 
> > @IDXATTR: oMTIndxGuid 
> > @IDXATTR: cOMClassID 
> > @IDXATTR: volTableIdxGUID 
> > @IDXATTR: l 
> > @IDXATTR: mSMQDigests 
> > @IDXATTR: msTSExpireDate4 
> > @IDXATTR: flatName 
> > @IDXATTR: msSFU30YpServers 
> > @IDXATTR: packageFlags 
> > @IDXATTR: mSMQOwnerID 
> > @IDXATTR: objectCategory 
> > @IDXATTR: msSFU30IsValidContainer 
> > @IDXATTR: msTSProperty02 
> > @IDXATTR: mS-DS-CreatorSID 
> > @IDXATTR: proxyAddresses 
> > @IDXATTR: msPKI-Cert-Template-OID 
> > @IDXATTR: uNCName 
> > @IDXATTR: mS-SQL-Name 
> > @IDXATTR: fSMORoleOwner 
> > @IDXATTR: msSFU30NisDomain 
> > @IDXATTR: otherMailbox 
> > @IDXATTR: location 
> > @IDXATTR: msSFU30NetgroupHostAtDomain 
> > @IDXATTR: uSNChanged 
> > @IDXATTR: sIDHistory 
> > @IDXATTR: birthLocation 
> > @IDXATTR: msDS-SecondaryKrbTgtNumber 
> > @IDXATTR: msTSProperty01 
> > @IDXATTR: msTSManagingLS4 
> > @IDXATTR: msSFU30OrderNumber 
> > @IDXATTR: msDS-HABSeniorityIndex 
> > @IDXATTR: primaryGroupID 
> > @IDXATTR: mSMQQueueType 
> > @IDXATTR: msDFSR-ReplicationGroupGuid 
> > @IDXATTR: msDS-PhoneticDepartment 
> > @IDXATTR: mail 
> > @IDXATTR: msSFU30Name 
> > @IDXATTR: msSFU30NetgroupUserAtDomain 
> > @IDXATTR: fromServer 
> > @IDXATTR: displayName 
> > @IDXATTR: msTSLicenseVersion2 
> > @IDXATTR: groupType 
> > @IDXATTR: msTSLicenseVersion3 
> > @IDXATTR: msTSLicenseVersion4 
> > @IDXATTR: userAccountControl 
> > @IDXATTR: physicalLocationObject 
> > @IDXATTR: servicePrincipalName 
> > @IDXATTR: msTSExpireDate 
> > @IDXATTR: serviceClassName 
> > @IDXATTR: lDAPDisplayName 
> > @IDXATTR: zarafaAccount 
> > @IDXATTR: terminalServer 
> > @IDXATTR: givenName 
> > @IDXATTR: msTSManagingLS3 
> > @IDXATTR: msSFU30MaxUidNumber 
> > @IDXATTR: msDS-Entry-Time-To-Die 
> > @IDXATTR: msTSLSProperty01 
> > @IDXATTR: msDS-PhoneticFirstName 
> > @IDXATTR: trustPartner 
> > @IDXATTR: msTSLSProperty02 
> > @IDXATTR: msTSExpireDate3 
> > @IDXATTR: objectGUID 
> > @IDXATTR: showInAdvancedViewOnly 
> > @IDXATTR: rpcNsTransferSyntax 
> > @IDXATTR: sAMAccountName 
> > @IDXATTR: mS-SQL-Version 
> > @IDXATTR: msDS-Site-Affinity 
> > @IDXATTR: sn 
> > @IDXATTR: name 
> > @IDXATTR: nETBIOSName 
> > @IDXATTR: sAMAccountType 
> > @IDXATTR: msTSManagingLS 
> > @IDXATTR: msDFSR-DfsPath 
> > @IDXATTR: altSecurityIdentities 
> > @IDXATTR: USNIntersite 
> > @IDXATTR: msSFU30MasterServerName 
> > @IDXATTR: msDS-PhoneticLastName 
> > @IDXATTR: cn 
> > @IDXATTR: netbootGUID 
> > @IDXATTR: lastLogonTimestamp 
> > @IDXATTR: legacyExchangeDN 
> > @IDXATTR: mSMQLabel 
> > @IDXATTR: uSNCreated 
> > @IDXATTR: mS-SQL-Database 
> > @IDXATTR: msDS-PhoneticDisplayName 
> > @IDXATTR: msSFU30MaxGidNumber 
> > @IDXATTR: rpcNsObjectID 
> > @IDXATTR: timeVolChange 
> > @IDXATTR: msTSExpireDate2 
> > @IDXATTR: groupAttributes 
> > @IDXATTR: physicalDeliveryOfficeName 
> > @IDXATTR: msFVE-RecoveryGuid 
> > @IDXATTR: msDS-AdditionalSamAccountName 
> > @IDXATTR: objectSid 
> > @IDXATTR: keywords 
> > @IDXATTR: mS-SQL-Alias 
> > @IDXATTR: invocationId 
> > @IDXATTR: msTSLicenseVersion 
> > @IDXATTR: requiredCategories 
> > @IDXATTR: msDS-AzObjectGuid 
> > distinguishedName: @INDEXLIST 
> > 
> > There is any way to improve LDAP responses times ? It seems there
> > is only 
> > one process which is managing LDAP queries (no forks/threads?) 
> > 
> > Thank you in advance for your help !! 
> > 
> > -- 
> > To unsubscribe from this list go to the following URL and read the 
> > instructions: https://lists.samba.org/mailman/options/samba 
> 
> 
> 
> -- 
> To unsubscribe from this list go to the following URL and read the 
> instructions: https://lists.samba.org/mailman/options/samba 
> 
> 
> 
> 
> 
> -- 
> 
> 
> 
> www.it-optics.com 
> 
> Gaëtan SLONGO | Head of Infrastructure Department 
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> 
> Company : 
> 
> +32 (0)65 84 23 85 
> 
> Direct : 
> 
> +32 (0)65 32 85 88 
> 
> Fax : 
> 
> +32 (0)65 84 66 76 
> 
> Skype ID : 
> 
> gslongo.pro 
> 
> GPG Key : 
> 
> gslongo-gpg_key.asc 
> 
> 
> 
> 
> 
> 
> 
> - Please consider your environmental responsibility before printing
> this e-mail - 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> To unsubscribe from this list go to the following URL and read the 
> instructions: https://lists.samba.org/mailman/options/samba 
> 
> 
> 
> 
> 
> -- 
> 
> 
> 
> www.it-optics.com 
> 
> Gaëtan SLONGO | Head of Infrastructure Department 
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> 
> Company : 
> 
> +32 (0)65 84 23 85 
> 
> Direct : 
> 
> +32 (0)65 32 85 88 
> 
> Fax : 
> 
> +32 (0)65 84 66 76 
> 
> Skype ID : 
> 
> gslongo.pro 
> 
> GPG Key : 
> 
> gslongo-gpg_key.asc 
> 
> 
> 
> 
> 
> 
> 
> - Please consider your environmental responsibility before printing
> this e-mail - 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> To unsubscribe from this list go to the following URL and read the 
> instructions: https://lists.samba.org/mailman/options/samba 
> 
> 
> 
> -- 
> 
> 
> 
> 
> www.it-optics.com 
>         
> Gaëtan SLONGO | Head of Infrastructure Department 
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> Company :         +32 (0)65 84 23 85 
> Direct :         +32 (0)65 32 85 88 
> Fax :         +32 (0)65 84 66 76 
> Skype ID :         gslongo.pro 
> GPG Key :         gslongo-gpg_key.asc 
>         
> 
> - Please consider your environmental responsibility before printing
> this e-mail - 
> 
> 
> 
> 
> 
> 
> 
> -- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba





-- 



www.it-optics.com 
	
Gaëtan SLONGO | Head of Infrastructure Department
Boulevard Initialis, 28 - 7000 Mons, BELGIUM

Company :
	
+32 (0)65 84 23 85

Direct :
	
+32 (0)65 32 85 88

Fax :
	
+32 (0)65 84 66 76

Skype ID :
	
gslongo.pro

GPG Key :
	
gslongo-gpg_key.asc 



	



- Please consider your environmental responsibility before printing this e-mail
-

Andrew Bartlett

2017-Mar-27 17:26 UTC

head link

[Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?

On Mon, 2017-03-27 at 11:11 +0200, Gaetan SLONGO via samba
wrote:> Thank you for your message Andrew. Then .. Waiting for this
> improvement... :-) 
> 
> Indexes seem correct to me. But I'm not sure at 100% because I cannot
> find clear explanation regarding the searchFlags attribute value ..
> 1, 2, ... ? Maybe you have one ? 
https://msdn.microsoft.com/en-us/library/ms679765(v=vs.85).aspx
> Thanks ! 
> 
> ----- Mail original -----
> 
> De: "Andrew Bartlett" <abartlet at samba.org> 
> À: "Gaetan SLONGO" <gslongo at it-optics.com>, "L.P.H.
van Belle" <belle
> @bazuin.nl> 
> Cc: samba at lists.samba.org 
> Envoyé: Lundi 27 Mars 2017 11:06:56 
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ? 
> 
> On Mon, 2017-03-27 at 10:43 +0200, Gaetan SLONGO via samba wrote: 
> > Zarafa is not on the same server as Samba 
> > 
> > We only have 2 AD/DC Samba 4.5 (CentOS 7) and we put required
> > indexes 
> > on LDAP . 
> > 
> > Arround 1000 mailboxes but not all are simultaneously in use
> > (approx 
> > 1/3 in use). 
> > MTA is postfix (and is still connected to Samba AD, this one is
> > not 
> > causing the issue). 
> > As a workarround, we currently deployed a synchronization
> > connector 
> > from AD to OpenLDAP. It solves the performance issue during the 
> > investigation because Zarafa was totally unusable at all when 
> > connected to Samba... But We plan to move to Zimbra by the end of
> > the 
> > year so I think the work arround can stay in place until the 
> > migration. However this performance issue could be a bottleneck in 
> > other applications, in the future... 
> > 
> > I did not found any config setting allowing tu enable multi-
> > threading 
> > on Samba LDAP backend (maybe an hidden one ?).. I think it could
> > help 
> > a lot 
> 
> Given these discussions, I'm keen to add it. I was going to add this 
> for Samba 4.6.0, but the initial approach I used was slower in some 
> cases (the connect/bind/disconnect case). Sadly at the time there 
> wasn't this level of concern regarding the LDAP performance, so we 
> focussed on what we could achieve, which was making NETLOGON multi- 
> process. 
> 
> This remains on my radar, along with any other approaches we find
> along 
> the way to make search-heavy operation practical. 
> 
> I'm sorry this is causing so much trouble, and I look forward to 
> helping improve this area. 
> 
> In the meantime, adding the indexes that your client tools need will 
> help a lot. 
> 
> Andrew Bartlett 
> 
> > ----- Mail original ----- 
> > 
> > De: "L.P.H. van Belle via samba" <samba at
lists.samba.org> 
> > À: samba at lists.samba.org 
> > Envoyé: Lundi 27 Mars 2017 10:26:22 
> > Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> > unusable), performance tunning ? 
> > 
> > Can you tell more about your setup? 
> > 
> > Is zarafa and samba on the same server for example. 
> > 
> > Which MTA are you using postfix/exim? 
> > 
> > 
> > 
> > My top was about 150 users, and all my printers are connected also
> > so 
> > about 200 devices do ldap searches. 
> > 
> > but my setup is split over 10+ servers ( 2 are AD DC ) 
> > 
> > 
> > 
> > So best is to tell what you can about your setup, anonimize if 
> > needed. 
> > 
> > 
> > 
> > Greetz, 
> > 
> > 
> > 
> > Louis 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > Van: Gaetan SLONGO [mailto:gslongo at it-optics.com] 
> > Verzonden: maandag 27 maart 2017 10:12 
> > Aan: L.P.H. van Belle 
> > CC: samba at lists.samba.org 
> > Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> > unusable), performance tunning ? 
> > 
> > 
> > 
> > 
> > What we found is Zarafa makes a very big amount of queries, which 
> > makes Samba run at 100% CPU (one process, LDAP does not seems to
> > be 
> > multi-threaded..?)... but we have hundreds of users... 
> > 
> > What do you think could be wrong in the current database/setup ?
> > We 
> > verified all the setup and everything seems OK 
> > 
> > 
> > De: "L.P.H. van Belle via samba" <samba at
lists.samba.org> 
> > À: samba at lists.samba.org 
> > Envoyé: Lundi 27 Mars 2017 09:58:55 
> > Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> > unusable), performance tunning ? 
> > 
> > No, you have to do that manualy, or look the the samba4 ADS script 
> > for kopano ( or zarafa ) 
> > 
> > But I mostly follow the documentation. 
> > 
> > 
> > 
> > And when i run : 
> > 
> > time ldbsearch -H /var/lib/samba/private/sam.ldb -s base -b 
> > @INDEXLIST 
> > 
> > .... 
> > 
> > real 0m0.230s 
> > 
> > user 0m0.184s 
> > 
> > sys 0m0.044s 
> > 
> > 
> > 
> > so if yours take more that 20 sec there is something very wrong. 
> > 
> > I suggest check you samba AD database and samba4 ADDC setup, 
> > 
> > i dont think this is zarafa related. 
> > 
> > 
> > 
> > 
> > 
> > Greetz, 
> > 
> > 
> > 
> > Louis 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > Van: Gaetan SLONGO [mailto:gslongo at it-optics.com] 
> > Verzonden: maandag 27 maart 2017 8:46 
> > Aan: L.P.H. van Belle 
> > CC: samba at lists.samba.org 
> > Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> > unusable), performance tunning ? 
> > 
> > 
> > 
> > 
> > Hi ! 
> > 
> > Thanks for answer. Yes we use zarafaAccount in search filter. 
> > There is an installer provided for Samba4 to install new schemas ? 
> > 
> > Thanks ! 
> > 
> > 
> > De: "L.P.H. van Belle via samba" <samba at
lists.samba.org> 
> > À: samba at lists.samba.org 
> > Envoyé: Jeudi 23 Mars 2017 11:54:50 
> > Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> > unusable), performance tunning ? 
> > 
> > 
> > Are use using zarafaAccount=1 withing the search filters? 
> > I use this things like this : 
> > 
> >
(&(objectClass=person)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s)
> > )) 
> > 
> > Or for groups. 
> >
(&(objectclass=group)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s))
> > ) 
> > 
> > That helps a lot. 
> > 
> > ! If you switch to kopano beware to change the SCHEMA and filters 
> > zarafaAccount changed to kopanoAccount 
> > 
> > 
> > Greetz. 
> > 
> > Louis 
> > 
> > 
> > > -----Oorspronkelijk bericht----- 
> > > Van: samba [mailto:samba-bounces at lists.samba.org] Namens
Gaetan 
> > > SLONGO via 
> > > samba 
> > > Verzonden: donderdag 23 maart 2017 11:12 
> > > Aan: samba at lists.samba.org 
> > > Onderwerp: [Samba] [Samba 4.5] Very slow LDAP Queries (almost 
> > > unusable), 
> > > performance tunning ? 
> > > Urgentie: Hoog 
> > > 
> > > 
> > > Dear users, 
> > > 
> > > We are facing to a big latency issue regarding the LDAP Server 
> > > (both 
> > > encrypted & plain). 
> > > 
> > > We have a Zarafa mail server which makes a lot of queries and
> > > puts 
> > > a samba 
> > > process to 100% usage. This latency makes the mail server 
> > > unusable.. The 
> > > mail server was previously on OpenLDAP and there was not 
> > > performance 
> > > issues. 
> > > 
> > > A simple LDAP query can take up to 25 sec to perform !! 
> > > 
> > > We have added some indexes : 
> > > 
> > > [root at califix ~]# ldbsearch -H /var/lib/samba/private/sam.ldb
-s 
> > > base -b 
> > > @INDEXLIST 
> > > # record 1 
> > > dn: @INDEXLIST 
> > > @IDXONE: 1 
> > > @IDXVERSION: 2 
> > > @IDXATTR: objectClass 
> > > @IDXATTR: msDS-Cached-Membership-Time-Stamp 
> > > @IDXATTR: userPrincipalName 
> > > @IDXATTR: rpcNsInterfaceID 
> > > @IDXATTR: fileExtPriority 
> > > @IDXATTR: dnsRoot 
> > > @IDXATTR: mSMQLabelEx 
> > > @IDXATTR: dNSTombstoned 
> > > @IDXATTR: msDS-PhoneticCompanyName 
> > > @IDXATTR: msSFU30Domains 
> > > @IDXATTR: dhcpType 
> > > @IDXATTR: ou 
> > > @IDXATTR: gidNumber 
> > > @IDXATTR: msFVE-VolumeGuid 
> > > @IDXATTR: msTSManagingLS2 
> > > @IDXATTR: implementedCategories 
> > > @IDXATTR: oMTIndxGuid 
> > > @IDXATTR: cOMClassID 
> > > @IDXATTR: volTableIdxGUID 
> > > @IDXATTR: l 
> > > @IDXATTR: mSMQDigests 
> > > @IDXATTR: msTSExpireDate4 
> > > @IDXATTR: flatName 
> > > @IDXATTR: msSFU30YpServers 
> > > @IDXATTR: packageFlags 
> > > @IDXATTR: mSMQOwnerID 
> > > @IDXATTR: objectCategory 
> > > @IDXATTR: msSFU30IsValidContainer 
> > > @IDXATTR: msTSProperty02 
> > > @IDXATTR: mS-DS-CreatorSID 
> > > @IDXATTR: proxyAddresses 
> > > @IDXATTR: msPKI-Cert-Template-OID 
> > > @IDXATTR: uNCName 
> > > @IDXATTR: mS-SQL-Name 
> > > @IDXATTR: fSMORoleOwner 
> > > @IDXATTR: msSFU30NisDomain 
> > > @IDXATTR: otherMailbox 
> > > @IDXATTR: location 
> > > @IDXATTR: msSFU30NetgroupHostAtDomain 
> > > @IDXATTR: uSNChanged 
> > > @IDXATTR: sIDHistory 
> > > @IDXATTR: birthLocation 
> > > @IDXATTR: msDS-SecondaryKrbTgtNumber 
> > > @IDXATTR: msTSProperty01 
> > > @IDXATTR: msTSManagingLS4 
> > > @IDXATTR: msSFU30OrderNumber 
> > > @IDXATTR: msDS-HABSeniorityIndex 
> > > @IDXATTR: primaryGroupID 
> > > @IDXATTR: mSMQQueueType 
> > > @IDXATTR: msDFSR-ReplicationGroupGuid 
> > > @IDXATTR: msDS-PhoneticDepartment 
> > > @IDXATTR: mail 
> > > @IDXATTR: msSFU30Name 
> > > @IDXATTR: msSFU30NetgroupUserAtDomain 
> > > @IDXATTR: fromServer 
> > > @IDXATTR: displayName 
> > > @IDXATTR: msTSLicenseVersion2 
> > > @IDXATTR: groupType 
> > > @IDXATTR: msTSLicenseVersion3 
> > > @IDXATTR: msTSLicenseVersion4 
> > > @IDXATTR: userAccountControl 
> > > @IDXATTR: physicalLocationObject 
> > > @IDXATTR: servicePrincipalName 
> > > @IDXATTR: msTSExpireDate 
> > > @IDXATTR: serviceClassName 
> > > @IDXATTR: lDAPDisplayName 
> > > @IDXATTR: zarafaAccount 
> > > @IDXATTR: terminalServer 
> > > @IDXATTR: givenName 
> > > @IDXATTR: msTSManagingLS3 
> > > @IDXATTR: msSFU30MaxUidNumber 
> > > @IDXATTR: msDS-Entry-Time-To-Die 
> > > @IDXATTR: msTSLSProperty01 
> > > @IDXATTR: msDS-PhoneticFirstName 
> > > @IDXATTR: trustPartner 
> > > @IDXATTR: msTSLSProperty02 
> > > @IDXATTR: msTSExpireDate3 
> > > @IDXATTR: objectGUID 
> > > @IDXATTR: showInAdvancedViewOnly 
> > > @IDXATTR: rpcNsTransferSyntax 
> > > @IDXATTR: sAMAccountName 
> > > @IDXATTR: mS-SQL-Version 
> > > @IDXATTR: msDS-Site-Affinity 
> > > @IDXATTR: sn 
> > > @IDXATTR: name 
> > > @IDXATTR: nETBIOSName 
> > > @IDXATTR: sAMAccountType 
> > > @IDXATTR: msTSManagingLS 
> > > @IDXATTR: msDFSR-DfsPath 
> > > @IDXATTR: altSecurityIdentities 
> > > @IDXATTR: USNIntersite 
> > > @IDXATTR: msSFU30MasterServerName 
> > > @IDXATTR: msDS-PhoneticLastName 
> > > @IDXATTR: cn 
> > > @IDXATTR: netbootGUID 
> > > @IDXATTR: lastLogonTimestamp 
> > > @IDXATTR: legacyExchangeDN 
> > > @IDXATTR: mSMQLabel 
> > > @IDXATTR: uSNCreated 
> > > @IDXATTR: mS-SQL-Database 
> > > @IDXATTR: msDS-PhoneticDisplayName 
> > > @IDXATTR: msSFU30MaxGidNumber 
> > > @IDXATTR: rpcNsObjectID 
> > > @IDXATTR: timeVolChange 
> > > @IDXATTR: msTSExpireDate2 
> > > @IDXATTR: groupAttributes 
> > > @IDXATTR: physicalDeliveryOfficeName 
> > > @IDXATTR: msFVE-RecoveryGuid 
> > > @IDXATTR: msDS-AdditionalSamAccountName 
> > > @IDXATTR: objectSid 
> > > @IDXATTR: keywords 
> > > @IDXATTR: mS-SQL-Alias 
> > > @IDXATTR: invocationId 
> > > @IDXATTR: msTSLicenseVersion 
> > > @IDXATTR: requiredCategories 
> > > @IDXATTR: msDS-AzObjectGuid 
> > > distinguishedName: @INDEXLIST 
> > > 
> > > There is any way to improve LDAP responses times ? It seems
> > > there 
> > > is only 
> > > one process which is managing LDAP queries (no forks/threads?) 
> > > 
> > > Thank you in advance for your help !! 
> > > 
> > > -- 
> > > To unsubscribe from this list go to the following URL and read
> > > the 
> > > instructions: https://lists.samba.org/mailman/options/samba 
> > 
> > 
> > 
> > -- 
> > To unsubscribe from this list go to the following URL and read the 
> > instructions: https://lists.samba.org/mailman/options/samba 
> > 
> > 
> > 
> > 
> > 
> > -- 
> > 
> > 
> > 
> > www.it-optics.com 
> > 
> > Gaëtan SLONGO | Head of Infrastructure Department 
> > Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> > 
> > Company : 
> > 
> > +32 (0)65 84 23 85 
> > 
> > Direct : 
> > 
> > +32 (0)65 32 85 88 
> > 
> > Fax : 
> > 
> > +32 (0)65 84 66 76 
> > 
> > Skype ID : 
> > 
> > gslongo.pro 
> > 
> > GPG Key : 
> > 
> > gslongo-gpg_key.asc 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > - Please consider your environmental responsibility before
> > printing 
> > this e-mail - 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > -- 
> > To unsubscribe from this list go to the following URL and read the 
> > instructions: https://lists.samba.org/mailman/options/samba 
> > 
> > 
> > 
> > 
> > 
> > -- 
> > 
> > 
> > 
> > www.it-optics.com 
> > 
> > Gaëtan SLONGO | Head of Infrastructure Department 
> > Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> > 
> > Company : 
> > 
> > +32 (0)65 84 23 85 
> > 
> > Direct : 
> > 
> > +32 (0)65 32 85 88 
> > 
> > Fax : 
> > 
> > +32 (0)65 84 66 76 
> > 
> > Skype ID : 
> > 
> > gslongo.pro 
> > 
> > GPG Key : 
> > 
> > gslongo-gpg_key.asc 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > - Please consider your environmental responsibility before
> > printing 
> > this e-mail - 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > -- 
> > To unsubscribe from this list go to the following URL and read the 
> > instructions: https://lists.samba.org/mailman/options/samba 
> > 
> > 
> > 
> > -- 
> > 
> > 
> > 
> > 
> > www.it-optics.com 
> > 
> > Gaëtan SLONGO | Head of Infrastructure Department 
> > Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> > Company : +32 (0)65 84 23 85 
> > Direct : +32 (0)65 32 85 88 
> > Fax : +32 (0)65 84 66 76 
> > Skype ID : gslongo.pro 
> > GPG Key : gslongo-gpg_key.asc 
> > 
> > 
> > - Please consider your environmental responsibility before
> > printing 
> > this e-mail - 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> 
> -- 
> Andrew Bartlett http://samba.org/~abartlet/ 
> Authentication Developer, Samba Team http://samba.org 
> Samba Developer, Catalyst IT http://catalyst.net.nz/services/samba 
> 
> 
> 
> 
> -- 
> 
> 
> 
> 
> www.it-optics.com 
> 	
> Gaëtan SLONGO | Head of Infrastructure Department 
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM 
> Company : 	+32 (0)65 84 23 85 
> Direct : 	+32 (0)65 32 85 88 
> Fax : 	+32 (0)65 84 66 76 
> Skype ID : 	gslongo.pro 
> GPG Key : 	gslongo-gpg_key.asc 
> 	
> 
> - Please consider your environmental responsibility before printing
> this e-mail - 
> 
> 
> 
> 
> 
> 
> 
> -- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba

Possibly Parallel Threads

Search for more apparently analagous threads

samba - Mar 2017 - [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?

[Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?

[Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?

[Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?

[Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?

Possibly Parallel Threads