samba - Dec 2017 - samba 4.7.3 DLZ performance regression

On Fri, 2017-12-15 at 10:16 +1300, Garming Sam via samba
wrote:
> Hi,
> 
> We've just had another person notice this issue and we're currently
> looking into it. If we need someone to test the patch fix, or if we need
> more information, we can let you know.
> 
> I've filed a bug as well:
> 
> https://bugzilla.samba.org/show_bug.cgi?id=13191
Attached is a possible set of patches. 

As indicated, a search for a not-existing name, be it one that will
match with a wildcard or one that wont, will still be slow.

Andrew Bartlett

-- 
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team         https://samba.org
Samba Development and Support, Catalyst IT   
https://catalyst.net.nz/services/samba



-------------- next part --------------

Thanks Andrew, much better than vanilla 4.7.3 but it is still performs 
much worse than vanilla 4.7.0 especially for DLZ zones.

$ dig @127.0.0.1 google.com
$ perl -e "foreach(1..500) { system('dig', '@127.0.0.1',
'google.com'); }" | grep time > 4.7.0_external.log
$ perl -e "foreach(1..500) { system('dig', '@127.0.0.1',
'mediture.dom'); }" | grep time > 4.7.0_internal.log

$ cat 4.7.0_internal.log.gz | perl dns_stats.pl
Max process time (ms): 7
Standard deviation of process time (ms): 0.456807778713746
Mean process time (ms): 2.088

$ cat 4.7.0_external.log.gz | perl dns_stats.pl
Max process time (ms): 3
Standard deviation of process time (ms): 0.332814559570528
Mean process time (ms): 1.884

Then after installing Samba4, restarting samba4 and restarting BIND:

$ dig @127.0.0.1 google.com
$ perl -e "foreach(1..500) { system('dig', '@127.0.0.1',
'google.com'); }" | grep time > 4.7.3_external.log
$ perl -e "foreach(1..500) { system('dig', '@127.0.0.1',
'mediture.dom'); }" | grep time > 4.7.3_internal.log

$ cat 4.7.3_external.log.gz | perl dns_stats.pl
Max process time (ms): 985
Standard deviation of process time (ms): 169.45778909383
Mean process time (ms): 66.36

$ cat 4.7.3_internal.log.gz | perl dns_stats.pl
Max process time (ms): 975
Standard deviation of process time (ms): 238.891289268035
Mean process time (ms): 404.935871743487

On 12/14/2017 05:44 PM, Andrew Bartlett wrote:
>   Attached is a possible set of patches.
>
> As indicated, a search for a not-existing name, be it one that will
> match with a wildcard or one that wont, will still be slow.
>
> Andrew Bartlett
>

This e-mail and any attachments may contain CONFIDENTIAL information, including
PROTECTED HEALTH INFORMATION. If you are not the intended recipient, any use or
disclosure of this information is STRICTLY PROHIBITED; you are requested to
delete this e-mail and any attachments, notify the sender immediately, and
notify the Mediture Privacy Officer at privacyofficer at mediture.com.

On Thu, 2017-12-14 at 18:52 -0600, Arthur Ramsey wrote:
> Thanks Andrew, much better than vanilla 4.7.3 but it is still performs 
> much worse than vanilla 4.7.0 especially for DLZ zones.
I understand.  The key issue is that to support the wildcards we
changed the way we use the database, the SCOPE_ONELEVEL index (compared
with SCOPE_BASE used exclusively in the past) over 7000 possible values
is the choke point on the database I'm testing with.

This is much improved in master, but sadly the primary fix turns out to
be the GUID index change (which is too much to backport), rather than
(as I had hoped) the supporting changes.

Are the stats you included on the patched 4.7.3, or the original
release?

Thanks,

Andrew Bartlett
-- 
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team         https://samba.org
Samba Development and Support, Catalyst IT   
https://catalyst.net.nz/services/samba