Recently i've migrated our SVN server (virtual machine) from C6 to C7
(more precisely - migrated data to freshly installed virtual machine).
And we have problem with very slow authentication. Server is configured
with SSSD, user data are fetching from our LDAP server. SVN is
configured with apache (pwauth for authentication + LDAP search for
Require ldap-group).
It takes pwauth even 10 seconds to authenticate. Whet it comes to svn's
externals it could take as long as 9 minutes to _svn up_ project (when
there are no commits to fetch). Every external may take even 15 seconds
(and sometimes even more).
SSSD was configured at first with authconfig / authconfig-tui.
I was struggling with SSSD configuration but with no success. I'm not
sure where to look (SSSD, apache?). How can i debug this issue?
sssd.conf:
[domain/default]
autofs_provider = ldap
cache_credentials = True
krb5_realm = #
ldap_search_base = ou=Main,o=company
id_provider = ldap
auth_provider = ldap
chpass_provider = ldap
ldap_uri = ldap://ldap.our.domain/
ldap_group_search_base = ou=Group,ou=Main,o=company
ldap_user_search_base = ou=People,ou=Main,o=company
ldap_id_use_start_tls = False
ldap_tls_cacertdir = /etc/openldap/cacerts
ldap_tls_reqcert = allow
#debug_level = 4
refresh_expired_interval = 120
enumerate = True
ldap_referrals = False
[sssd]
services = nss, pam, autofs
config_file_version = 2
domains = default
[nss]
homedir_substring = /home
entry_cache_timeout = 5400
[pam]
pam_id_timeout=20
apache:
LDAPCacheTTL 30
<VirtualHost 10.0.32.19:80>
ErrorLog logs/svn_http_error_log
CustomLog logs/svn_http_access_log "%t %u %{SVN-ACTION}e"
env=SVN-ACTION
ServerName svn.our.domain
DirectoryIndex none
DefineExternalAuth pwauth pipe /usr/bin/pwauth
#AddExternalGroup unixgroup /usr/sbin/unixgroup
#SetExternalGroupMethod unixgroup environment
<Location />
SVNPathAuthz off
DAV svn
SVNPath /home/repos/subversion_free_avr
AuthBasicAuthoritative off
AuthBasicProvider socache external
AuthExternal pwauth
AuthnCacheProvideFor external
AuthType Basic
AuthName "Subversion repository"
AuthLDAPURL ldap://ldap.our.domain/ou=Main,o=company
AuthLDAPGroupAttribute memberUid
AuthLDAPGroupAttributeIsDN off
Require ldap-group cn=programmers,ou=group,ou=main,o=company
#GroupExternal unixgroup
#Require group programmers
#Require valid-user
#AuthzSVNAccessFile /home/repos/svn.access
</Location>
</VirtualHost>
On same server we have redmine (with database on separate server and
LDAP auth) and git repositories (with gitbucket as frontend, also LDAP
auth) but those repos aren't extensively used right now. Redmine works
not-so-bad, so i guess it is not overall server performance issue.
Disks performance (measured under normal workload):
[root at luah pam.d]# hdparm -tT /dev/vda
/dev/vda: (system)
Timing cached reads: 11412 MB in 2.00 seconds = 5710.28 MB/sec
Timing buffered disk reads: 522 MB in 3.63 seconds = 143.79 MB/sec
[root at luah pam.d]# hdparm -tT /dev/vdd
/dev/vdd: (/home where all data resides)
Timing cached reads: 10020 MB in 2.00 seconds = 5013.17 MB/sec
Timing buffered disk reads: 172 MB in 3.20 seconds = 53.73 MB/sec
It's comparable with other our VMs.
Any ideas?
--
Over And Out
MoonWolf
Am 11.04.2016 um 15:44 schrieb Marcin Trendota <moonwolf.rh at gmail.com>:> > Any ideas?DNS? -- LF
On Mon, Apr 11, 2016 at 05:22:43PM +0200, Leon Fauster wrote:> Am 11.04.2016 um 15:44 schrieb Marcin Trendota <moonwolf.rh at gmail.com>: > > > > Any ideas? > > > DNS?Is LDAP listed in the /etc/nsswitch.conf? -- Scott Robbins PGP keyID EB3467D6 ( 1B48 077D 66F6 9DB0 FDC2 A409 FA54 EB34 67D6 ) gpg --keyserver pgp.mit.edu --recv-keys EB3467D6
On 12 Apr 2016 16:29, "Scott Robbins" <scottro11 at gmail.com> wrote:> > On Tue, Apr 12, 2016 at 09:45:17AM +0200, Marcin Trendota wrote: > > W dniu 11.04.2016 o 20:07, Scott Robbins pisze: > > > > >>> Any ideas? > > >> DNS? > > > Is LDAP listed in the /etc/nsswitch.conf? > > > > In nsswitch.conf i have: > > passwd: files sss > > shadow: files sss > > group: files sss > > > > DNS works fine. I think that sssd communicates with LDAP server with > > every authentication - i have tons of following entries in log: > > > > http:// <http://pastebin.com/rZVjk0gW>pastebin.com<http://pastebin.com/rZVjk0gW>/ <http://pastebin.com/rZVjk0gW>rZVjk0gW <http://pastebin.com/rZVjk0gW>> > > > And it repeats for same user over and over again. Is this correctbehavior?> RedHat never really mastered LDAP, unfortunately. I have a by now ancient > article, that mentions it. ><snip> What utter nonsense. Just because you poorly configured your system does not mean that Red Hat never really mastered it... And translating very old experiences to CentOS 7 is even more ridiculous and counter productive. To the OP enumerate is always painful, I'd remove that for a start. My experience with the DAV SVN though is that clients are horrible in their requests... So many it hits it so hard... After various testing I ended up going with the Apache LDAP cache module and doing the auth at the Apache level, not system. Was far better in performance with the SVN server being hit fairly hard. I can try and dig out an example configuration if you would like. The bonus here as well is that svn users are separated cleanly from system users... No reason for a dev to have a shell account on there ;)
On 04/12/2016 09:51 AM, James Hogarth wrote:> To the OP enumerate is always painful, I'd remove that for a start.This was my experience too, for what it's worth. When I first set up a new system pointed at LDAP it was absurdly slow to authenticate. Setting Enumerate to False in /etc/sssd/sssd.conf made all the difference.
James Hogarth wrote:> On 12 Apr 2016 16:29, "Scott Robbins" <scottro11 at gmail.com> wrote: >> On Tue, Apr 12, 2016 at 09:45:17AM +0200, Marcin Trendota wrote: >> > W dniu 11.04.2016 o 20:07, Scott Robbins pisze:<SNIP>> After various testing I ended up going with the Apache LDAP cache module > and doing the auth at the Apache level, not system. > > Was far better in performance with the SVN server being hit > fairly hard. I can try and dig out an example configuration if > you would like. > > The bonus here as well is that svn users are separated cleanly > from system users... No reason for a dev to have a shell account > on there ;)I'd be *very* interested in that configuration, if you post it here, or offlist, to me. mark