I upgraded most of our servers from 1.2.x to 2.0.8 and am noticing a really big increase in server load. This is across the board, not any specific server. Check this screenshot of a load graph: http://grab.by/7N8V Is there anything i should be looking at that could cause such a drastic load increase? Conf file is available here: http://wa.ter.net/download/doveconf.txt Regards, Cor
* Cor Bosman <cor at xs4all.nl>:> I upgraded most of our servers from 1.2.x to 2.0.8 and am noticing a > really big increase in server load. This is across the board, not any > specific server. Check this screenshot of a load graph: > http://grab.by/7N8VYes, this looks like my graphs. Same increase. Factor 5.> Is there anything i should be looking at that could cause such a > drastic load increase? Conf file is available here: > http://wa.ter.net/download/doveconf.txtWe need to find out WHAT is common in our two config files! Could you post "doveconf -n" output (I think it shows only non-default values), the current output is a bit longish. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de
* Ralf Hildebrandt <Ralf.Hildebrandt at charite.de>:> > http://wa.ter.net/download/doveconf.txt > > We need to find out WHAT is common in our two config files!My config is attached -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de -------------- next part -------------- # 2.0.5: /usr/local/etc/dovecot.conf # OS: Linux 2.6.32-24-generic-pae i686 Debian squeeze/sid auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login disable_plaintext_auth = no auth_master_user_separator = * mail_location = maildir:~/Maildir # wegen webmail! mail_max_userip_connections = 512 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date # Namespace f?r courier-compatibilitaet namespace { inbox = yes location = prefix = INBOX. separator = . type = private } # in kilobytes auth_cache_size = 2048 # Authorisierung via PAM, /etc/pam.d/dovecot passdb { driver = pam args = cache_key=%u } # Altlast, einfach mal so uebernommen # RHI 11.10.2010 # neu entfernt wegen "pass = yes" das eh nicht geht! #passdb { # driver = shadow #} # fuer user*masteruser logins passdb { args = /usr/dovecot-2/etc/dovecot/dovecot.masteruser driver = passwd-file master = yes # pass = yes # das funktioniert laut doku gar nicht mit PAM! } # User via passwd userdb { driver = passwd args = cache_key=%u # neu } # plugin Konfiguration plugin { # mailboxen anlegen und subscriben autocreate = Trash autocreate2 = spam autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = spam autosubscribe3 = Sent autosubscribe4 = Drafts # FullTextSearch fts = squat # Quota quota = maildir quota_rule = INBOX.Trash:storage=+2048M quota_warning = storage=99%% quota-warning 99 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=85%% quota-warning 85 %u # Sieve sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_max_redirects = 10 sieve_max_script_size = 2M # Trash (wenn Mailbox voll, dann Trash und spam leeren) trash = /usr/dovecot-2/etc/dovecot/dovecot-trash.conf } # Wir sprechen alles protocols = imap sieve pop3 lmtp # unix domain socket fuer Postfix service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } # unix domain socket fuer LMTP service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } # imap-login Prozess # high performance mode service imap-login { service_count = 0 } # der eigentliche IMAPD service imap { process_limit = 2048 executable = imap -L } # managesieve-login, wird nur genutzt von webmail aus service managesieve-login { service_count = 0 } # der eigentliche managesieve service managesieve { process_limit = 128 } # pop3-login Prozess # high performance mode service pop3-login { service_count = 0 # kein pop3, nur pop3s! inet_listener pop3 { port = 0 } } # der eigentliche pop3 service pop3 { process_limit = 512 } # die ganzen SSL Zertifikate ssl_ca = </etc/ssl/certs/ca-certificates.crt ssl_cert = </etc/ssl/certs/cert-188235905-postamt.charite.de-g02.pem ssl_key = </etc/ssl/private/postamt.key # schoene Ausgabe in ps auxwww verbose_proctitle = yes version_ignore = yes mail_fsync = never mail_plugins = quota protocol imap { mail_plugins = quota imap_quota trash fts fts_squat # autocreate } # pop3 hat workarounds protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_lock_session = yes pop3_uidl_format = %v-%u } protocol lda { mail_plugins = quota sieve trash postmaster_address = postmaster at charite.de quota_full_tempfail = yes syslog_facility = local4 } protocol lmtp { mail_plugins = quota sieve trash postmaster_address = postmaster at charite.de quota_full_tempfail = yes syslog_facility = local4 } # der schickt die Quota warnmails service quota-warning { executable = script /usr/local/scripts/quota-warning2 user = root unix_listener quota-warning { mode = 0666 user = vmail group = users } } # Da kommt die Config her fuer subprozesse service config { unix_listener config { mode = 0666 } }
Ralf, did you do the profiling yet? On 12/08/10 09:50, Ralf Hildebrandt wrote:> [...] > Yes, this looks like my graphs. Same increase. Factor 5.
* David Ford <david at blue-labs.org>:> Ralf, did you do the profiling yet?With gprof or what exactly is on your mind?> On 12/08/10 09:50, Ralf Hildebrandt wrote: > >[...] > >Yes, this looks like my graphs. Same increase. Factor 5.-- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de
On Qua, 2010-12-08 at 15:39 +0100, Cor Bosman wrote:> I upgraded most of our servers from 1.2.x to 2.0.8 and am noticing a really big increase in server load. This is across the board, not any specific server. Check this screenshot of a load graph: http://grab.by/7N8V > > Is there anything i should be looking at that could cause such a drastic load increase? Conf file is available here: http://wa.ter.net/download/doveconf.txt >We also had a load increase (about 10x) when going from 1.1.10 to 2.0.7 a couple of weeks ago. It ended up being due to a slight increase in memory usage by the imap processes that made the servers start using swap and the load to spike. A memory upgrade brought it all to normal loads. Wonder if this is your case (vmstat, check for si and so). -- Jose Celestino | http://japc.uncovering.org/files/japc-pgpkey.asc ---------------------------------------------------------------- "Assumption is the Mother of Screw-Up" -- Mr. John Elwood Hale
On 8.12.2010, at 22.19, Cor Bosman wrote:> Oh, and I dont know if we did in 1.2. I think so, but cant be positive. I tried making the config the same. I have the config still around if you want to see it.In v1.2 it was called login_process_per_connection
login_process_per_connection = yes So seems like we did have that set in 1.2. Cor
On 8.12.2010, at 22.52, Cor Bosman wrote:> 1 server with service_count = 0, and src/imap/main.c patchBy this you mean service_count=0 for both service imap-login and service imap blocks, right?
> >> 1 server with service_count = 0, and src/imap/main.c patch > > By this you mean service_count=0 for both service imap-login and service imap blocks, right? > >Yes, on both imap-login and imap, The 2 servers without the patch only have it on imap-login. Cor
trace looks like an interesting new tool: http://lwn.net/Articles/415728/ Wonder if that would tell something about this problem.
On 9.12.2010, at 1.52, Timo Sirainen wrote:> trace looks like an interesting new tool: http://lwn.net/Articles/415728/ > > Wonder if that would tell something about this problem.Like the answer to question: Is the number of voluntary context switches by imap processes close to the number of called syscalls? And if so, which system calls does v2.0 call more often than v1.2?
> Oops, now I finally understand why Mail.app kept asking for my password for each mail I sent: it helpfully decided to start signing mails with the only client cert I had, without asking me.. Forget about those signatures in the last two mails :) >Heh, is that the key you used to get into our test server for this problem? :) Cor