dovecot - Feb 2011 - Dovecot 2.0.9/1.2.11: crash in driver-pgsql.c

Hello,

I finally took time to isolate and reproduce a long time Dovecot crash
that is bugging me since at least Dovecot 1.0.x. It's still there in
Dovecot 2.0.9, which I just tested.

Here is the backtrace from my logs:

Feb 16 16:56:57 dovecot: auth: Error: pgsql: Query timed out, aborting
Feb 16 17:25:16 dovecot: auth: Panic: file driver-pgsql.c: line 349
(result_finish): assertion failed: (db->io == NULL)
Feb 16 17:25:16 dovecot: auth: Error: Raw backtrace:
/usr/lib/dovecot/libdovecot.so.0 [0x7fdbfe6d4272] ->
/usr/lib/dovecot/libdovecot.so.0 [0x7fdbfe6d42d6] ->
/usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fdbfe6d46c8] ->
dovecot/auth [0x42f555] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xdd)
[0x7fdbfe6df9bd] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x5e) [0x7fdbfe6e05be]
-> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7fdbfe6df618] ->
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fdbfe6cd883]
-> dovecot/auth(main+0x2d4) [0x419474] ->
/lib/libc.so.6(__libc_start_main+0xe6) [0x7fdbfc4f91a6] -> dovecot/auth
[0x40da09]
Feb 16 17:26:35 dovecot: master: Error: service(auth): child 140263 killed
with signal 6 (core not dumped)

Both my userdb and passdb use a PostgreSQL database. The bug happens when
the PostgreSQL host suddenly becomes unavailable. I can easily reproduce it
by launching Dovecot with my host being up, and then null routing its IP
address (to simulate an outage):

route add -host 10.0.0.2 reject

(10.0.0.2 being the IP address of the PostgreSQL host.)
>From that point, if I try to authenticate (using a POP3 connection, for
instance), dovecot-auth will crash one or two minutes later.

In Dovecot 2.0.9, despite the crash, authentication will resume once the
PostgreSQL is back up (which can be simulated with: route delete -host
10.0.0.2 reject). In Dovecot 1.x however (latest version I tested is
1.2.11), it will stay in this state forever (logs are flooded with Password
query failed: Not connected to database.)

Core dump is available on http://paste.excellency.fr/dovecot.core

# dovecot -n

# 2.0.9: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.37 x86_64 Debian 5.0.6 
auth_mechanisms = plain login
auth_username_chars = 
disable_plaintext_auth = no
first_valid_gid = 2000
first_valid_uid = 2000
listen = *, [::]
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_fsync = always
mail_location = maildir:~/
mail_max_userip_connections = 100
mail_nfs_index = yes
mail_nfs_storage = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date
mmap_disable = yes
passdb {
  args = /etc/dovecot/dovecot-pgsql.conf
  driver = sql
}
plugin {
  quota = maildir
  sieve = filter.sieve
  sieve_max_actions = 64
  sieve_max_redirects = 100
}
protocols = imap pop3
service auth {
  unix_listener auth-client {
    mode = 0666
  }
}
service imap-login {
  service_count = 0
}
service imap {
  process_limit = 1024
}
service managesieve-login {
  service_count = 0
}
service pop3-login {
  service_count = 0
}
service pop3 {
  process_limit = 1024
}
ssl_cert = </etc/ssl/certs/my.pem
ssl_key = </etc/ssl/private/my.key
userdb {
  args = /etc/dovecot/dovecot-pgsql.conf
  driver = sql
}
protocol imap {
  imap_client_workarounds = delay-newmail
  mail_plugins = quota imap_quota
}
protocol pop3 {
  mail_plugins = quota
}

# cat /etc/dovecot/dovecot-pgsql.conf

driver = pgsql
connect = host=10.0.0.2 dbname=foobar user=dovecot password=foobar

default_pass_scheme = CRYPT

password_query = SELECT password FROM mailbox WHERE domain = '%d' AND
local_part = '%n'
user_query = SELECT home, uid, gid, system_user, quota_rule FROM mailbox
WHERE domain = '%d' AND local_part = '%n'

-- 
Cyril

On Wed, 2011-02-16 at 18:14 +0100, dovecot at excellency.fr wrote:
> Feb 16 16:56:57 dovecot: auth: Error: pgsql: Query timed out, aborting
> Feb 16 17:25:16 dovecot: auth: Panic: file driver-pgsql.c: line 349
> (result_finish): assertion failed: (db->io == NULL) 
Could you try if http://hg.dovecot.org/dovecot-2.0/rev/7d265040cd02
fixes this properly?