thr3ads.net - dovecot - auth-worker service failed since recent MariaDB upgrade [Jan 2019]

If this information is useful, please help other people find it:
Share via:

Bogomil Vasilev

2019-Jan-20 13:05 UTC

auth-worker service failed since recent MariaDB upgrade

Hello,

I rebuilt mariadb and dovecot with debug symbols.
The backtrace is this:

Program terminated with signal SIGABRT, Aborted.
#0  0x00007fb040e0cd7f in raise () from /usr/lib/libc.so.6
(gdb) bt full
#0  0x00007fb040e0cd7f in raise () from /usr/lib/libc.so.6
No symbol table info available.
#1  0x00007fb040df7672 in abort () from /usr/lib/libc.so.6
No symbol table info available.
#2  0x00007fb040e4f878 in __libc_message () from /usr/lib/libc.so.6
No symbol table info available.
#3  0x00007fb040e5618a in malloc_printerr () from /usr/lib/libc.so.6
No symbol table info available.
#4  0x00007fb040e57b27 in _int_free () from /usr/lib/libc.so.6
No symbol table info available.
#5  0x00007fb04114db89 in mysql_close (mysql=0x5592728448e0) at 
/usr/src/debug/mariadb-10.3.12/libmariadb/libmariadb/mariadb_lib.c:1940
         p = <optimized out>
#6  mysql_close (mysql=0x5592728448e0) at 
/usr/src/debug/mariadb-10.3.12/libmariadb/libmariadb/mariadb_lib.c:1913
         p = <optimized out>
#7  0x00005592724e2dac in driver_sqlpool_disconnect (_db=0x559272843be0) 
at driver-sqlpool.c:590
         conn__foreach_end = 0x559272844090
         db = 0x559272843be0
         conn = 0x559272844080
#8  0x00005592724d78fc in db_sql_unref (_conn=0x55927283b5e8) at 
db-sql.c:128
         conn = 0x559272843818
#9  0x00005592724cf89c in userdb_deinit (userdb=0x55927283b5c0) at 
userdb.c:191
         idx = 0
         __func__ = "userdb_deinit"
#10 0x00005592724afbb9 in auth_deinit (auth=0x55927283b320) at 
auth.c:335
         passdb = <optimized out>
         userdb = 0x55927283b4e0
         passdb = <optimized out>
         userdb = <optimized out>
#11 auths_deinit () at auth.c:433
         auth__foreach_end = 0x559272842e18
         auth = 0x559272842e10
#12 0x00005592724ae664 in main_deinit () at main.c:271
         l = <optimized out>
         l = <optimized out>
         l_end = <optimized out>
#13 main (argc=<optimized out>, argv=<optimized out>) at main.c:398
         c = <optimized out>
(gdb)


My `dovecot -n` output is this:

  0 ? root ? /home/smirky/stuff/ABS/dovecot # dovecot -n
# 2.3.4 (0ecbaf23d): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.4 (60b0f48d)
# OS: Linux 5.0.0-rc2-mainline x86_64 Arch Linux
# Hostname: archy
dict {
   quotadict = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
}
mail_home = /home/vmail/%d/%u
mail_location = maildir:~
mail_plugins = quota
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date i
ndex ihave duplicate mime foreverypart extracttext imapsieve 
vnd.dovecot.imapsieve
namespace inbox {
   inbox = yes
   location    mailbox Drafts {
     auto = create
     special_use = \Drafts
   }
   mailbox Sent {
     auto = subscribe
     special_use = \Sent
   }
   mailbox "Sent Messages" {
     auto = no
     special_use = \Sent
   }
   mailbox Spam {
     auto = subscribe
     special_use = \Junk
   }
   mailbox Trash {
     auto = subscribe
     special_use = \Trash
   }
   prefix }
passdb {
   args = /etc/dovecot/dovecot-sql.conf
   driver = sql
}
plugin {
   imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve
   imapsieve_mailbox1_causes = COPY
   imapsieve_mailbox1_name = Spam
   imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve
   imapsieve_mailbox2_causes = COPY
   imapsieve_mailbox2_from = Junk
   imapsieve_mailbox2_name = *
   quota = dict:User quota::proxy::quotadict
   quota_rule2 = Trash:storage=+10%%
   quota_warning = storage=95%% quota-warning 95 %u
   quota_warning2 = storage=80%% quota-warning 80 %u
   quota_warning3 = -storage=100%% quota-warning below 100 %u
   sieve_before = /etc/dovecot/sieve/filter.sieve
   sieve_global_extensions = +vnd.dovecot.pipe
   sieve_pipe_bin_dir = /etc/dovecot/sieve
   sieve_plugins = sieve_imapsieve sieve_extprograms
}
protocols = imap pop3 sieve
service auth-worker {
   unix_listener auth-worker {
     group = vmail
     mode = 0666
     user = vmail
   }
}
service auth {
   unix_listener /var/spool/postfix/private/auth {
     group = postfix
     mode = 0660
     user = postfix
   }
   user = root
}
service dict {
   unix_listener dict {
     group = vmail
     mode = 0660
     user = vmail
   }
   user = root
}
service imap-login {
   inet_listener imap {
     address = 127.0.0.1
   }
}
service managesieve-login {
   inet_listener sieve {
     port = 4190
   }
}
service pop3-login {
   inet_listener pop3 {
     address = 127.0.0.1
   }
}
service quota-warning {
   executable = script /usr/local/bin/quota-warning.sh
   unix_listener quota-warning {
     group = vmail
     mode = 0660
     user = vmail
   }
   user = vmail
}
service stats {
   unix_listener stats-reader {
     group = vmail
     mode = 0660
     user = vmail
   }
   unix_listener stats-writer {
     group = vmail
     mode = 0660
     user = vmail
   }
}
ssl_cert = </etc/letsencrypt/live/smirky.net/fullchain.pem
ssl_dh = # hidden, use -P to show it
ssl_key = # hidden, use -P to show it
userdb {
   args = /etc/dovecot/dovecot-sql.conf
   driver = sql
}
protocol pop3 {
   mail_plugins = quota
   pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
   pop3_uidl_format = %08Xu%08Xv
}
protocol lda {
   mail_plugins = quota
   postmaster_address = postmaster at smirky.net
}
protocol imap {
   mail_max_userip_connections = 20
   mail_plugin_dir = /usr/lib/dovecot/modules
   mail_plugins = quota imap_quota imap_sieve
}


---
BOGOMIL VASILEV
For contacts, use this e-mail. My GPG key is 646F3ABF6E457336
Website: HTTPS://WWW.SMIRKY.NET/

On 2019-01-20 13:22, Stephan Bosch wrote:
> Op 20/01/2019 om 10:50 schreef Bogomil Vasilev via dovecot:
> 
>> Hello,
>> 
>> I have started having issues with auth-worker. Dovecot version:
>> 
>> 0 ? root ? ~ # dovecot --version
>> 2.3.4 (0ecbaf23d)
>> 0 ? root ? ~ # pacman -Q dovecot
>> dovecot 2.3.4-2
>> 
>> OS: Arch Linux
>> 
>> From /var/log/mail.log:
>> 
>> Jan 20 11:31:29 archy dovecot[23070]: auth-worker: Fatal: master: 
>> service(auth-worker): child 9885 killed with signal 6 (core not dumped 
>> - https://dovecot.org/bugreport.html#coredumps - set 
>> /proc/sys/fs/suid_dum
>> pable to 2)
>> 
>> From dmesg:
>> 
>> [42113.729398] auth[8134]: segfault at 5579d8e1c8e0 ip 
>> 00005579d8e1c8e0 sp 00007ffd44ab0df8 error 15
>> [42113.729406] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 
>> 00 00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 7a af 07 e3 
>> 7f 00 00 <40> f4 b5 07 e3 7f 00 00 31 a0 00 00 00 00 00 00 e0 9f
df d8
>> 79
>> 55
>> 
>> This started happening on January 18th, after I upgraded Dovecot from 
>> "2.3.4-1" to "2.3.4-2".
>> The package was rebuilt from our package maintainer due to bumping 
>> MariaDB from 10.1.x to 10.3.x.
>> Before that, I did not have any issues at all.
>> However, I don't see any impact from this error too, at least not
for
>> now.
>> I made a coredump as the error suggests and I'm pasting it here, as
>> per the bugreport rules:
>> 
>> 0 ? root ? ~ # gdb /usr/lib/dovecot/auth /home/smirky/dovecot.dump
>> GNU gdb (GDB) 8.2.1
>> Copyright (C) 2018 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later 
>> <http://gnu.org/licenses/gpl.html>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.
>> Type "show copying" and "show warranty" for
details.
>> This GDB was configured as "x86_64-pc-linux-gnu".
>> Type "show configuration" for configuration details.
>> For bug reporting instructions, please see:
>> <http://www.gnu.org/software/gdb/bugs/>.
>> Find the GDB manual and other documentation resources online at:
>> <http://www.gnu.org/software/gdb/documentation/>.
>> 
>> For help, type "help".
>> Type "apropos word" to search for commands related to
"word"...
>> Reading symbols from /usr/lib/dovecot/auth...(no debugging symbols 
>> found)...done.
>> [New LWP 9549]
>> [Thread debugging using libthread_db enabled]
>> Using host libthread_db library "/usr/lib/libthread_db.so.1".
>> Core was generated by `dovecot/auth -w'.
>> Program terminated with signal SIGABRT, Aborted.
>> #0  0x00007fe1bb8ead7f in raise () from /usr/lib/libc.so.6
>> (gdb) bt full
>> #0  0x00007fe1bb8ead7f in raise () from /usr/lib/libc.so.6
>> No symbol table info available.
>> #1  0x00007fe1bb8d5672 in abort () from /usr/lib/libc.so.6
>> No symbol table info available.
>> #2  0x00007fe1bb92d878 in __libc_message () from /usr/lib/libc.so.6
>> No symbol table info available.
>> #3  0x00007fe1bb93418a in malloc_printerr () from /usr/lib/libc.so.6
>> No symbol table info available.
>> #4  0x00007fe1bb935b27 in _int_free () from /usr/lib/libc.so.6
>> No symbol table info available.
>> #5  0x00007fe1bbc2bb89 in mysql_close () from /usr/lib/libmariadb.so.3
>> No symbol table info available.
>> #6  0x0000563481946dac in ?? ()
>> No symbol table info available.
>> #7  0x000056348193b8fc in db_sql_unref ()
>> No symbol table info available.
>> #8  0x000056348193389c in userdb_deinit ()
>> No symbol table info available.
>> #9  0x0000563481913bb9 in auths_deinit ()
>> No symbol table info available.
>> #10 0x0000563481912664 in main ()
>> No symbol table info available.
>> (gdb)
>> 
>> The way I see it, indeed it is highly-likely to be related to the 
>> MariaDB version bump,
>> but I cannot be entirely sure if it's not a bug in Dovecot,
discovered
>> by this bump.
>> Please let me know if you need any other info, related to my setup.
> 
> It is very hard to be sure. Dovecot could be using the MariaDB API 
> wrong (e.g., calling mysql_close() more than once) causing it to call 
> abort().
> 
> You should install debug symbols for Dovecot and perhaps also MariaDB 
> to obtain a useful gdb backtrace. Now, it has no symbol table for any 
> of the function calls.
> 
> In most cases is useful to also provide your configuration (output from 
> `dovecot -n`).
> 
> Regards,
> 
> Stephan.

Aki Tuomi

2019-Jan-20 13:11 UTC

head link

auth-worker service failed since recent MariaDB upgrade

<!doctype html>
<html>
 <head> 
  <meta charset="UTF-8"> 
 </head>
 <body>
  <div>
   Can you try if 
https://github.com/dovecot/core/commit/3c5101ffdd2a8115e03ed7180d53578765dea4c9.patch
helps?
   <br>
  </div>
  <div>
   <br>
  </div>
  <div>
   Aki
  </div>
  <blockquote type="cite">
   <div>
    On 20 January 2019 at 15:05 Bogomil Vasilev via dovecot <
    <a
href="mailto:dovecot@dovecot.org">dovecot@dovecot.org</a>>
wrote:
   </div>
   <div>
    <br>
   </div>
   <div>
    <br>
   </div>
   <div>
    Hello,
   </div>
   <div>
    <br>
   </div>
   <div>
    I rebuilt mariadb and dovecot with debug symbols.
   </div>
   <div>
    The backtrace is this:
   </div>
   <div>
    <br>
   </div>
   <div>
    Program terminated with signal SIGABRT, Aborted.
   </div>
   <div>
    #0 0x00007fb040e0cd7f in raise () from /usr/lib/libc.so.6
   </div>
   <div>
    (gdb) bt full
   </div>
   <div>
    #0 0x00007fb040e0cd7f in raise () from /usr/lib/libc.so.6
   </div>
   <div>
    No symbol table info available.
   </div>
   <div>
    #1 0x00007fb040df7672 in abort () from /usr/lib/libc.so.6
   </div>
   <div>
    No symbol table info available.
   </div>
   <div>
    #2 0x00007fb040e4f878 in __libc_message () from /usr/lib/libc.so.6
   </div>
   <div>
    No symbol table info available.
   </div>
   <div>
    #3 0x00007fb040e5618a in malloc_printerr () from /usr/lib/libc.so.6
   </div>
   <div>
    No symbol table info available.
   </div>
   <div>
    #4 0x00007fb040e57b27 in _int_free () from /usr/lib/libc.so.6
   </div>
   <div>
    No symbol table info available.
   </div>
   <div>
    #5 0x00007fb04114db89 in mysql_close (mysql=0x5592728448e0) at
   </div>
   <div>
    /usr/src/debug/mariadb-10.3.12/libmariadb/libmariadb/mariadb_lib.c:1940
   </div>
   <div>
    p = <optimized out>
   </div>
   <div>
    #6 mysql_close (mysql=0x5592728448e0) at
   </div>
   <div>
    /usr/src/debug/mariadb-10.3.12/libmariadb/libmariadb/mariadb_lib.c:1913
   </div>
   <div>
    p = <optimized out>
   </div>
   <div>
    #7 0x00005592724e2dac in driver_sqlpool_disconnect (_db=0x559272843be0)
   </div>
   <div>
    at driver-sqlpool.c:590
   </div>
   <div>
    conn__foreach_end = 0x559272844090
   </div>
   <div>
    db = 0x559272843be0
   </div>
   <div>
    conn = 0x559272844080
   </div>
   <div>
    #8 0x00005592724d78fc in db_sql_unref (_conn=0x55927283b5e8) at
   </div>
   <div>
    db-sql.c:128
   </div>
   <div>
    conn = 0x559272843818
   </div>
   <div>
    #9 0x00005592724cf89c in userdb_deinit (userdb=0x55927283b5c0) at
   </div>
   <div>
    userdb.c:191
   </div>
   <div>
    idx = 0
   </div>
   <div>
    __func__ = "userdb_deinit"
   </div>
   <div>
    #10 0x00005592724afbb9 in auth_deinit (auth=0x55927283b320) at
   </div>
   <div>
    auth.c:335
   </div>
   <div>
    passdb = <optimized out>
   </div>
   <div>
    userdb = 0x55927283b4e0
   </div>
   <div>
    passdb = <optimized out>
   </div>
   <div>
    userdb = <optimized out>
   </div>
   <div>
    #11 auths_deinit () at auth.c:433
   </div>
   <div>
    auth__foreach_end = 0x559272842e18
   </div>
   <div>
    auth = 0x559272842e10
   </div>
   <div>
    #12 0x00005592724ae664 in main_deinit () at main.c:271
   </div>
   <div>
    l = <optimized out>
   </div>
   <div>
    l = <optimized out>
   </div>
   <div>
    l_end = <optimized out>
   </div>
   <div>
    #13 main (argc=<optimized out>, argv=<optimized out>) at
main.c:398
   </div>
   <div>
    c = <optimized out>
   </div>
   <div>
    (gdb)
   </div>
   <div>
    <br>
   </div>
   <div>
    <br>
   </div>
   <div>
    My `dovecot -n` output is this:
   </div>
   <div>
    <br>
   </div>
   <div>
    0 « root » /home/smirky/stuff/ABS/dovecot # dovecot -n
   </div>
   <div>
    # 2.3.4 (0ecbaf23d): /etc/dovecot/dovecot.conf
   </div>
   <div>
    # Pigeonhole version 0.5.4 (60b0f48d)
   </div>
   <div>
    # OS: Linux 5.0.0-rc2-mainline x86_64 Arch Linux
   </div>
   <div>
    # Hostname: archy
   </div>
   <div>
    dict {
   </div>
   <div>
    quotadict = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
   </div>
   <div>
    }
   </div>
   <div>
    mail_home = /home/vmail/%d/%u
   </div>
   <div>
    mail_location = maildir:~
   </div>
   <div>
    mail_plugins = quota
   </div>
   <div>
    managesieve_notify_capability = mailto
   </div>
   <div>
    managesieve_sieve_capability = fileinto reject envelope
   </div>
   <div>
    encoded-character vacation subaddress comparator-i;ascii-numeric
   </div>
   <div>
    relational regex imap4flags copy include variables body enotify
   </div>
   <div>
    environment mailbox date i
   </div>
   <div>
    ndex ihave duplicate mime foreverypart extracttext imapsieve
   </div>
   <div>
    vnd.dovecot.imapsieve
   </div>
   <div>
    namespace inbox {
   </div>
   <div>
    inbox = yes
   </div>
   <div>
    location    </div>
   <div>
    mailbox Drafts {
   </div>
   <div>
    auto = create
   </div>
   <div>
    special_use = \Drafts
   </div>
   <div>
    }
   </div>
   <div>
    mailbox Sent {
   </div>
   <div>
    auto = subscribe
   </div>
   <div>
    special_use = \Sent
   </div>
   <div>
    }
   </div>
   <div>
    mailbox "Sent Messages" {
   </div>
   <div>
    auto = no
   </div>
   <div>
    special_use = \Sent
   </div>
   <div>
    }
   </div>
   <div>
    mailbox Spam {
   </div>
   <div>
    auto = subscribe
   </div>
   <div>
    special_use = \Junk
   </div>
   <div>
    }
   </div>
   <div>
    mailbox Trash {
   </div>
   <div>
    auto = subscribe
   </div>
   <div>
    special_use = \Trash
   </div>
   <div>
    }
   </div>
   <div>
    prefix    </div>
   <div>
    }
   </div>
   <div>
    passdb {
   </div>
   <div>
    args = /etc/dovecot/dovecot-sql.conf
   </div>
   <div>
    driver = sql
   </div>
   <div>
    }
   </div>
   <div>
    plugin {
   </div>
   <div>
    imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve
   </div>
   <div>
    imapsieve_mailbox1_causes = COPY
   </div>
   <div>
    imapsieve_mailbox1_name = Spam
   </div>
   <div>
    imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve
   </div>
   <div>
    imapsieve_mailbox2_causes = COPY
   </div>
   <div>
    imapsieve_mailbox2_from = Junk
   </div>
   <div>
    imapsieve_mailbox2_name = *
   </div>
   <div>
    quota = dict:User quota::proxy::quotadict
   </div>
   <div>
    quota_rule2 = Trash:storage=+10%%
   </div>
   <div>
    quota_warning = storage=95%% quota-warning 95 %u
   </div>
   <div>
    quota_warning2 = storage=80%% quota-warning 80 %u
   </div>
   <div>
    quota_warning3 = -storage=100%% quota-warning below 100 %u
   </div>
   <div>
    sieve_before = /etc/dovecot/sieve/filter.sieve
   </div>
   <div>
    sieve_global_extensions = +vnd.dovecot.pipe
   </div>
   <div>
    sieve_pipe_bin_dir = /etc/dovecot/sieve
   </div>
   <div>
    sieve_plugins = sieve_imapsieve sieve_extprograms
   </div>
   <div>
    }
   </div>
   <div>
    protocols = imap pop3 sieve
   </div>
   <div>
    service auth-worker {
   </div>
   <div>
    unix_listener auth-worker {
   </div>
   <div>
    group = vmail
   </div>
   <div>
    mode = 0666
   </div>
   <div>
    user = vmail
   </div>
   <div>
    }
   </div>
   <div>
    }
   </div>
   <div>
    service auth {
   </div>
   <div>
    unix_listener /var/spool/postfix/private/auth {
   </div>
   <div>
    group = postfix
   </div>
   <div>
    mode = 0660
   </div>
   <div>
    user = postfix
   </div>
   <div>
    }
   </div>
   <div>
    user = root
   </div>
   <div>
    }
   </div>
   <div>
    service dict {
   </div>
   <div>
    unix_listener dict {
   </div>
   <div>
    group = vmail
   </div>
   <div>
    mode = 0660
   </div>
   <div>
    user = vmail
   </div>
   <div>
    }
   </div>
   <div>
    user = root
   </div>
   <div>
    }
   </div>
   <div>
    service imap-login {
   </div>
   <div>
    inet_listener imap {
   </div>
   <div>
    address = 127.0.0.1
   </div>
   <div>
    }
   </div>
   <div>
    }
   </div>
   <div>
    service managesieve-login {
   </div>
   <div>
    inet_listener sieve {
   </div>
   <div>
    port = 4190
   </div>
   <div>
    }
   </div>
   <div>
    }
   </div>
   <div>
    service pop3-login {
   </div>
   <div>
    inet_listener pop3 {
   </div>
   <div>
    address = 127.0.0.1
   </div>
   <div>
    }
   </div>
   <div>
    }
   </div>
   <div>
    service quota-warning {
   </div>
   <div>
    executable = script /usr/local/bin/quota-warning.sh
   </div>
   <div>
    unix_listener quota-warning {
   </div>
   <div>
    group = vmail
   </div>
   <div>
    mode = 0660
   </div>
   <div>
    user = vmail
   </div>
   <div>
    }
   </div>
   <div>
    user = vmail
   </div>
   <div>
    }
   </div>
   <div>
    service stats {
   </div>
   <div>
    unix_listener stats-reader {
   </div>
   <div>
    group = vmail
   </div>
   <div>
    mode = 0660
   </div>
   <div>
    user = vmail
   </div>
   <div>
    }
   </div>
   <div>
    unix_listener stats-writer {
   </div>
   <div>
    group = vmail
   </div>
   <div>
    mode = 0660
   </div>
   <div>
    user = vmail
   </div>
   <div>
    }
   </div>
   <div>
    }
   </div>
   <div>
    ssl_cert = </etc/letsencrypt/live/smirky.net/fullchain.pem
   </div>
   <div>
    ssl_dh = # hidden, use -P to show it
   </div>
   <div>
    ssl_key = # hidden, use -P to show it
   </div>
   <div>
    userdb {
   </div>
   <div>
    args = /etc/dovecot/dovecot-sql.conf
   </div>
   <div>
    driver = sql
   </div>
   <div>
    }
   </div>
   <div>
    protocol pop3 {
   </div>
   <div>
    mail_plugins = quota
   </div>
   <div>
    pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
   </div>
   <div>
    pop3_uidl_format = %08Xu%08Xv
   </div>
   <div>
    }
   </div>
   <div>
    protocol lda {
   </div>
   <div>
    mail_plugins = quota
   </div>
   <div>
    postmaster_address = 
    <a
href="mailto:postmaster@smirky.net">postmaster@smirky.net</a>
   </div>
   <div>
    }
   </div>
   <div>
    protocol imap {
   </div>
   <div>
    mail_max_userip_connections = 20
   </div>
   <div>
    mail_plugin_dir = /usr/lib/dovecot/modules
   </div>
   <div>
    mail_plugins = quota imap_quota imap_sieve
   </div>
   <div>
    }
   </div>
   <div>
    <br>
   </div>
   <div>
    <br>
   </div>
   <div>
    ---
   </div>
   <div>
    BOGOMIL VASILEV
   </div>
   <div>
    For contacts, use this e-mail. My GPG key is 646F3ABF6E457336
   </div>
   <div>
    Website: 
    <a href="HTTPS://WWW.SMIRKY.NET/" rel="noopener"
target="_blank">HTTPS://WWW.SMIRKY.NET/</a>
   </div>
   <div>
    <br>
   </div>
   <div>
    On 2019-01-20 13:22, Stephan Bosch wrote:
   </div>
   <div>
    <br>
   </div>
   <blockquote type="cite">
    <div>
     Op 20/01/2019 om 10:50 schreef Bogomil Vasilev via dovecot:
    </div>
    <div>
     <br>
    </div>
    <blockquote type="cite">
     <div>
      Hello,
     </div>
     <div>
      <br>
     </div>
     <div>
      I have started having issues with auth-worker. Dovecot version:
     </div>
     <div>
      <br>
     </div>
     <div>
      0 « root » ~ # dovecot --version
     </div>
     <div>
      2.3.4 (0ecbaf23d)
     </div>
     <div>
      0 « root » ~ # pacman -Q dovecot
     </div>
     <div>
      dovecot 2.3.4-2
     </div>
     <div>
      <br>
     </div>
     <div>
      OS: Arch Linux
     </div>
     <div>
      <br>
     </div>
     <div>
      From /var/log/mail.log:
     </div>
     <div>
      <br>
     </div>
     <div>
      Jan 20 11:31:29 archy dovecot[23070]: auth-worker: Fatal: master:
     </div>
     <div>
      service(auth-worker): child 9885 killed with signal 6 (core not dumped
     </div>
     <div>
      - 
      <a href="https://dovecot.org/bugreport.html#coredumps"
rel="noopener"
target="_blank">https://dovecot.org/bugreport.html#coredumps</a>
- set
     </div>
     <div>
      /proc/sys/fs/suid_dum
     </div>
     <div>
      pable to 2)
     </div>
     <div>
      <br>
     </div>
     <div>
      From dmesg:
     </div>
     <div>
      <br>
     </div>
     <div>
      [42113.729398] auth[8134]: segfault at 5579d8e1c8e0 ip
     </div>
     <div>
      00005579d8e1c8e0 sp 00007ffd44ab0df8 error 15
     </div>
     <div>
      [42113.729406] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00
     </div>
     <div>
      00 00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 7a af 07 e3
     </div>
     <div>
      7f 00 00 <40> f4 b5 07 e3 7f 00 00 31 a0 00 00 00 00 00 00 e0 9f df
d8
     </div>
     <div>
      79
     </div>
     <div>
      55
     </div>
     <div>
      <br>
     </div>
     <div>
      This started happening on January 18th, after I upgraded Dovecot from
     </div>
     <div>
      "2.3.4-1" to "2.3.4-2".
     </div>
     <div>
      The package was rebuilt from our package maintainer due to bumping
     </div>
     <div>
      MariaDB from 10.1.x to 10.3.x.
     </div>
     <div>
      Before that, I did not have any issues at all.
     </div>
     <div>
      However, I don't see any impact from this error too, at least not for
     </div>
     <div>
      now.
     </div>
     <div>
      I made a coredump as the error suggests and I'm pasting it here, as
     </div>
     <div>
      per the bugreport rules:
     </div>
     <div>
      <br>
     </div>
     <div>
      0 « root » ~ # gdb /usr/lib/dovecot/auth /home/smirky/dovecot.dump
     </div>
     <div>
      GNU gdb (GDB) 8.2.1
     </div>
     <div>
      Copyright (C) 2018 Free Software Foundation, Inc.
     </div>
     <div>
      License GPLv3+: GNU GPL version 3 or later
     </div>
     <div>
      <
      <a href="http://gnu.org/licenses/gpl.html"
rel="noopener"
target="_blank">http://gnu.org/licenses/gpl.html</a>>
     </div>
     <div>
      This is free software: you are free to change and redistribute it.
     </div>
     <div>
      There is NO WARRANTY, to the extent permitted by law.
     </div>
     <div>
      Type "show copying" and "show warranty" for details.
     </div>
     <div>
      This GDB was configured as "x86_64-pc-linux-gnu".
     </div>
     <div>
      Type "show configuration" for configuration details.
     </div>
     <div>
      For bug reporting instructions, please see:
     </div>
     <div>
      <
      <a href="http://www.gnu.org/software/gdb/bugs/>"
rel="noopener" target="_blank"
data-mce-href="http://www.gnu.org/software/gdb/bugs/">">http://www.gnu.org/software/gdb/bugs/></a>.
     </div>
     <div>
      Find the GDB manual and other documentation resources online at:
     </div>
     <div>
      <
      <a href="http://www.gnu.org/software/gdb/documentation/>"
rel="noopener" target="_blank"
data-mce-href="http://www.gnu.org/software/gdb/documentation/">">http://www.gnu.org/software/gdb/documentation/></a>.
     </div>
     <div>
      <br>
     </div>
     <div>
      For help, type "help".
     </div>
     <div>
      Type "apropos word" to search for commands related to
"word"...
     </div>
     <div>
      Reading symbols from /usr/lib/dovecot/auth...(no debugging symbols
     </div>
     <div>
      found)...done.
     </div>
     <div>
      [New LWP 9549]
     </div>
     <div>
      [Thread debugging using libthread_db enabled]
     </div>
     <div>
      Using host libthread_db library "/usr/lib/libthread_db.so.1".
     </div>
     <div>
      Core was generated by `dovecot/auth -w'.
     </div>
     <div>
      Program terminated with signal SIGABRT, Aborted.
     </div>
     <div>
      #0 0x00007fe1bb8ead7f in raise () from /usr/lib/libc.so.6
     </div>
     <div>
      (gdb) bt full
     </div>
     <div>
      #0 0x00007fe1bb8ead7f in raise () from /usr/lib/libc.so.6
     </div>
     <div>
      No symbol table info available.
     </div>
     <div>
      #1 0x00007fe1bb8d5672 in abort () from /usr/lib/libc.so.6
     </div>
     <div>
      No symbol table info available.
     </div>
     <div>
      #2 0x00007fe1bb92d878 in __libc_message () from /usr/lib/libc.so.6
     </div>
     <div>
      No symbol table info available.
     </div>
     <div>
      #3 0x00007fe1bb93418a in malloc_printerr () from /usr/lib/libc.so.6
     </div>
     <div>
      No symbol table info available.
     </div>
     <div>
      #4 0x00007fe1bb935b27 in _int_free () from /usr/lib/libc.so.6
     </div>
     <div>
      No symbol table info available.
     </div>
     <div>
      #5 0x00007fe1bbc2bb89 in mysql_close () from /usr/lib/libmariadb.so.3
     </div>
     <div>
      No symbol table info available.
     </div>
     <div>
      #6 0x0000563481946dac in ?? ()
     </div>
     <div>
      No symbol table info available.
     </div>
     <div>
      #7 0x000056348193b8fc in db_sql_unref ()
     </div>
     <div>
      No symbol table info available.
     </div>
     <div>
      #8 0x000056348193389c in userdb_deinit ()
     </div>
     <div>
      No symbol table info available.
     </div>
     <div>
      #9 0x0000563481913bb9 in auths_deinit ()
     </div>
     <div>
      No symbol table info available.
     </div>
     <div>
      #10 0x0000563481912664 in main ()
     </div>
     <div>
      No symbol table info available.
     </div>
     <div>
      (gdb)
     </div>
     <div>
      <br>
     </div>
     <div>
      The way I see it, indeed it is highly-likely to be related to the
     </div>
     <div>
      MariaDB version bump,
     </div>
     <div>
      but I cannot be entirely sure if it's not a bug in Dovecot, discovered
     </div>
     <div>
      by this bump.
     </div>
     <div>
      Please let me know if you need any other info, related to my setup.
     </div>
    </blockquote>
    <div>
     It is very hard to be sure. Dovecot could be using the MariaDB API
    </div>
    <div>
     wrong (e.g., calling mysql_close() more than once) causing it to call
    </div>
    <div>
     abort().
    </div>
    <div>
     <br>
    </div>
    <div>
     You should install debug symbols for Dovecot and perhaps also MariaDB
    </div>
    <div>
     to obtain a useful gdb backtrace. Now, it has no symbol table for any
    </div>
    <div>
     of the function calls.
    </div>
    <div>
     <br>
    </div>
    <div>
     In most cases is useful to also provide your configuration (output from
    </div>
    <div>
     `dovecot -n`).
    </div>
    <div>
     <br>
    </div>
    <div>
     Regards,
    </div>
    <div>
     <br>
    </div>
    <div>
     Stephan.
    </div>
   </blockquote>
  </blockquote>
  <div>
   <br>
  </div>
  <div class="io-ox-signature">
   ---
   <br>Aki Tuomi
  </div> 
 </body>
</html>

Bogomil Vasilev

2019-Jan-20 14:30 UTC

head link

auth-worker service failed since recent MariaDB upgrade

Hello Aki,

This patch seems to have done the trick - hopefully it works fine and 
doesn't mask the problem.
 From the patch content, I see that it's from 4th of December.
Is there a reason why this hasn't made it yet into upstream?
Did you come across this bug on your own and patch it temporarily?

Best regards,

---
BOGOMIL VASILEV
For contacts, use this e-mail. My GPG key is 646F3ABF6E457336
Website: HTTPS://WWW.SMIRKY.NET/

On 2019-01-20 15:11, Aki Tuomi wrote:
> Can you try if  
>
https://github.com/dovecot/core/commit/3c5101ffdd2a8115e03ed7180d53578765dea4c9.patch
> helps?
> 
> Aki
> On 20 January 2019 at 15:05 Bogomil Vasilev via dovecot < 
> dovecot at dovecot.org> wrote:
> 
> Hello,
> 
> I rebuilt mariadb and dovecot with debug symbols.
> The backtrace is this:
> 
> Program terminated with signal SIGABRT, Aborted.
> #0 0x00007fb040e0cd7f in raise () from /usr/lib/libc.so.6
> (gdb) bt full
> #0 0x00007fb040e0cd7f in raise () from /usr/lib/libc.so.6
> No symbol table info available.
> #1 0x00007fb040df7672 in abort () from /usr/lib/libc.so.6
> No symbol table info available.
> #2 0x00007fb040e4f878 in __libc_message () from /usr/lib/libc.so.6
> No symbol table info available.
> #3 0x00007fb040e5618a in malloc_printerr () from /usr/lib/libc.so.6
> No symbol table info available.
> #4 0x00007fb040e57b27 in _int_free () from /usr/lib/libc.so.6
> No symbol table info available.
> #5 0x00007fb04114db89 in mysql_close (mysql=0x5592728448e0) at
> /usr/src/debug/mariadb-10.3.12/libmariadb/libmariadb/mariadb_lib.c:1940
> p = <optimized out>
> #6 mysql_close (mysql=0x5592728448e0) at
> /usr/src/debug/mariadb-10.3.12/libmariadb/libmariadb/mariadb_lib.c:1913
> p = <optimized out>
> #7 0x00005592724e2dac in driver_sqlpool_disconnect (_db=0x559272843be0)
> at driver-sqlpool.c:590
> conn__foreach_end = 0x559272844090
> db = 0x559272843be0
> conn = 0x559272844080
> #8 0x00005592724d78fc in db_sql_unref (_conn=0x55927283b5e8) at
> db-sql.c:128
> conn = 0x559272843818
> #9 0x00005592724cf89c in userdb_deinit (userdb=0x55927283b5c0) at
> userdb.c:191
> idx = 0
> __func__ = "userdb_deinit"
> #10 0x00005592724afbb9 in auth_deinit (auth=0x55927283b320) at
> auth.c:335
> passdb = <optimized out>
> userdb = 0x55927283b4e0
> passdb = <optimized out>
> userdb = <optimized out>
> #11 auths_deinit () at auth.c:433
> auth__foreach_end = 0x559272842e18
> auth = 0x559272842e10
> #12 0x00005592724ae664 in main_deinit () at main.c:271
> l = <optimized out>
> l = <optimized out>
> l_end = <optimized out>
> #13 main (argc=<optimized out>, argv=<optimized out>) at
main.c:398
> c = <optimized out>
> (gdb)
> 
> My `dovecot -n` output is this:
> 
> 0 ? root ? /home/smirky/stuff/ABS/dovecot # dovecot -n
> # 2.3.4 (0ecbaf23d): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.4 (60b0f48d)
> # OS: Linux 5.0.0-rc2-mainline x86_64 Arch Linux
> # Hostname: archy
> dict {
> quotadict = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
> }
> mail_home = /home/vmail/%d/%u
> mail_location = maildir:~
> mail_plugins = quota
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope
> encoded-character vacation subaddress comparator-i;ascii-numeric
> relational regex imap4flags copy include variables body enotify
> environment mailbox date i
> ndex ihave duplicate mime foreverypart extracttext imapsieve
> vnd.dovecot.imapsieve
> namespace inbox {
> inbox = yes
> location > mailbox Drafts {
> auto = create
> special_use = \Drafts
> }
> mailbox Sent {
> auto = subscribe
> special_use = \Sent
> }
> mailbox "Sent Messages" {
> auto = no
> special_use = \Sent
> }
> mailbox Spam {
> auto = subscribe
> special_use = \Junk
> }
> mailbox Trash {
> auto = subscribe
> special_use = \Trash
> }
> prefix > }
> passdb {
> args = /etc/dovecot/dovecot-sql.conf
> driver = sql
> }
> plugin {
> imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve
> imapsieve_mailbox1_causes = COPY
> imapsieve_mailbox1_name = Spam
> imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve
> imapsieve_mailbox2_causes = COPY
> imapsieve_mailbox2_from = Junk
> imapsieve_mailbox2_name = *
> quota = dict:User quota::proxy::quotadict
> quota_rule2 = Trash:storage=+10%%
> quota_warning = storage=95%% quota-warning 95 %u
> quota_warning2 = storage=80%% quota-warning 80 %u
> quota_warning3 = -storage=100%% quota-warning below 100 %u
> sieve_before = /etc/dovecot/sieve/filter.sieve
> sieve_global_extensions = +vnd.dovecot.pipe
> sieve_pipe_bin_dir = /etc/dovecot/sieve
> sieve_plugins = sieve_imapsieve sieve_extprograms
> }
> protocols = imap pop3 sieve
> service auth-worker {
> unix_listener auth-worker {
> group = vmail
> mode = 0666
> user = vmail
> }
> }
> service auth {
> unix_listener /var/spool/postfix/private/auth {
> group = postfix
> mode = 0660
> user = postfix
> }
> user = root
> }
> service dict {
> unix_listener dict {
> group = vmail
> mode = 0660
> user = vmail
> }
> user = root
> }
> service imap-login {
> inet_listener imap {
> address = 127.0.0.1
> }
> }
> service managesieve-login {
> inet_listener sieve {
> port = 4190
> }
> }
> service pop3-login {
> inet_listener pop3 {
> address = 127.0.0.1
> }
> }
> service quota-warning {
> executable = script /usr/local/bin/quota-warning.sh
> unix_listener quota-warning {
> group = vmail
> mode = 0660
> user = vmail
> }
> user = vmail
> }
> service stats {
> unix_listener stats-reader {
> group = vmail
> mode = 0660
> user = vmail
> }
> unix_listener stats-writer {
> group = vmail
> mode = 0660
> user = vmail
> }
> }
> ssl_cert = </etc/letsencrypt/live/smirky.net/fullchain.pem
> ssl_dh = # hidden, use -P to show it
> ssl_key = # hidden, use -P to show it
> userdb {
> args = /etc/dovecot/dovecot-sql.conf
> driver = sql
> }
> protocol pop3 {
> mail_plugins = quota
> pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
> pop3_uidl_format = %08Xu%08Xv
> }
> protocol lda {
> mail_plugins = quota
> postmaster_address = postmaster at smirky.net
> }
> protocol imap {
> mail_max_userip_connections = 20
> mail_plugin_dir = /usr/lib/dovecot/modules
> mail_plugins = quota imap_quota imap_sieve
> }
> 
> ---
> BOGOMIL VASILEV
> For contacts, use this e-mail. My GPG key is 646F3ABF6E457336
> Website: HTTPS://WWW.SMIRKY.NET/
> 
> On 2019-01-20 13:22, Stephan Bosch wrote:
> 
> Op 20/01/2019 om 10:50 schreef Bogomil Vasilev via dovecot:
> 
> Hello,
> 
> I have started having issues with auth-worker. Dovecot version:
> 
> 0 ? root ? ~ # dovecot --version
> 2.3.4 (0ecbaf23d)
> 0 ? root ? ~ # pacman -Q dovecot
> dovecot 2.3.4-2
> 
> OS: Arch Linux
> 
> From /var/log/mail.log:
> 
> Jan 20 11:31:29 archy dovecot[23070]: auth-worker: Fatal: master:
> service(auth-worker): child 9885 killed with signal 6 (core not dumped
> - https://dovecot.org/bugreport.html#coredumps - set
> /proc/sys/fs/suid_dum
> pable to 2)
> 
> From dmesg:
> 
> [42113.729398] auth[8134]: segfault at 5579d8e1c8e0 ip
> 00005579d8e1c8e0 sp 00007ffd44ab0df8 error 15
> [42113.729406] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00
> 00 00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 7a af 07 e3
> 7f 00 00 <40> f4 b5 07 e3 7f 00 00 31 a0 00 00 00 00 00 00 e0 9f df
d8
> 79
> 55
> 
> This started happening on January 18th, after I upgraded Dovecot from
> "2.3.4-1" to "2.3.4-2".
> The package was rebuilt from our package maintainer due to bumping
> MariaDB from 10.1.x to 10.3.x.
> Before that, I did not have any issues at all.
> However, I don't see any impact from this error too, at least not for
> now.
> I made a coredump as the error suggests and I'm pasting it here, as
> per the bugreport rules:
> 
> 0 ? root ? ~ # gdb /usr/lib/dovecot/auth /home/smirky/dovecot.dump
> GNU gdb (GDB) 8.2.1
> Copyright (C) 2018 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> < http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> Type "show copying" and "show warranty" for details.
> This GDB was configured as "x86_64-pc-linux-gnu".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> < ">http://www.gnu.org/software/gdb/bugs/>.
> Find the GDB manual and other documentation resources online at:
> < ">http://www.gnu.org/software/gdb/documentation/>.
> 
> For help, type "help".
> Type "apropos word" to search for commands related to
"word"...
> Reading symbols from /usr/lib/dovecot/auth...(no debugging symbols
> found)...done.
> [New LWP 9549]
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/usr/lib/libthread_db.so.1".
> Core was generated by `dovecot/auth -w'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x00007fe1bb8ead7f in raise () from /usr/lib/libc.so.6
> (gdb) bt full
> #0 0x00007fe1bb8ead7f in raise () from /usr/lib/libc.so.6
> No symbol table info available.
> #1 0x00007fe1bb8d5672 in abort () from /usr/lib/libc.so.6
> No symbol table info available.
> #2 0x00007fe1bb92d878 in __libc_message () from /usr/lib/libc.so.6
> No symbol table info available.
> #3 0x00007fe1bb93418a in malloc_printerr () from /usr/lib/libc.so.6
> No symbol table info available.
> #4 0x00007fe1bb935b27 in _int_free () from /usr/lib/libc.so.6
> No symbol table info available.
> #5 0x00007fe1bbc2bb89 in mysql_close () from /usr/lib/libmariadb.so.3
> No symbol table info available.
> #6 0x0000563481946dac in ?? ()
> No symbol table info available.
> #7 0x000056348193b8fc in db_sql_unref ()
> No symbol table info available.
> #8 0x000056348193389c in userdb_deinit ()
> No symbol table info available.
> #9 0x0000563481913bb9 in auths_deinit ()
> No symbol table info available.
> #10 0x0000563481912664 in main ()
> No symbol table info available.
> (gdb)
> 
> The way I see it, indeed it is highly-likely to be related to the
> MariaDB version bump,
> but I cannot be entirely sure if it's not a bug in Dovecot, discovered
> by this bump.
> Please let me know if you need any other info, related to my setup.
> It is very hard to be sure. Dovecot could be using the MariaDB API
> wrong (e.g., calling mysql_close() more than once) causing it to call
> abort().
> 
> You should install debug symbols for Dovecot and perhaps also MariaDB
> to obtain a useful gdb backtrace. Now, it has no symbol table for any
> of the function calls.
> 
> In most cases is useful to also provide your configuration (output from
> `dovecot -n`).
> 
> Regards,
> 
> Stephan.
---
Aki Tuomi

Maybe Matching Threads

Search for more maybe matching threads

dovecot - Jan 2019 - auth-worker service failed since recent MariaDB upgrade

auth-worker service failed since recent MariaDB upgrade

auth-worker service failed since recent MariaDB upgrade

auth-worker service failed since recent MariaDB upgrade

Maybe Matching Threads