Hi all,
I recently upgraded from courier to dovecot 1.1.12 on a Solaris 9 system
with about 100 users. We have been testing dovecot for sometime in a
mixed Linux/Solaris environment and are aware of the index endianess
issue with multiple archs. To solve this, we run with INDEX=MEMORY (as
seen in the docs) so that only Linux clients have anything to do with
the indexes. We generally have much more of them anyways. So far
everything is working smoothly, but when someone does a search through
directory with a large number of emails, dovecot dies and prints the
following message:
[ID 107833 mail.crit] Panic: Trying to allocate 2147483648 bytes
I googled around and saw this same issue elsewhere, with NFS indexes
usually being the problem. However, we have indexes turned off, so I
don't understand why the same problem is still happening.
Just to be sure, I ran with:
mmap_disable = yes
dotlock_use_excl = no
mail_nfs_storage = yes
But there has been no change, I can still reproduce it at will.
Setting mail_nfs_index = yes caused my connections to stop working, so
that didn't help either. Besides, I wouldn't think that would be the
answer considering we set INDEX=MEMORY.
This thread also seems to be related, but we're definitely running with
this code in 1.1.12, yet it still happens:
http://dovecot.org/list/dovecot/2008-September/033615.html
I grabbed a backtrace if it is helpful:
http://pastebin.com/f5f3d282c
Also, here is dovecot -n:
# 1.1.12: /usr/local/etc/dovecot.conf
# OS: SunOS 5.9 sun4u
protocols: imap imaps pop3 pop3s
ssl_cert_file: /usr/local/ssl/certs/nbcs.key+crt.pem
ssl_key_file: /usr/local/ssl/certs/nbcs.key+crt.pem
login_dir: /usr/local/var/run/dovecot/login
login_executable(default): /usr/local/libexec/dovecot/imap-login
login_executable(imap): /usr/local/libexec/dovecot/imap-login
login_executable(pop3): /usr/local/libexec/dovecot/pop3-login
login_greeting_capability(default): yes
login_greeting_capability(imap): yes
login_greeting_capability(pop3): no
mail_location: maildir:~/Maildir:INDEX=MEMORY
mmap_disable: yes
mail_nfs_storage: yes
mail_executable(default): /usr/local/libexec/dovecot/imap
mail_executable(imap): /usr/local/libexec/dovecot/imap
mail_executable(pop3): /usr/local/libexec/dovecot/pop3
mail_plugins(default): quota imap_quota
mail_plugins(imap): quota imap_quota
mail_plugins(pop3): quota pop3_quota
mail_plugin_dir(default): /usr/local/lib/dovecot/imap
mail_plugin_dir(imap): /usr/local/lib/dovecot/imap
mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3
pop3_uidl_format(default): %08Xu%08Xv
pop3_uidl_format(imap): %08Xu%08Xv
pop3_uidl_format(pop3): UID%u-%v
namespace:
type: private
separator: .
prefix: INBOX.
inbox: yes
list: yes
subscriptions: yes
auth default:
passdb:
driver: pam
args: *
userdb:
driver: passwd
plugin:
quota: fs
Any help would be appreciated!
Thanks,
-Dave
--
===============================David Halik
System Administrator
OIT-CSS Rutgers University
dhalik at jla.rutgers.edu
================================
On Thu, 2009-03-19 at 15:25 -0400, David Halik wrote:> So far > everything is working smoothly, but when someone does a search through > directory with a large number of emails, dovecot dies and prints the > following message: > > [ID 107833 mail.crit] Panic: Trying to allocate 2147483648 bytesSo you can easily reproduce this?> I grabbed a backtrace if it is helpful:Yes, very helpful! Is it possible to also examine variable values? Could you try applying this patch: http://hg.dovecot.org/dovecot-1.1/rev/98a59fd61d7f and also the attached patch. What error does it fail in? -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 480 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20090319/3c58010b/attachment-0004.bin> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20090319/3c58010b/attachment-0005.bin>
Timo Sirainen wrote:> On Thu, 2009-03-19 at 15:25 -0400, David Halik wrote: > >> So far >> everything is working smoothly, but when someone does a search through >> directory with a large number of emails, dovecot dies and prints the >> following message: >> >> [ID 107833 mail.crit] Panic: Trying to allocate 2147483648 bytes >> > > So you can easily reproduce this? >Everytime and on different systems with a similar setup. Since I can't do debugging on the production server, I'm using a server with a similar setup. The only difference is probably physical memory and swap size. I'm now getting a different size in the panic error, but you get the idea: Mar 20 13:32:10 er0.rutgers.edu IMAP(dhalik): : [ID 107833 mail.crit] Panic: Trying to allocate 2147483648 bytes Also, I've been keeping an eye on top while this was running and it seems that Dovecot is growing at an amazing rate, which is why it runs out of room I guess: 8081 dhalik 1 58 0 1030M 871M run 0:33 53.29% imap This was while doing a full Pine text search for a word in a folder with about 1300 emails. Shortly after it died. Keep in mind that I am only able to reproduce this problem with the homedir on NFS and INDEX=MEMORY on. The same setup on local disk had no issues.> >> I grabbed a backtrace if it is helpful: >> > > Yes, very helpful! Is it possible to also examine variable values? >I'd love to, but it seems that is all the I can get out of Sun Studio unless you know some tricks. I verified that I am building with full debug symbols (-g -xs) and that there is no optimization.> Could you try applying this patch: > http://hg.dovecot.org/dovecot-1.1/rev/98a59fd61d7f > > and also the attached patch. What error does it fail in? >I applied both patches, rebuilt dovecot 1.1.12 with full debug, and I'm not seeing anything different. Was there supposed to be a new error? It still exhibits the same behavior and eventually core dumps when it runs out of room, same as before. Here's the most recent traceback: http://pastebin.com/f4deddc23 I appreciate the help Timo. -- ===============================David Halik System Administrator OIT-CSS Rutgers University dhalik at jla.rutgers.edu ================================