Pavel Stano
2014-May-30 15:02 UTC
[Dovecot] attachment sis + EMLINK (too many links) = segfault bug (2.2.12)
Hi, we use attachment dedup with lots of emails (still migrating to it from maildir). We use netapp storage with wafl filesystem over nfs. Problem is that netapp has hard limit of 100k hardlinks to one file. And we encountered it. Problem is that dovecot start do segfault (lmtp,dsync,pop3 etc) when it happend when tried to deliver new emails with that attachment. Here is strace of dsync: 6740 link("/nfsmnt/mailatch1/f9/10/hashes/f9108ddaa156ac15738e41ed3bedec1eda50175d", "/nfsmnt/mailatch1/f9/10/f9108ddaa156ac15738e41ed3bedec1eda50175d-7bb7a20ddb598853541a000028db4a9f") = -1 EMLINK (Too many links) 6740 --- SIGSEGV (Segmentation fault) @ 0 (0) --- ls -lh: -rw------- 100000 vmail vmail 4.7K Apr 28 16:54 /nfsmnt/mailatch1/f9/10/hashes/f9108ddaa156ac15738e41ed3bedec1eda50175d We were using mail_attachment_min_size=4kb, we solve it by increasing it to 8kb. It would be nice to somehow fix this problem. Like not crash when EMLINK happend and maybe do not deduplicate attachments but deliver email without dedup. Or create second file in hashes/ and start hardlinking it instead of original. AFAIK ext4 has also hard-link limit 64k (http://en.wikipedia.org/wiki/Hard_link#Limitations_of_hard_links) So this can happen to anyone with lots of emails. Thanks -- [ Ohodnotte kvalitu mailu: http://nicereply.com/websupport/Stano/ ] Pavel Stano | Troubleshooter http://WebSupport.sk *** BERTE A VYCHUTNAVAJTE *** -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20140530/e13f3ba0/attachment-0001.sig>
Pavel Stano
2014-Jun-05 14:20 UTC
[Dovecot] attachment sis + EMLINK (too many links) = segfault bug (2.2.12)
Hi, attached patch fix problem with segfault when EMLINK happend when file has too many hard link. On Fri, 30 May 2014 17:02:47 +0200 Pavel Stano <stano at websupport.sk> wrote:> Hi, > > we use attachment dedup with lots of emails (still migrating to it > from maildir). > We use netapp storage with wafl filesystem over nfs. > Problem is that netapp has hard limit of 100k hardlinks to one file. > And we encountered it. > > Problem is that dovecot start do segfault (lmtp,dsync,pop3 etc) when > it happend when tried to deliver new emails with that attachment. > Here is strace of dsync: > > 6740 > link("/nfsmnt/mailatch1/f9/10/hashes/f9108ddaa156ac15738e41ed3bedec1eda50175d", > "/nfsmnt/mailatch1/f9/10/f9108ddaa156ac15738e41ed3bedec1eda50175d-7bb7a20ddb598853541a000028db4a9f") > = -1 EMLINK (Too many links) 6740 --- SIGSEGV (Segmentation fault) @ > 0 (0) --- > > ls -lh: > -rw------- 100000 vmail vmail 4.7K Apr 28 > 16:54 /nfsmnt/mailatch1/f9/10/hashes/f9108ddaa156ac15738e41ed3bedec1eda50175d > > We were using mail_attachment_min_size=4kb, we solve it by increasing > it to 8kb. > > It would be nice to somehow fix this problem. Like not crash when > EMLINK happend and maybe do not deduplicate attachments but deliver > email without dedup. > Or create second file in hashes/ and start hardlinking it instead of > original. > > AFAIK ext4 has also hard-link limit 64k > (http://en.wikipedia.org/wiki/Hard_link#Limitations_of_hard_links) > So this can happen to anyone with lots of emails. > > Thanks-- [ Ohodnotte kvalitu mailu: http://nicereply.com/websupport/Stano/ ] Pavel Stano | Troubleshooter http://WebSupport.sk *** BERTE A VYCHUTNAVAJTE *** -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.2.13-EMLINK.patch Type: text/x-patch Size: 500 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20140605/83a62774/attachment.bin> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20140605/83a62774/attachment.sig>
Maybe Matching Threads
- replication + attachment sis + zlib bug ? (HEAD version from xi.rename-it.nl)
- sis deduplication broken from 2.2.16 upwards
- sis deduplication broken from 2.2.16 upwards
- Remove all messages in a mailbox
- link(2) EMLINK error behavior with --link-dest and --hard-links