Tom Pawlowski
2012-May-16 21:20 UTC
[Dovecot] Error in dovecot 2.0.13: "Too many levels of symbolic links"
Hi all, I've run into an issue we have with dovecot versions 2.0.12 through 2.0.14 on latest CentOS 6.2 when dealing with Maildirs located on NFS partitions. Although I saw Mr. Egbert's thread, his resolution does not work on our end. When dealing with users with over 500 messages in any of their Maildir folders, they occasionally receive an error from the server: Internal error occurred. Refer to server log for more information. The corresponding log entries: dovecot: imap(user1): Error: readdir(/u1/user1/Maildir/.AUTO-DELETED-SPAM/new) failed: Too many levels of symbolic links dovecot: imap(user1): Error: readdir(/u1/user1/Maildir/.logwatch/new) failed: Too many levels of symbolic links dovecot: imap(user2): Error: readdir(/u2/user2/Maildir/cur) failed: Too many levels of symbolic links dovecot: imap(user2): Error: readdir(/u2/user2/Maildir/.Trash/new) failed: Too many levels of symbolic links dovecot: imap(user3): Error: readdir(/u1/user3/Maildir/.Trash/new) failed: Too many levels of symbolic links dovecot: imap(user3): Error: readdir(/u4/user3/Maildir/.AUTO-DELETED-SPAM/new) failed: Too many levels of symbolic links dovecot: imap(user4): Error: readdir(/u1/user4/Maildir/.AUTO-DELETED-SPAM/new) failed: Too many levels of symbolic links dovecot: imap(user4): Error: readdir(/u1/user4/Maildir/new) failed: Too many levels of symbolic links A 'find . -type l' shows there are absolutely no symbolic links to be found in those directories. Running readdir.c that Timo provided always returns errno == 0 on {cur,new} of all the directories reported in the logs. It does not occur consistently in production, but it does tend to occur to a fuzzy set of the same users on different partitions. We can replicate it every time in testing. The file count in these directories are all over the place, from less than a thousand to multiple thousands. However, it's hard to calculate what may have been in new prior to the clients processing/sorting that mail. It may have been more on the higher side than lower. The error tends to favor new over cur, but there are exceptions. The tests we have been trying: Create a new folder using alpine, save anywhere from 1000 to 10000 identical emails to the new folder. Change to that new folder, thus prompting the move from new to cur. The client receives the message and aborts the opening. Anywhere from 400-520 files are successfully moved each time. It takes numerous attempts to finally move all the files over--which I suspect some IMAP clients do, hence the inconsistent errors in production. We then create a brand new folder with the same amount of files and verify that the error is reproduceable--which it is. I'm leaning towards some sort of bug on the NFS client level, as Timo mentioned, but the odd thing is we were running this setup since early April and did not begin seeing these errors until early May--with user load being relatively consistent during that time. I would think the bug would have been evident before then. I'm also curious as to why dovecot is getting an ELOOP error code from it. Any ideas or pointers would be appreciated, sorry for the length. :) -- Tom Pawlowski OIT-CSS System Administrator office: Hill 145 email: tompru at jla.rutgers.edu phone: (732) 445-2634
Timo Sirainen
2012-May-16 21:35 UTC
[Dovecot] Error in dovecot 2.0.13: "Too many levels of symbolic links"
On 17.5.2012, at 0.20, Tom Pawlowski wrote:> dovecot: imap(user1): Error: readdir(/u1/user1/Maildir/.AUTO-DELETED-SPAM/new) failed: Too many levels of symbolic links..> Create a new folder using alpine, save anywhere from 1000 to 10000 identical > emails to the new folder. > > Change to that new folder, thus prompting the move from new to cur. The client > receives the message and aborts the opening. Anywhere from 400-520 files are > successfully moved each time. It takes numerous attempts to finally move > all the files over--which I suspect some IMAP clients do, hence the > inconsistent errors in production.See if the attached patch makes any difference?> I'm leaning towards some sort of bug on the NFS client level, as TimoWhat NFS server are you using? Perhaps it's related to that. -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 486 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20120517/f092e615/attachment-0004.obj>
Tom Pawlowski
2012-May-17 16:04 UTC
[Dovecot] Error in dovecot 2.0.13: "Too many levels of symbolic links"
Well, the good news is this didn't end up being a dovecot or NetApp issue. RHEL 6.2 has a kernel bug: https://bugzilla.redhat.com/show_bug.cgi?id=770250 Solution is to upgrade to the latest kernel released on 5/15: kernel-2.6.32-220.17.1.el6 We no longer get this error after the update. Thanks for taking a look! * Timo Sirainen <tss at iki.fi> [2012-05-17 00:35:49 +0300]:> On 17.5.2012, at 0.20, Tom Pawlowski wrote: > > > dovecot: imap(user1): Error: readdir(/u1/user1/Maildir/.AUTO-DELETED-SPAM/new) failed: Too many levels of symbolic links > .. > > Create a new folder using alpine, save anywhere from 1000 to 10000 identical > > emails to the new folder. > > > > Change to that new folder, thus prompting the move from new to cur. The client > > receives the message and aborts the opening. Anywhere from 400-520 files are > > successfully moved each time. It takes numerous attempts to finally move > > all the files over--which I suspect some IMAP clients do, hence the > > inconsistent errors in production. > > > See if the attached patch makes any difference? > > > I'm leaning towards some sort of bug on the NFS client level, as Timo > > What NFS server are you using? Perhaps it's related to that. >-- Tom Pawlowski OIT-CSS System Administrator office: Hill 145 email: tompru at jla.rutgers.edu phone: (732) 445-2634