dovecot - Dec 2007 - maildirsize quota counting locking account?

Hi,

I've just straced a dovecot imap process and found one thread sitting 
there doing hundreds and hundreds of maildirsize updates as follows:
> 
> open("/home/virtual/mydomain.com/home/admin/Maildir/maildirsize",
O_RDWR|O_APPEND) = 7
> read(7, "10485760000S\n1562717408 107547\n-"..., 5120) = 2446
> read(7, "", 2674)                       = 0
> write(1, "* 1810 FETCH (UID 1815 RFC822.SI"..., 2031) = 2031
> setsockopt(1, SOL_TCP, TCP_CORK, [0], 4) = 0
> epoll_wait(4, {{EPOLLIN, {u32=101056896, u64=2370923004288}}}, 5, 2584) = 1
> read(0, "2838 UID fetch 1816 (UID RFC822."..., 2974) = 51
> setsockopt(1, SOL_TCP, TCP_CORK, [1], 4) = 0
> close(7)                                = 0
> open("/home/virtual/mydomain.com/home/admin/Maildir/maildirsize",
O_RDWR|O_APPEND) = 7
> read(7, "10485760000S\n1562717408 107547\n-"..., 5120) = 2446
> read(7, "", 2674)                       = 0
>
open("/home/virtual/mydomain.com/home/admin/Maildir/.Folders.ALERTS.Payments/cur/1195331425.M52296P11937V0000000000000807I000000001015A5EB_0.tg3,S=2079:2,Sa",
O_RDONLY) = 10
> fstat(10, {st_mode=S_IFREG|0600, st_size=2079, ...}) = 0
> pread(10, "Return-Path: <apache at tg7.alumina"..., 4096, 0) =
2079
> pread(10, "", 2017, 2079)               = 0
> pread(10, "Return-Path: <apache at tg7.alumina"..., 4096, 0) =
2079
> write(1, "* 1811 FETCH (UID 1816 RFC822.SI"..., 2180) = 2180
> close(10)                               = 0
>
stat("/home/virtual/mydomain.com/home/admin/Maildir/.Folders.ALERTS.Payments/new",
{st_mode=S_IFDIR|0700, st_size=6, ...}) = 0
>
stat("/home/virtual/mydomain.com/home/admin/Maildir/.Folders.ALERTS.Payments/cur",
{st_mode=S_IFDIR|0700, st_size=618496, ...}) = 0
> close(7)                                = 0
> open("/home/virtual/mydomain.com/home/admin/Maildir/maildirsize",
O_RDWR|O_APPEND) = 7
> read(7, "10485760000S\n1562717408 107547\n-"..., 5120) = 2446
> read(7, "", 2674)                       = 0
> write(1, "2838 OK Fetch completed.\r\n", 26) = 26
> setsockopt(1, SOL_TCP, TCP_CORK, [0], 4) = 0
> epoll_wait(4, {}, 5, 800)               = 0
> epoll_wait(4, {{EPOLLIN, {u32=101056896, u64=2370923004288}}}, 5, 5000) = 1
> read(0, "2839 UID fetch 1817 (UID RFC822."..., 2923) = 51
> setsockopt(1, SOL_TCP, TCP_CORK, [1], 4) = 0
> close(7)                                = 0
> open("/home/virtual/mydomain.com/home/admin/Maildir/maildirsize",
O_RDWR|O_APPEND) = 7
During this time (at least 5-10 minutes) , I could not check the account 
via my email client Thunderbird. Clicking on an email would just sit 
there as if it is loading but nothing would happen - the read pane just 
remains blank.

Is Dovecot going through every email (of many 10,000's) and counting up 
the sizes? Could this be made more efficient so it doesn't lock the account?

This could I guess be a Thunderbird issue - it is a disappointingly 
imperfect client.

Dan

On 19.12.2007, at 13.42, Daniel Watts wrote:
> I've just straced a dovecot imap process and found one thread  
> sitting there doing hundreds and hundreds of maildirsize updates as  
> follows:
What Dovecot version? v1.0.8 fixed one bug that caused this.

Also it's of course quite pointless to even read maildirsize for each  
FETCH. This has been fixed in v1.1..
>> read(0, "2838 UID fetch 1816 (UID RFC822."..., 2974) = 51
..
>> read(0, "2839 UID fetch 1817 (UID RFC822."..., 2923) = 51
This shows that Thunderbird is sending a separate FETCH for each  
message for some reason. Even without maildirsize updates it might be  
slow.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20071219/6b3788b9/attachment-0002.bin>

Timo Sirainen wrote:
> On 19.12.2007, at 13.42, Daniel Watts wrote:
> 
>> I've just straced a dovecot imap process and found one thread
sitting
>> there doing hundreds and hundreds of maildirsize updates as follows:
> 
> What Dovecot version? v1.0.8 fixed one bug that caused this.
1.0.7 - will update as soon as gentoo's portage includes 1.0.8..
hmmm though it seems to have skipped it and gone to 1.1 beta:

*  net-mail/dovecot
       Latest version available: 1.1_beta11
       Latest version installed: 1.0.7
       Size of files: 2,594 kB
       Homepage:      http://www.dovecot.org/
       Description:   An IMAP and POP3 server written with security 
primarily in mind
       License:       LGPL-2.1

Do you do the ebuilds yourself for gentoo? I think you should update the 
Description - makes it seem like dovecot is just about security and not 
performance when that isn't the case!
> Also it's of course quite pointless to even read maildirsize for each 
> FETCH. This has been fixed in v1.1..
> 
>>> read(0, "2838 UID fetch 1816 (UID RFC822."..., 2974) = 51
> ..
>>> read(0, "2839 UID fetch 1817 (UID RFC822."..., 2923) = 51
> 
> This shows that Thunderbird is sending a separate FETCH for each message 
> for some reason. Even without maildirsize updates it might be slow.
> 
Yeah no idea what thunderbird thinks it is doing. Often have to close 
the program and restart it (of course this is nothing to do with dovecot 
- i suspect tb's caching management).

Thanks timo,
Dan