dovecot - Jun 2010 - dovecot evaluation on a 30 gb mailbox

hi all

i use qmail toaster. i recently changed over to dovecot.

posting below my evaluation on the same

my machine
dual core dual xeon 5140 processor, with 4 gb ram, centos linux with
qmailtoaster

i had setup a qmail tap account which backups up data for a client of mine

mailbox size over 30 gb

setup dovecot with squirrelmail and logged in

it had over one hundred and thirty nine thousand emails

139019 emails in the 30 gb mailbox

Reports

1) first login : took 7 minutes to parse 139019 mails but never timed out.
this was simply impossible with courier

2) the next login it took 5 seconds by which time the mail numbers
increased to 139045

3) next i changed squirrelmail display preference to list 500 mails per page
and then clicked on inbox ... expecting dovecot to take atleast around 1-2
mins to show the mails

... the mails were displayed in just 8 seconds .. awesome performance

qmailtoaster is simply waiting for a dovecot integration

i am evaluating the stability of dovecot now

in the next 2-3 weeks i post stability report

rajesh

On 20.6.2010, at 3.33, Rajesh M wrote:
> 3) next i changed squirrelmail display preference to list 500 mails per
page
> and then clicked on inbox ... expecting dovecot to take atleast around 1-2
> mins to show the mails
> 
> ... the mails were displayed in just 8 seconds .. awesome performance
Fetching metadata for 500 messages took 8 seconds? Still sounds slow. Although
maybe it's because of maildir. I'm anyway not happy until Dovecot can
open a mailbox with millions of messages and access them in nearly constant time
(few milliseconds, with SSD).

On 6/19/2010 7:33 PM, Rajesh M wrote:
> setup dovecot with squirrelmail and logged in
>    
I'd recommend also installing and configuring imapproxy - it can be 
beneficial with squirrelmail.
-- 
Daniel

> Rajesh M wrote:
>>> Rajesh M wrote:
>>>> eric
>>>>
>>>> i studied LDA a bit
>>>  >
>>>> if i use lda that means all my 5000+ users' email index
files will be
>>>> continuously updated when every email arrives -- means a lot of
writes
>>>> to
>>>> disk ... is that correct ?
>>> Yes, but I think you make it sound worse than it is. Updating the
index
>>> as each email arrives doesn't involve rebuilding the whole
index like
>>> what happens at login. It only updates what's there instead of
>>> rebuilding the whole thing. I can't speak from experience, but
I expect
>>> that it's very efficient, and you'll never notice any
increased load on
>>> the machine. Perhaps the equivalent of adding journaling to a
>>> filesystem. You can ask the dovecot list to verify this.
>>>
>>> Also, it just occurred to me that if you're using ext3, you
should be
>>> sure to optimize the filesystem. Off hand, I'd use the noatime
and
>>> nodiratime mount options (man mount) and dir_index filesystem
option
>>> (man tune2fs).
>>>
>>>> if yes then i dont want that since only around 200 or so people
out of
>>>> the
>>>> above 5000 use webmail.
>>> Just to be clear, it affects all imap usage, not only webmail
(although
>>> your uses may not use imap other than webmail).
>>>
>>>> i will study a some more and revert
>>> I hope you can figure a way to try this out. I expect that
QMT's move
>>> to
>>> dovecot in the future will include the deliver LDA, and your work
here
>>> will be a big help getting us there.
>>>
>>> Let me know if I can be of any further help.
>>>
>>> --
>>> -Eric 'shubes'
>>>
>>
>>
>> hi eric
>>
>> few questions
>>
>> i wil enable lda exclusively for this one 30 gb email box using .qmail
>> file see the performance improvement .. is that ok ?
>
> I believe that would be fine.
>
>> can i safely enable atime for this user maildir  ?
>
> Sure. That's simply a performance consideration. I don't believe
that
> atime is used for much of anything, so either atime or noatime is safe.
> noatime is just a little faster because the inode (where that piece of
> information is stored) doesn't need to be updated each time a file is
> accessed.
>
>> thanks for your help
>
> Sure. I'm eager to see your results.
>
> --
> -Eric 'shubes'
>

hi eric

this is what i plan to do

as per
http://wiki.dovecot.org/LDA/Qmail

i need to enter the following line in a .qmail file under the Maildir
directory of the user
|/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $EXT@$USER

i would add an email forward using qmailadmin web interface (which creates
the .qmail file) and then manually edit the .qmail file and put the above
line in it (so that ownership and permissions are taken care off)

now i would also need to edit the dovecot.conf file
could you guide me on what changes do i need to dovecot.conf file ?


rajesh

On 2010-06-25 9:58 AM, Angelo Chen wrote:
> possible to setup a imap server without smtp? just want to store some old
emails.
Of course, one has nothing to do with the other.

-- 

Best regards,

Charles

Rajesh M wrote:
>> Rajesh M wrote:
>>>> Rajesh M wrote:
>>>>> eric
>>>>>
>>>>> i studied LDA a bit
>>>>  >
>>>>> if i use lda that means all my 5000+ users' email index
files will be
>>>>> continuously updated when every email arrives -- means a
lot of writes
>>>>> to
>>>>> disk ... is that correct ?
>>>> Yes, but I think you make it sound worse than it is. Updating
the index
>>>> as each email arrives doesn't involve rebuilding the whole
index like
>>>> what happens at login. It only updates what's there instead
of
>>>> rebuilding the whole thing. I can't speak from experience,
but I expect
>>>> that it's very efficient, and you'll never notice any
increased load on
>>>> the machine. Perhaps the equivalent of adding journaling to a
>>>> filesystem. You can ask the dovecot list to verify this.
>>>>
>>>> Also, it just occurred to me that if you're using ext3, you
should be
>>>> sure to optimize the filesystem. Off hand, I'd use the
noatime and
>>>> nodiratime mount options (man mount) and dir_index filesystem
option
>>>> (man tune2fs).
>>>>
>>>>> if yes then i dont want that since only around 200 or so
people out of
>>>>> the
>>>>> above 5000 use webmail.
>>>> Just to be clear, it affects all imap usage, not only webmail
(although
>>>> your uses may not use imap other than webmail).
>>>>
>>>>> i will study a some more and revert
>>>> I hope you can figure a way to try this out. I expect that
QMT's move
>>>> to
>>>> dovecot in the future will include the deliver LDA, and your
work here
>>>> will be a big help getting us there.
>>>>
>>>> Let me know if I can be of any further help.
>>>>
>>>> --
>>>> -Eric 'shubes'
>>>>
>>>
>>> hi eric
>>>
>>> few questions
>>>
>>> i wil enable lda exclusively for this one 30 gb email box using
.qmail
>>> file see the performance improvement .. is that ok ?
>> I believe that would be fine.
>>
>>> can i safely enable atime for this user maildir  ?
>> Sure. That's simply a performance consideration. I don't
believe that
>> atime is used for much of anything, so either atime or noatime is safe.
>> noatime is just a little faster because the inode (where that piece of
>> information is stored) doesn't need to be updated each time a file
is
>> accessed.
>>
>>> thanks for your help
>> Sure. I'm eager to see your results.
>>
>> --
>> -Eric 'shubes'
>>
> 
> 
> hi eric
> 
> this is what i plan to do
> 
> as per
> http://wiki.dovecot.org/LDA/Qmail
> 
> i need to enter the following line in a .qmail file under the Maildir
> directory of the user
> |/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $EXT@$USER
the location is /usr/libexec/dovecot/deliver on my system.
> i would add an email forward using qmailadmin web interface (which creates
> the .qmail file) and then manually edit the .qmail file and put the above
> line in it (so that ownership and permissions are taken care off)
That sounds ok. The permissions/ownership I'm seeing on .qmail files is:
-rw-------   1 vpopmail vchkpw   101 Mar 26  2008 .qmail
> now i would also need to edit the dovecot.conf file
> could you guide me on what changes do i need to dovecot.conf file ?
Others on this list would be more help than me regarding this. From the 
looks of it though (after browsing the dovecot.conf file, which is well 
documented btw) I don't think you'd need to change a thing. We'll 
certainly find out if that's the case or not. I'd just give it a go. 
Perhaps on a test account initially. ;)

-- 
-Eric 'shubes'