dovecot - Dec 2006 - dbox support state

hi,
what is the current state of dovecot dbox support? is it stable, fast
and usable? is it have current advantage over maildir or it will only
have when it'll be optimized? is it worth to choose as a default mailbox
format in a new dovecot install?
thanks.
yours.

-- 
  Levente                               "Si vis pacem para bellum!"

On Wed, 2006-12-13 at 00:52 +0100, Farkas Levente wrote:
> hi,
> what is the current state of dovecot dbox support? is it stable, fast
> and usable? is it have current advantage over maildir or it will only
> have when it'll be optimized? is it worth to choose as a default
mailbox
> format in a new dovecot install?
I did pretty heavy stress testing for it some months ago, but I haven't
tried since then if it's still working. I'm planning on moving my own
mails to it when I have enough free time (should first figure out how to
make procmail use Dovecot LDA).

I think it should be pretty fast already. With my imaptest client that
kept fetching, appending and expunging messages using 5 connections it
was a lot faster than maildir. Actually maildir really seems to suck
when doing that kind of operations, even mbox is much faster with it..

One optimization left for it would be to not store flags and keywords to
the dbox files at all, but keep everything in index files. Once I get
that implemented, I'll start benchmarking it. Of course the problem with
that is that it relies on index files completely then.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20061221/3e193306/attachment.bin>

On Thu, December 21, 2006 11:24 am, Timo Sirainen <tss at iki.fi>
said:
> 
> One optimization left for it would be to not store flags and keywords to
> the dbox files at all, but keep everything in index files. Once I get
> that implemented, I'll start benchmarking it. Of course the problem
with
> that is that it relies on index files completely then.
> 
With dbox, have you considered having one collection of dbox files per
namespace, rather than one collection of dbox files per mailbox/folder, and then
use indexes to track which emails belong to each mailbox/folder?  This would
drastically speed up copying between folders, because you'd just update the
index rather than writing the copied email to a new dbox file.

Just a thought.

Bill

On Fri, December 22, 2006 2:14 am, Steffen Kaiser <skdovecot at
smail.inf.fh-bonn-rhein-sieg.de> said:
> On Thu, 21 Dec 2006, Timo Sirainen wrote:
> 
>> One optimization left for it would be to not store flags and keywords
to
>> the dbox files at all, but keep everything in index files. Once I get
>> that implemented, I'll start benchmarking it. Of course the problem
with
>> that is that it relies on index files completely then.
> 
> IMHO this is not good for disaster recovery and makes life needlessly hard
> in such case. Esp. when both files become out-of-sync, you need a cunning
> tool to re-construct any good state, rather than a simple
"REINDEX". This
> should apply to each kind of "index" file. Also, consider
upgrades to
> newer index file formats etc.pp.
> 
> Are there really that many writes of message attributes? I mean, there is
> only an improvement because you have just one write into the indexes
> rather than two, one to the indexes and one into the message.
> You've wrote that you reserve space for the flags in mbox, in case they
> get updated, so you reduce the probability to rewrite the whole mbox for
> each flag. Doing so should apply to dbox as well?
It should be an option.  Big mail systems can benefit from this optimization,
but many systems may not see anything noticeable.

You can do a lot of cool things with your storage once you're dealing with
write-once mail data, where all actions after receiving a message get written
somewhere outside of where the mail content is stored... including expunges. 
This is a step in that direction.

Bill