thr3ads.net - dovecot - [Dovecot] Benchmarking [Jan 2007]

If this information is useful, please help other people find it:
Share via:

Timo Sirainen

2007-Jan-25 20:25 UTC

[Dovecot] Benchmarking

I wasted too much time today doing this, but they do show a few
interesting things:

1) mmap_disable=yes is faster than mmap_disable=no. I didn't expect
this. I'm not yet sure why this is, but possibly because then it updates
the mail index by reading the changes from dovecot.index.log, instead of
reopening and re-mmaping dovecot.index. At least that's the biggest
difference between them.

2) Dovecot is faster than Cyrus, at least in test. I didn't really
expect this either, because with maildir Dovecot has to do extra
rename() calls for all flag changes. I'm guessing it's expunges that are
slowing Cyrus down the most (that causes it to rewrite its cache file).

3) fsync_disable=yes + mmap_disable=yes almost doubles the performance
with 1 client, and over quadruples the performance with 10 clients. Why
fsync_disable=yes + mmap_disable=no doesn't give that big of a
performance difference? I've no idea. This is anyway such a large
performance increment that I'll probably go and backport this setting to
v1.0 too :)

4) Dovecot's mbox code kicks maildir's ass.

5) dbox is surprisingly slow. I'll have to find out why. I don't think
it should be that slow.

6) Courier.. Well, it probably doesn't come as a shock that Dovecot is
almost twice as fast. Note that Courier doesn't call fsync(), so it
can't be compared to Dovecot's fsync_disable=no numbers or to Cyrus. Oh,
one other difference between Courier and others was that it didn't
support LITERAL+ extension, so it maybe could have done a bit less
appends than others.

7) CVS HEAD and branch_1_0 don't seem to have that big of a performance
difference.


All the tests were done using my IMAP tester:
http://dovecot.org/tools/imaptest.c

The mailbox data that was being appended was Dovecot's mailing list:
http://dovecot.org/archives/dovecot.mbox

The linefeeds were converted to CR+LF. However, for the Dovecot CVS HEAD
runs the mails contained also those extra X-UID, Status, etc. headers
which I had to remove before Cyrus would accept them (the lines didn't
contain CRs). So CVS HEAD was actually doing a bit more work than the
others.

System:

Intel Core 2 Duo E6600, 2GB memory, 300GB Seagate ATA disk with XFS
Linux 2.6.19.2, x86-64 SMP, Debian unstable

Dovecot: CVS HEAD unless specified otherwise
Cyrus: v2.2.13-10 (Debian) - NOTE: comparable to fsync_disable:no
Courier: v4.1.1.20060828-2 (Debian) - NOTE: comparable to fsync_disable:yes!

Settings:

Dovecot: login_process_per_connection=no, login_processes_count=10,
  mail_drop_priv_before_exec=yes, passdb+userdb = passwd-file
Cyrus: Default Debian settings, except sasl_pwcheck_method: alwaystrue
Courier: Default Debian settings, using PAM + passwd

Results:

imaptest parameters: clients=1 msgs=30 secs=120 seed=1

mmap_disable:no fsync_disable:no
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
1135 1135 1134 1134  577  298  298  378 1134
1125 1125 1124 1124  573  295  295  373 1124
1123 1123 1122 1122  572  294  294  371 1122

mmap_disable:yes fsync_disable:no
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
1401 1401 1400 1400  710  388  388  480 1400
1398 1398 1397 1397  710  386  385  478 1397
1395 1395 1394 1394  709  384  384  476 1394

mmap_disable:no fsync_disable:yes
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
1356 1356 1355 1355  687  369  369  460 1355
1360 1360 1359 1359  690  371  370  462 1359
1363 1363 1362 1362  693  372  372  463 1362

mmap_disable:yes fsync_disable:yes
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
2486 2486 2485 2485 1221  669  669  811 2485
2475 2475 2474 2474 1216  667  667  807 2474
2497 2497 2496 2496 1226  673  673  815 2496

mmap_disable:yes fsync_disable:no - VERSION: CVS branch_1_0
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
1359 1359 1358 1358  690  370  370  461 1358 

mmap_disable:yes fsync_disable:no format:mbox
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
2101 2101 2100 2100 1046  568  568  684 2100 
2064 2064 2063 2063 1031  553  552  669 2063 

mmap_disable:yes fsync_disable:yes format:mbox
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
3362 3362 3361 3361 1653  912  911 1095 3361 

mmap_disable:yes fsync_disable:yes format:dbox
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
 820  820  819  819  406  209  209  267  819 

Cyrus:
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
 535  535  533  533  261  148  148  195  534 
 520  520  519  519  254  142  142  189  520 

Courier:
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
 833  833  832  831  407  250  250  350  832 
 822  822  821  820  404  245  245  344  821 

---

imaptest parameters: clients=1 msgs=1000 secs=120 seed=1

mmap_disable:no fsync_disable:no
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
 365  365  364  364  181  226  226  487  364
 376  376  375  375  186  236  236  503  375
 369  369  368  368  183  230  230  493  368

mmap_disable:yes fsync_disable:no
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
 426  426  425  425  212  269  269  564  425
 428  428  427  427  213  271  271  567  427
 424  424  423  423  211  267  266  562  423

mmap_disable:no fsync_disable:yes
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
 497  497  496  496  240  320  320  664  496
 501  501  500  500  243  322  322  671  500
 495  495  494  494  240  318  317  662  494

mmap_disable:yes fsync_disable:yes
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
 742  742  741  741  378  500  500  991  741
 747  747  746  746  379  504  504  996  746
 743  743  742  742  378  501  501  992  742

mmap_disable:yes fsync_disable:no format:mbox
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
 485  485  484  484  235  311  311  649  484 

Cyrus:
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
 290  290  289  289  144  173  173  394  289 
 286  286  285  285  141  172  172  387  285 

Courier:
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
 461  461  460  459  223  295  295  615  460 
 458  458  457  457  222  293  293  611  457 

---

imaptest parameters: clients=10 msgs=30 secs=120 seed=1

mmap_disable:no fsync_disable:no
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc
 444  442  432  430  210  214  213  490  434
 479  479  470  468  213  216  214  508  469
 476  472  464  463  211  206  203  485  460

mmap_disable:yes fsync_disable:no
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc
 440  440  433  431  195  184  183  460  430
 436  436  430  430  199  177  176  455  426
 435  432  425  425  208  165  165  455  425

mmap_disable:no fsync_disable:yes
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc
 680  679  674  670  325  299  298  721  670
 676  675  669  669  307  297  297  693  666
 688  687  677  677  311  305  303  731  678

mmap_disable:yes fsync_disable:yes
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
1804 1803 1793 1790  904  765  764 1901 1792 
1804 1804 1801 1800  888  789  787 1949 1793 
1800 1800 1789 1786  875  799  797 1964 1782 

mmap_disable:yes fsync_disable:no format:mbox
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
 644  644  633  633  305  224  221  526  634 

Cyrus:
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
 444  441  429  421  209  186  184  381  436 
 424  422  420  412  193  186  185  395  415 
 443  442  431  419  227  180  180  373  433 

Courier:
Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
 981  976  974  973  463  406  405  667  971 
 971  966  965  965  451  384  383  655  961 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20070125/e38618a6/attachment.bin>

Timo Sirainen

2007-Jan-25 20:36 UTC

head link

[Dovecot] Benchmarking

On Thu, 2007-01-25 at 22:25 +0200, Timo Sirainen wrote:> Logi Sele Fetc Fet2 Stor Dele Expu Appe Disc 
> 1135 1135 1134 1134  577  298  298  378 1134
Oh, and about these numbers. Number of logins (Logi) and number of
appends (Appe) are the most important ones I think. All the numbers mean
number of IMAP commands sent not the number of messages affected. For
example "Expu" means number of EXPUNGE commands sent, not the actual
number of messages expunged.

The benchmark does these commands:

LOGIN user pass
SELECT INBOX
FETCH 1:* (UID FLAGS ENVELOPE BODY)
FETCH one-random-message (BODY[])
STORE random-messageset FLAGS.SILENT (random-flags and
random-keywords-from-5-different)
STORE random-messageset +FLAGS.SILENT \Deleted
EXPUNGE
APPEND new message, unless we're over threshold
 - randomly APPEND more messages
LOGOUT

So the msgs-parameter to imaptest means how many messages it tries to
keep in the mailbox. The msgs=30 works pretty well, but msgs=1000 was
never really reached in the 120 second tests. It was more like 100-150
messages where it was kept. I guess I should try the msgs=1000 with
longer test runs.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20070125/d8acbce8/attachment.bin>

Farkas Levente

2007-Jan-25 21:06 UTC

head link

[Dovecot] Benchmarking

Timo Sirainen wrote:> 4) Dovecot's mbox code kicks maildir's ass.
> 
> 5) dbox is surprisingly slow. I'll have to find out why. I don't
think
> it should be that slow.
does this means you suggest to use mbox in stead of maildir? and dbox is
far from usable?

-- 
  Levente                               "Si vis pacem para bellum!"

Charles Marcus

2007-Jan-25 21:10 UTC

head link

[Dovecot] Benchmarking

Timo Sirainen wrote:> I wasted too much time today doing this, but they do show a few
> interesting things:
Hi Timo...

Thanks for this info!

A couple of (probably dumb) questions ...
> 3) fsync_disable=yes + mmap_disable=yes almost doubles the
> performance with 1 client, and over quadruples the performance with
> 10 clients. Why fsync_disable=yes + mmap_disable=no doesn't give that
> big of a performance difference? I've no idea. This is anyway such a
> large performance increment that I'll probably go and backport this
> setting to v1.0 too :)
In other words, these will be the new default settings? And I guess you 
also now recommend using these settings in existing setups, unless there 
is a platform/fs specific reason not to?
> 4) Dovecot's mbox code kicks maildir's ass.
Hmmm... trying to understand the significance of this. I always thought 
maildir was better=faster. I can still readily see how maildir is more 
reliable (one message getting corrupted means you only lose that one 
message), but again, I always thought it was faster too.
> 5) dbox is surprisingly slow. I'll have to find out why. I don't
think
> it should be that slow.
:( from what I read about dbox, it looks very promising. Well, since it 
is still very new, I guess it isn't surprising. I'm sure you'll
figure
out why and soon it'll be kicking mbox's ass.

-- 

Best regards,

Charles

Axel Thimm

2007-Jan-26 20:19 UTC

head link

[Dovecot] Benchmarking

On Thu, Jan 25, 2007 at 10:25:03PM +0200, Timo Sirainen
wrote:> 4) Dovecot's mbox code kicks maildir's ass.
Perhaps this is just a filesystem issue, XFS is known for handling
well large streaming-like data and is less optimized for many small
files. Maybe reiserfs would give you the opposite relationship.
> Intel Core 2 Duo E6600, 2GB memory, 300GB Seagate ATA disk with XFS
> Linux 2.6.19.2, x86-64 SMP, Debian unstable-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20070126/77ac3bd3/attachment.bin>

Reasonably Related Threads

Search for more reasonably related threads

dovecot - Jan 2007 - Benchmarking

[Dovecot] Benchmarking

[Dovecot] Benchmarking

[Dovecot] Benchmarking

[Dovecot] Benchmarking

[Dovecot] Benchmarking

Reasonably Related Threads