dovecot - Mar 2012 - dsync is SLOW compared to rsync

Hi all,
        We are currently using snapshots and rsync to backup a large
mail server to a backup mail server. I have been looking into using
dsync to replace rsync in hopes that it would make backups more
efficient. I decided to test the performance using a single mailbox.
Unfortunately dsync seems to run much slower than rsync. Rsync was able
to sync the mailbox in 2 seconds. dsync took over a minute. The test was
run so that the source and destination are on the same filesystem. 
        We would like to using the new replication system, but that
doesn't seem likely since the performance of the underlying dsync is so
much slower than rsync. Even with the extra work that dsync is doing I
can't believe the difference in performance would be that great. I
realize that dsync is actively being worked on and I hope bringing
attention to performance issue will provoke some ideas on how to improve
it.
        Here is the output of the tests using dovecot 2.1.3:


[root at n24 bu]# du -hs /home/10.0.1.101/1009/users/testuser%
domain.com/Maildir/

517M    /home/10.0.1.101/1009/users/testuser%domain.com/Maildir/

[root at n24 bu]# time rsync -va /home/10.0.1.101/1009/users/testuser%
domain.com/Maildir/ .
sending incremental file list
Maildir/
Maildir/dovecot-uidlist
[ ... deleted cruft ... ]
Maildir/cur/1332387577.M381054P27635.n24,S=14215502,W=14448554:2,
Maildir/new/
Maildir/tmp/

sent 540927820 bytes  received 1222 bytes  216371616.80 bytes/sec
total size is 540855755  speedup is 1.00

real    0m2.677s
user    0m3.184s
sys     0m1.513s

[root at n24 bu]# time dsync backup -u testuser at domain.com \
mdbox:/home/bu/testuser

real    1m9.519s
user    1m7.592s
sys     0m1.126s
[root at n24 bu]# time dsync backup -u testuser at domain.com \
sdbox:/home/bu/testuser2

real    1m2.164s
user    1m0.882s
sys     0m0.993s
[root at n24 bu]#

Hi,

maybe try "dsync -o mail_fsync=never".

Cheers,
Christoph

-- 
Christoph Bu?enius
Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik
Technische Universit?t M?nchen
+49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <>
Garching

On 22.3.2012, at 23.25, Jeff Gustafson wrote:
> [root at n24 bu]# time dsync backup -u testuser at domain.com \
> mdbox:/home/bu/testuser
> 
> real    1m9.519s
> user    1m7.592s
> sys     0m1.126s
Most of the time is spent on usermode CPU code. I doubt the problem is dsync
itself, most likely the problem is mdbox's saving code. Or possibly
index/cache code. Try the same dsync backup for:

 - mbox:/tmp/mbox
 - mbox:/tmp/mbox:INDEX=MEMORY
 - sdbox:/tmp/sdbox