Hello We are using Dovecot 2.0.17 with mdbox+zlib and we are evaluating alternative storage. These are the relevant settings mdbox_rotate_interval = 1d mdbox_rotate_size=60m [...] zlib_save_level = 9 # 1..9 zlib_save = gz # or bz2 [...] zlib plugin enabled in dovecot-lda These are my questions: 1? General mdbox question. We are happy usings zlib_save, but we would like to try bzip2 compression. Can we change zlib to bzlib without problems? In other words, can we mix & match gzipped and bzipped messages in the same mdbox file ? 2? With doveadm altmove we can move messages matching a pattern to alternate storage. I have read an older post in this mailing list about passing specific mdbox options to doveadm altmove. Should be a good idea to have a "main" mdbox area with zlib 9 and mdbox_rotate=1d and mdox_rotate_size=60m and an "alternate area" with bzip9, mdbox_rotate=1 week and rotate_size > 60 MB? In the alternate area should only be "old" mail (1 year older). 3? We have separate raids for indexes (raid 1+0) and mdboxes (raid 5). We have almost 3 TB of gzipped mdboxes, and our backup (bacula) is taking almost 16 hours to make a full backup. Our main problem is that indexes are backupd hours before mdboxes, and that bring us some inconsistencies. What is the "right" way of backing up for separate indexes and mdboxes? Thank you for your support. Maria
Maria Arrea <maria_arrea at gmx.com> wrote:> 3? We have separate raids for indexes (raid 1+0) and mdboxes (raid 5). > We have almost 3 TB of gzipped mdboxes, and our backup (bacula) is > taking almost 16 hours to make a full backup. Our main problem is that > indexes are backupd hours before mdboxes, and that bring us some > inconsistencies. What is the "right" way of backing up for separate > indexes and mdboxes?Bacula? Excellent! If you put your indexes and mdboxes on a LVM (seperate volume group obviously, because of the different storages both reside on), you can snapshot both volumes at the same time, using an FD-based script, mount them somewhere else (like /backup/data and /backup/index) and then you have (nearly) all the time you want to backup them. And because of the snapshots taken at the same time, no inconsistencies can happen. (OK, not entirely true, since both snapshots are taken with a minimal gap between them, but if you start the backup during off-hours the inconsistencies should be minimal.) If you don't use a LVM but have a storage which is able to do snapshots on its own, you can also use this. If you don't have a LVM and no storage with such functionality, then you are kind of lost, as far as I can see. My additional advise on the long time of full backups: don't do them! Switch to accurate backups and only do incremental and differential backups, replacing the full backups with virtual full ones. This will greatly reduce the load on your mail systems and speed up the backup operation at the same time. But you have to use "accurate" type backups, or your virtual full backup will grow and grow and grow, since Bacula without "accurate" is unable to know about deleted files and will include them in every virtual full backup. Gr??e, Sven. -- Sigmentation fault. Core dumped.
On Mon, 2012-02-27 at 01:07 +0100, Maria Arrea wrote:> 1? General mdbox question. We are happy usings zlib_save, but we > would like to try bzip2 compression. Can we change zlib to bzlib > without problems? In other words, can we mix & match gzipped and > bzipped messages in the same mdbox file ?Yes.> 2? With doveadm altmove we can move messages matching a pattern to > alternate storage. I have read an older post in this mailing list > about passing specific mdbox options to doveadm altmove. Should be a > good idea to have a "main" mdbox area with zlib 9 and mdbox_rotate=1d > and mdox_rotate_size=60m and an "alternate area" with bzip9, > mdbox_rotate=1 week and rotate_size > 60 MB? In the alternate area > should only be "old" mail (1 year older).I guess it makes sense.> 3? We have separate raids for indexes (raid 1+0) and mdboxes (raid > 5). We have almost 3 TB of gzipped mdboxes, and our backup (bacula) is > taking almost 16 hours to make a full backup. Our main problem is that > indexes are backupd hours before mdboxes, and that bring us some > inconsistencies. What is the "right" way of backing up for separate > indexes and mdboxes?The eventual solution is a Dovecot backup agent, but for now I guess there isn't anything better than taking filesystem snapshots.