I've just enabled zlib for our users, and am looking at how to compress
the existing files. The routine for doing this at
http://wiki2.dovecot.org/Plugins/Zlib seems a bit complicated. What do
you think about simply doing:
find /var/vmail -type f -name "*,S=*" -mtime +1 -exec gzip -S Z -6
'{}' +
I.e. find all maildir-files:
- with size in the name ("*,S=*")
- modified before I enabled zlib plugin
- compress them
- add the Z suffix
- keep timestamps (gzip does that by default)
It's of course racy without the maildirlock, but are there any other
problems with this approach ?
-jf
The cleanest (though not necessarily simplest) way to go about this would be to use dsync to create a new maildir and incrementally direct traffic to a separate Dovecot instance. Unless you have a legacy application that relies on maildir, switching to mdbox would be a good idea too. I expect that with Dovecot compression is something that can "just be turned on", but for fear of any possible issue, I chose to migrate mailboxes in batches with the way mentioned above. On Dec 24, 2011, at 7:20 AM, Jan-Frode Myklebust wrote:> I've just enabled zlib for our users, and am looking at how to compress > the existing files. The routine for doing this at > http://wiki2.dovecot.org/Plugins/Zlib seems a bit complicated. What do > you think about simply doing: > > find /var/vmail -type f -name "*,S=*" -mtime +1 -exec gzip -S Z -6 '{}' + > > > I.e. find all maildir-files: > > - with size in the name ("*,S=*") > - modified before I enabled zlib plugin > - compress them > - add the Z suffix > - keep timestamps (gzip does that by default) > > > It's of course racy without the maildirlock, but are there any other > problems with this approach ? > > > -jf
On Wed, Dec 28, 2011 at 03:56:33PM -0800, Dovecot-GDH wrote:> The cleanest (though not necessarily simplest) way to go about this would be to use dsync to create a new maildir and incrementally direct traffic to a separate Dovecot instance. > > Unless you have a legacy application that relies on maildir, switching to mdbox would be a good idea too.We just got rid of the legacy app that worked directly against the maildirs, which is the reason we now can turn on compression. I intend to switch to mdbox, but first I need to free up some disks by compressing the existing maildirs (12 TB maildirs, should probably compress down to less than half).> > I expect that with Dovecot compression is something that can "just be turned on", but for fear of any possible issue, I chose to migrate mailboxes in batches with the way mentioned above. >Migrating to mdbox is much scarier to me than an easily reversible compression of existing maildir files. Could you please give a bit more details about how you did this migration? Did you change user home dirctory in the process? Seeing the scripts you used to run the migration would be very interesting.. -jf
On 24.12.2011, at 17.20, Jan-Frode Myklebust wrote:> I've just enabled zlib for our users, and am looking at how to compress > the existing files. The routine for doing this at > http://wiki2.dovecot.org/Plugins/Zlib seems a bit complicated. What do > you think about simply doing: > > find /var/vmail -type f -name "*,S=*" -mtime +1 -exec gzip -S Z -6 '{}' + > > > I.e. find all maildir-files: > > - with size in the name ("*,S=*") > - modified before I enabled zlib pluginAs long as it doesn't find any already compressed mails..> - compress them > - add the Z suffixMake sure there's also :2, suffix already. If someone hasn't logged in for a while there are such files in new/ directory.> It's of course racy without the maildirlock, but are there any other > problems with this approach ?Other than being racy, I guess it should work.