dovecot - Sep 2009 - compression script for use with zlib module

Hi,

    I'm wondering if someone already implemented a compression script 
(and would like to share it) based on the step-by-step provided on wiki 
page of Zlib module.

    I can already do some find|xargs gzip .... but couldnt implement it 
completly as described on the wiki page. I know the suggested 
step-by-step is very detailed and probably a must simplier will be 
enough for almost anyone .... anyway, if someone have the full script 
and would like to share it, i would be glad to get it :)


http://wiki.dovecot.org/Plugins/Zlib


1) Find the mails you want to compress in a single maildir.
2) Compress the mails to tmp/
    * Update the compressed files' mtimes to be the same as they were in 
the original files (e.g. touch command)
3) Run maildirlock <path> <timeout>. It writes PID to stdout, save
it.
    * <path> is path to the Maildir's dovecot-uidlist (the control 
directory, if it's separate)
    * <timeout> specifies how long to wait for the lock before failing.
4) If maildirlock grabbed the lock successfully (exit code 0) you can 
continue.
5) For each mail you compressed:
   1. Verify that it still exists where you last saw it.
   2. If it doesn't exist, delete the compressed file. Its flags may 
have been changed or it may have been expunged. This happens rarely, so 
just let the next run handle it.
   3. If the file does exist, rename() (mv) the compressed file over the 
original file.
          * Dovecot can now read the file, but to avoid compressing it 
again on the next run, you'll probably want to rename it again to 
include e.g. a "Z" flag in the file name to mark that it was
compressed
(e.g. 1223212411.M907959P17184.host,S=3271:2,SZ). Remember that the 
Maildir specifications require that the flags are sorted by their ASCII 
value, although Dovecot itself doesn't care about that.
6) Unlock the maildir by sending a TERM signal to the maildirlock 
process (killing the PID it wrote to stdout).



-- 


	Atenciosamente / Sincerily,
	Leonardo Rodrigues
	Solutti Tecnologia
	http://www.solutti.com.br

	Minha armadilha de SPAM, N?O mandem email
	gertrudes at solutti.com.br
	My SPAMTRAP, do not email it

Hi,

I?m using the following two scripts:

zip-list.sh:

find "$home" \
        -type f \
        -name '*,S=*' \
        -mtime +1 \
        -size +10k

The above script list the files that will be compressed.

zip-compressh.sh
#!/bin/bash

uid='vmail'
gid='vmail'

sleepOnFailure='2'

cat zip-list.txt|while read i
do
        if [ ! -f "$i" ]
        then
                continue
        fi

        type=$(file -bi "$i")
        fgrep 'message/rfc822' <<<"$type"
>/dev/null
        if [ "$?" != '0' ]
        then
                continue;
        fi

        echo -n "$i..."

        modify=$(stat -c %y "$i"|cut -d. -f1)
        basename=$(basename "$i")

        gzip -9 -c "$i" >/tmp/"$basename"

        mv /tmp/"$basename" "$i"
        exitStatus="$?"
        while [ -f "$i" -a "$exitStatus" != '0' ]
        do
                echo -n "wait($sleepOnFailure)..."
                sleep "$sleepOnFailure"
                mv /tmp/"$basename" "$i" 2>/dev/null
                exitStatus="$?"
        done

        chown "$uid"."$gid" "$i"
        touch -m -d "$modify" "$i"

        echo "OK"
done

This script reads the zip-list.txt and compress the file. We do not lock
the maildir, but we use touch to maintain original dates. We do not check
any 'Z' flags.

[]s
Fernando
>
>     Hi,
>
>     I'm wondering if someone already implemented a compression script
> (and would like to share it) based on the step-by-step provided on wiki
> page of Zlib module.
>
>     I can already do some find|xargs gzip .... but couldnt implement it
> completly as described on the wiki page. I know the suggested
> step-by-step is very detailed and probably a must simplier will be
> enough for almost anyone .... anyway, if someone have the full script
> and would like to share it, i would be glad to get it :)
>
>
> http://wiki.dovecot.org/Plugins/Zlib
>
>
> 1) Find the mails you want to compress in a single maildir.
> 2) Compress the mails to tmp/
>     * Update the compressed files' mtimes to be the same as they were
in
> the original files (e.g. touch command)
> 3) Run maildirlock <path> <timeout>. It writes PID to stdout,
save it.
>     * <path> is path to the Maildir's dovecot-uidlist (the
control
> directory, if it's separate)
>     * <timeout> specifies how long to wait for the lock before
failing.
> 4) If maildirlock grabbed the lock successfully (exit code 0) you can
> continue.
> 5) For each mail you compressed:
>    1. Verify that it still exists where you last saw it.
>    2. If it doesn't exist, delete the compressed file. Its flags may
> have been changed or it may have been expunged. This happens rarely, so
> just let the next run handle it.
>    3. If the file does exist, rename() (mv) the compressed file over the
> original file.
>           * Dovecot can now read the file, but to avoid compressing it
> again on the next run, you'll probably want to rename it again to
> include e.g. a "Z" flag in the file name to mark that it was
compressed
> (e.g. 1223212411.M907959P17184.host,S=3271:2,SZ). Remember that the
> Maildir specifications require that the flags are sorted by their ASCII
> value, although Dovecot itself doesn't care about that.
> 6) Unlock the maildir by sending a TERM signal to the maildirlock
> process (killing the PID it wrote to stdout).
>
>
>
> --
>
>
> 	Atenciosamente / Sincerily,
> 	Leonardo Rodrigues
> 	Solutti Tecnologia
> 	http://www.solutti.com.br
>
> 	Minha armadilha de SPAM, N?O mandem email
> 	gertrudes at solutti.com.br
> 	My SPAMTRAP, do not email it
>