dovecot - Oct 2016 - Dovecot + SAN with dedup+compression

Hello

     We have a big Dovecot Server (over 20 TB of mail, 40K users, over 
30K iops) and we are evaluating a new SAN Array. We have competitive 
proposals from Pure Storage (All Flash) , HP 3Par (Hybrid), NetApp FaaS 
(Hybrid) & EMC Unity (Hybrid). We rely in mdbox+zlib for main storage 
and mdbox+lzma for secondary storage, does anybody in the list have 
experience with SAN-based deduplication/compression?
     Regards
     Maria.

Mar?a Arrea <maria.arrea at gmx.com> wrote:
> We have a big Dovecot Server (over 20 TB of mail, 40K users, over 30K
> iops) and we are evaluating a new SAN Array. We have competitive
> proposals from Pure Storage (All Flash) , HP 3Par (Hybrid), NetApp
> FaaS (Hybrid) & EMC Unity (Hybrid). We rely in mdbox+zlib for main
> storage and mdbox+lzma for secondary storage, does anybody in the list
> have experience with SAN-based deduplication/compression?
My experiences and experiments with NetApp and Dovecot mdbox show:

a) Compression is better done in Dovecot, don't enable the compression
   on the filer. 

b) Mail in Maildir or mdbox format does not deduplicate really well on
   the filer, expect to get about 10% to 15% savings. 
   
   It does not hurt the performance if you turn it on (at least on
   NetApp), but you won't get the big savings you get with
   Exchange-Databases. (I suspect this is because of the layout of the
   mails inside mdbox on disk.)

   If you really need to deduplicate, then use the builtin SIS from
   Dovecot.

Gr??e,
Sven.

-- 
Sigmentation fault. Core dumped.