> > I have about 100TB of mailboxes in Maildir format on NFS (NetApp FAS) > and works very well, for performance but also stability.Hmmm, I would like to read something else. Eg that the design/elementary properties of distributed storage result into that all such systems are performing about the same. Maybe there should be more focus on ceph performance development instead of this cephadm?> The main problem of using Ceph or GlusterFS to store Maildir is the high > use of metadata that dovecot require for check new messages and others > activity. On my storage/NFS the main part of the traffic and I/O is > metadata traffic on small file (high file count workload).That is why I am using mdbox files of 4MB. I hope that should give me hardly any write amplification. I am also seperating between ssd and hdd pools by auto archiving email to the hdd pools> > And Ceph or GlusterFS are very inefficient with this kind of workload > (many metadata GETATTR/ACCESS/LOOKUP and high numer of small files).I am using rbd. After luminuous I had some issues with the cephfs and do not want to store operational stuff on it yet.
> That is why I am using mdbox files of 4MB. I hope that should give me hardly any write amplification. I am also seperating between ssd and hdd pools by auto archiving email to the hdd pools > I am using rbd. After luminuous I had some issues with the cephfs and do not want to store operational stuff on it yet.I am very interested in that setup, since I am currently planning to reshape my cluster in a similar way (currently from sole distribution via director to distribution + HA). Could you post a short overview (scheme) and some important configurations of your setup? Did you do any performancetesting? Also, when you say rbd in a clustered context, is that one block device per node while the director still spreads the accounts over the nodes? Thanks in advance, M. Schmidt -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 840 bytes Desc: OpenPGP digital signature URL: <https://dovecot.org/pipermail/dovecot/attachments/20220407/048e4679/attachment.sig>