On Mon, Jul 24, 2017 at 07:56:23PM +0300, Aki Tuomi wrote:> Well, dovecot does not really guarantee access concurrency safety if you access indexes using more than one instance of dovecot at the same time.Pardon my ignorance, but how does Dovecot handle when an IMAP client connects multiple times concurrently? Does it not launch multiple instances?> Nevertheless, did you try w/o LizardFS, just to rule out any bugs or incompabilities?Moving everybody off of LizardFS is not an option, and this has affected many separate mailboxes. Now that we have implemented a director front end (instead of just a semi-persistent load balancer), the instances have been reduced, but it is still happening on accounts that are not being accessed across multiple servers. I will see if I can pin these down to a single server and move them onto non-shared storage there. -- Bruce Guenter <bruce at untroubled.org> http://untroubled.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Digital signature URL: <http://dovecot.org/pipermail/dovecot/attachments/20170731/0fe350e5/attachment.sig>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 31 Jul 2017, Bruce Guenter wrote:> On Mon, Jul 24, 2017 at 07:56:23PM +0300, Aki Tuomi wrote: >> Well, dovecot does not really guarantee access concurrency safety if you access indexes using more than one instance of dovecot at the same time. > > Pardon my ignorance, but how does Dovecot handle when an IMAP client > connects multiple times concurrently? Does it not launch multiple > instances?Aki means that multiple physically instances of Dovecot may access the index files not cleanly. If there is one instance only and you connect to this single instance multiple times, there is no problem. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBWYAfa3z1H7kL/d9rAQJ9TggAs8JIB1o8QvYdoTdAGLfxhdjXhvpiX9Pc 9vbRUl5Ha21ZYPL4iZ9zdaf+ftaj2SYcXEWORCSy5hTl85rw5TMKtr2uJd9c8H8C O/7T7jGRJ/2LFSeiHFbyFYyZTgtpC5QNbwtAnMp0SAwN51NYcYBfcM+TJHzbAcmA SCvDO8DbQPer82RJ2h9dXp3TTFYnYFeVOmgEjTEFiyMi69w07cKXKsJCxu+jLPE7 E8ZpRCS68xiCZriGpuoqDfpNBD21wBYxiEaaE9HEK4ZIhDx812Xgu5ORj5zxy/MR cTEKDcgLAtWb/EoLLDLP47Mtw6NQiB7nDZLUVyhvBRLzNl8RzIUh/A==kx9f -----END PGP SIGNATURE-----
> On 1 Aug 2017, at 6.23, Bruce Guenter <bruce at untroubled.org> wrote: > > On Mon, Jul 24, 2017 at 07:56:23PM +0300, Aki Tuomi wrote: >> Well, dovecot does not really guarantee access concurrency safety if you access indexes using more than one instance of dovecot at the same time. > > Pardon my ignorance, but how does Dovecot handle when an IMAP client > connects multiple times concurrently? Does it not launch multiple > instances?Each imap connection is one imap process. So if (unlimited) thunderbird connects to dovecot imap and sees 30 folders, it will open 30 imap connections and dovecot will launch imap process to manage each connection. And on top of that LMTP delivery again is another process to access the FS. If these processes get different view of the filesystem you will face corruption. Specially when you are dealing with multiple servers the metadata syncing across servers is a challenge for the cluster filesystems. I?m a bit surprised that the corruption does happen even with one server.> Moving everybody off of LizardFS is not an option, and this has affected > many separate mailboxes. Now that we have implemented a director front > end (instead of just a semi-persistent load balancer), the instances > have been reduced, but it is still happening on accounts that are not > being accessed across multiple servers. I will see if I can pin these > down to a single server and move them onto non-shared storage there.Without testing I still would say that it?s LizardFS to blame. There was similar problems with another cluster filesystem GlusterFS too. Sami
On Tue, Aug 01, 2017 at 01:14:10PM +0300, Sami Ketola wrote:> Each imap connection is one imap process. So if (unlimited) thunderbird connects > to dovecot imap and sees 30 folders, it will open 30 imap connections and dovecot > will launch imap process to manage each connection. And on top of that LMTP > delivery again is another process to access the FS.That is what I expected, yes.> If these processes get different view of the filesystem you will face corruption. > Specially when you are dealing with multiple servers the metadata syncing across > servers is a challenge for the cluster filesystems.Of course. It can be managed, but still best avoid it by using a director. We had been using a load balancer that persistently mapped remote IPs to one server or another, but discovered that was not sufficient (clients using both webmail and a desktop client came in from different remote IPs, oops), so we implemented the Dovecot director on our load balancer.> I?m a bit surprised that the corruption does happen even with one server.That is what I am seeing. Even when the account is accessing the maildir from a single server, there is still corruption.> Without testing I still would say that it?s LizardFS to blame. There was similar problems > with another cluster filesystem GlusterFS too.Is there anything I could test to diagnose this further? I have moved the account with the most frequent problems to physical storage, but this won't tell me why LizardFS is causing problems. -- Bruce Guenter <bruce at untroubled.org> http://untroubled.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Digital signature URL: <http://dovecot.org/pipermail/dovecot/attachments/20170801/ca1849e6/attachment.sig>