We are running our IMAP mail server on a VPS in the cloud (dovecot+exim+horde webmail), and we are using our on-premises NAS for daily backups through custom rsync scripts (which work well with maildir). There is a lot of storage available on the NAS, while not so much on the VPS. Since, we often get downtime on-premises due to power outages we cannot move the mail server on-premises (though the outages do not interfere too much with the backups). Additional storage in the cloud is at a premium as we would like to avoid further increasing costs. Now, my idea is to set up an NFS volume on the NAS, and configure the cloud-based server as the client. I would like to store only the most recent emails on the main server, and keep the whole data set on the NAS. Naturally I have read the dovecot wiki concerning NFS, and I have browsed this mailing list's archive for answers, but I still have many concerns. Ideally, I would like to use as much storage as possible from the main server, so that only really old emails need to be fetched over the network if requested over IMAP. Also I would like the search function to work regardless whether the NAS is available or not. I wonder if anybody tried a similar configuration before. My concerns are: - should I use NFS volume directly and configure FS-Cache, or should I configure a cache volume separately using e.g. bcache or some other tool? - how to configure the cache to keep as many files and metadata locally? - is it possible to fine tune the cache so it always keeps all the newly created/modified files locally? - is it possible to tune the cache so it keeps at least n files for each user (each maildir)? - will it work fine if I put NFS traffic into an ssh tunnel for security? - will dovecot freeze/crash when the NFS volume disconnects or is temporarily unavailable, what will actually happen in such circumstances? - will the search function work when the NFS volume is unavailable? (I also plan to use solr in the future) - what happens when the main server crashes and then recovers from the NFS share? is the share guaranteed to be consistent? I hope that I have not asked too many questions for a single message... ;) I would really appreciate any help, because I am rather a newbie in dovecot administration, and it overwhelms me. Perhaps, what I am trying to achieve is not the way to go. I am open to suggestions. Best regards, Maciej -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20201018/b4450877/attachment.html>
On 18.10.20 19:50, Maciej Kokoci?ski wrote:> Now, my idea is to set up?an NFS volume on the NAS, and configure the > cloud-based server as the client. I would like to store only the most > recent emails on the main?server, and keep the whole data set on the NAS.Doing NFS over the Internet is such a bad idea. Don't do it. Also don't do NFS over a WAN link inside a VPN. The latency will kill you. In addition to that, the flakiness of the on-prem NAS will hard-lock any process accessing or trying to access anything on the NFS share during times where it is not available. This setup would severely degrade not only the performance but also the availability of your mails. Do not do this. In your case I only see the option to either deal with the increased costs, move to a different ISP which is cheaper or move to a hosted mail solution, which may also be cheaper. But jerry-rigging an NFS share on a flaky box over a WAN link is such a bad idea. Please, reconsider. Gr??e, Sven.
What about moving the mailserver on-premises, buy a big UPS and do some batched smtp on the cloud, so if you are down, you will not lose incoming mails? -----Original Message----- From: Maciej Kokociski [mailto:maciej.kokocinski at hands.pl] Sent: Sunday, October 18, 2020 7:51 PM To: dovecot at dovecot.org Subject: Using NFS to extend local email storage We are running our IMAP mail server on a VPS in the cloud (dovecot+exim+horde webmail), and we are using our on-premises NAS for daily backups through custom rsync scripts (which work well with maildir). There is a lot of storage available on the NAS, while not so much on the VPS. Since, we often get downtime on-premises due to power outages we cannot move the mail server on-premises (though the outages do not interfere too much with the backups). Additional storage in the cloud is at a premium as we would like to avoid further increasing costs. Now, my idea is to set up an NFS volume on the NAS, and configure the cloud-based server as the client. I would like to store only the most recent emails on the main server, and keep the whole data set on the NAS. Naturally I have read the dovecot wiki concerning NFS, and I have browsed this mailing list's archive for answers, but I still have many concerns. Ideally, I would like to use as much storage as possible from the main server, so that only really old emails need to be fetched over the network if requested over IMAP. Also I would like the search function to work regardless whether the NAS is available or not. I wonder if anybody tried a similar configuration before. My concerns are: - should I use NFS volume directly and configure FS-Cache, or should I configure a cache volume separately using e.g. bcache or some other tool? - how to configure the cache to keep as many files and metadata locally? - is it possible to fine tune the cache so it always keeps all the newly created/modified files locally? - is it possible to tune the cache so it keeps at least n files for each user (each maildir)? - will it work fine if I put NFS traffic into an ssh tunnel for security? - will dovecot freeze/crash when the NFS volume disconnects or is temporarily unavailable, what will actually happen in such circumstances? - will the search function work when the NFS volume is unavailable? (I also plan to use solr in the future) - what happens when the main server crashes and then recovers from the NFS share? is the share guaranteed to be consistent? I hope that I have not asked too many questions for a single message... ;) I would really appreciate any help, because I am rather a newbie in dovecot administration, and it overwhelms me. Perhaps, what I am trying to achieve is not the way to go. I am open to suggestions. Best regards, Maciej
On Sun, 18 Oct 2020, Sven Hartge wrote:> On 18.10.20 19:50, Maciej Kokoci?ski wrote: > >> Now, my idea is to set up?an NFS volume on the NAS, and configure the >> cloud-based server as the client. I would like to store only the most >> recent emails on the main?server, and keep the whole data set on the NAS. > > Doing NFS over the Internet is such a bad idea. Don't do it. > > Also don't do NFS over a WAN link inside a VPN. The latency will kill you. > > In addition to that, the flakiness of the on-prem NAS will hard-lock any > process accessing or trying to access anything on the NFS share during times > where it is not available. > > This setup would severely degrade not only the performance but also the > availability of your mails. > > Do not do this. > > In your case I only see the option to either deal with the increased costs, > move to a different ISP which is cheaper or move to a hosted mail solution, > which may also be cheaper. > > But jerry-rigging an NFS share on a flaky box over a WAN link is such a bad > idea. Please, reconsider. > > Gr??e, > Sven. > >Sven is correct, NFS over the internet is not a great idea. You might want to look into moving the access from the server to the client. Setup an imap server at the location where you host the NAS and configure the client with a second imap connection where older archived email is stored. Isn't that one of the strengths of the imap protocol. The ability to access multiple imap folders from multiple locations. It's easy to do with standalone clients although I'm not sure how much trouble it would be to hack your webmail software to include a sedondary imap profile. Ted.