Hi, I use dovecot for a long time now, but only as a single isolated server each time. I joined a company a few years back. We had trouble with compagnies hosting our mail, supposedly full redundant and so on. The company is small, but we have many dealers around the world, and it's growing. Mail became the fist choice for clients to contact the dealers. No mail, and we loose sales. For now we have a single server ( with a backup ) on east coast. And sometimes peoples from EU complain about speed.... ah users :) What I'd like to implement is a redundant system with 2 servers, one in NA, one in EU. And I'd also like to be able to add another server if needed on the west coast. Idea is, that if a server goes down, the users will be able to still receive and send mails, and never loose mails. For geobalacing and failover, I read that I can do it with DNS ( I'm with easydns ). I'm at the first stage where I collect informations that I try to understand and foresee a solution. First idea is to set up servers with a mysql master, slaves and a glusterfs in replica mode on the servers. I tried glusterfs on FreeBSD and OMG, it's slow as hell ! ( well maybe it's a trouble on the VMs nics ) On centos it's way better. But I read there might be trouble/index corruption for the mail storage on "shared" space using maildir. I also had a look at dsync, but I wonder if it can be used on more than 2 servers. I found many pages on dovecot clusters using shared storage NFS mounted, but I feel it's not really what I need as the servers will be in different datacenters. So any guide, clue hint would be really appreciated for me to do my homework ! Regards. Cedric
Am 13.02.2016 um 04:00 schrieb Cedric Malitte:> Hi, > > I use dovecot for a long time now, but only as a single isolated server > each time. > > I joined a company a few years back. We had trouble with compagnies hosting > our mail, supposedly full redundant and so on. > > The company is small, but we have many dealers around the world, and it's > growing. > > Mail became the fist choice for clients to contact the dealers. > No mail, and we loose sales. > > For now we have a single server ( with a backup ) on east coast. > And sometimes peoples from EU complain about speed.... ah users :) > > What I'd like to implement is a redundant system with 2 servers, one in NA, > one in EU. > And I'd also like to be able to add another server if needed on the west > coast. > > Idea is, that if a server goes down, the users will be able to still > receive and send mails, and never loose mails. > > For geobalacing and failover, I read that I can do it with DNS ( I'm with > easydns ). > > I'm at the first stage where I collect informations that I try to > understand and foresee a solution. > > First idea is to set up servers with a mysql master, slaves and a glusterfs > in replica mode on the servers. > I tried glusterfs on FreeBSD and OMG, it's slow as hell ! ( well maybe it's > a trouble on the VMs nics ) > On centos it's way better. > But I read there might be trouble/index corruption for the mail storage on > "shared" space using maildir. > > I also had a look at dsync, but I wonder if it can be used on more than 2 > servers. > > I found many pages on dovecot clusters using shared storage NFS mounted, > but I feel it's not really what I need as the servers will be in different > datacenters. > > So any guide, clue hint would be really appreciated for me to do my > homework ! > > Regards. > > Cedric >complicated and expensive for a small company why not have one imap server in US and the other in EU so US Users should have their Mailboxes on the US Server EU users should have their Mailboxes on the EU Server Smtp you might manage via smtp routing, perhaps with subdomains us eu and aliasing Or you might take some paid service which has geo stuff allready up and and running, dont try or think of doing it yourself, unlees youre a guru Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Thanks Robert, Having 2 separate servers would mean domain splitting in a way. So I'll have to dispatch incoming mail, and maintain 2 or more users databases. Well I could only maintain one, and let the DB replicate itself to other servers and only grab the users they are allowed to. We have many domain aliases for many countries and a dozen users traveling around the globe. So geo balacing would not be the best for them. But the redundancy is missing.... if one server goes down, I loose a bunch of users and that's not what I am looking for. I looked for providers offering that, but I guess my bosses would not let me spend over 3000$ per month on that. Else I would already have the mails hosted by microsoft and have a cocktail on the beach :) I'm not any kind of guru, I do not have any disciples. I did kernel mod development on embedded systems with freebsd 4 and 5, I still develop some linux kernel drivers for proprietary systems from time to time. My main degree is electronic engineering and I also graduated in computing related to management in other words I learned a bit of everything to be able to manage teams developers and understand what they do. Looks like it's doable, so I'll continue to dig and experiment. Best regards Cedric 2016-02-13 16:05 GMT-05:00 Robert Schetterer <rs at sys4.de>:> Am 13.02.2016 um 04:00 schrieb Cedric Malitte: > > Hi, > > > > I use dovecot for a long time now, but only as a single isolated server > > each time. > > > > I joined a company a few years back. We had trouble with compagnies > hosting > > our mail, supposedly full redundant and so on. > > > > The company is small, but we have many dealers around the world, and it's > > growing. > > > > Mail became the fist choice for clients to contact the dealers. > > No mail, and we loose sales. > > > > For now we have a single server ( with a backup ) on east coast. > > And sometimes peoples from EU complain about speed.... ah users :) > > > > What I'd like to implement is a redundant system with 2 servers, one in > NA, > > one in EU. > > And I'd also like to be able to add another server if needed on the west > > coast. > > > > Idea is, that if a server goes down, the users will be able to still > > receive and send mails, and never loose mails. > > > > For geobalacing and failover, I read that I can do it with DNS ( I'm with > > easydns ). > > > > I'm at the first stage where I collect informations that I try to > > understand and foresee a solution. > > > > First idea is to set up servers with a mysql master, slaves and a > glusterfs > > in replica mode on the servers. > > I tried glusterfs on FreeBSD and OMG, it's slow as hell ! ( well maybe > it's > > a trouble on the VMs nics ) > > On centos it's way better. > > But I read there might be trouble/index corruption for the mail storage > on > > "shared" space using maildir. > > > > I also had a look at dsync, but I wonder if it can be used on more than 2 > > servers. > > > > I found many pages on dovecot clusters using shared storage NFS mounted, > > but I feel it's not really what I need as the servers will be in > different > > datacenters. > > > > So any guide, clue hint would be really appreciated for me to do my > > homework ! > > > > Regards. > > > > Cedric > > > > complicated and expensive for a small company > why not have one imap server in US and the other in EU > so US Users should have their Mailboxes on the US Server > EU users should have their Mailboxes on the EU Server > > Smtp you might manage via smtp routing, perhaps with subdomains us eu > and aliasing > > Or you might take some paid service which has geo stuff allready up and > and running, dont try or think of doing it yourself, unlees youre a guru > > Best Regards > MfG Robert Schetterer > > -- > [*] sys4 AG > > http://sys4.de, +49 (89) 30 90 46 64 > Franziskanerstra?e 15, 81669 M?nchen > > Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 > Vorstand: Patrick Ben Koetter, Marc Schiffbauer > Aufsichtsratsvorsitzender: Florian Kirstein >
On 02/13/2016 04:00 AM, Cedric Malitte wrote:> Hi, > > I use dovecot for a long time now, but only as a single isolated server > each time. > > I joined a company a few years back. We had trouble with compagnies hosting > our mail, supposedly full redundant and so on. > > The company is small, but we have many dealers around the world, and it's > growing. > > Mail became the fist choice for clients to contact the dealers. > No mail, and we loose sales. > > For now we have a single server ( with a backup ) on east coast. > And sometimes peoples from EU complain about speed.... ah users :) > > What I'd like to implement is a redundant system with 2 servers, one in NA, > one in EU. > And I'd also like to be able to add another server if needed on the west > coast. > > Idea is, that if a server goes down, the users will be able to still > receive and send mails, and never loose mails. > > For geobalacing and failover, I read that I can do it with DNS ( I'm with > easydns ). > > I'm at the first stage where I collect informations that I try to > understand and foresee a solution. > > First idea is to set up servers with a mysql master, slaves and a glusterfs > in replica mode on the servers. > I tried glusterfs on FreeBSD and OMG, it's slow as hell ! ( well maybe it's > a trouble on the VMs nics ) > On centos it's way better. > But I read there might be trouble/index corruption for the mail storage on > "shared" space using maildir. > > I also had a look at dsync, but I wonder if it can be used on more than 2 > servers. > > I found many pages on dovecot clusters using shared storage NFS mounted, > but I feel it's not really what I need as the servers will be in different > datacenters. > > So any guide, clue hint would be really appreciated for me to do my > homework ! > > Regards. > > CedricHi Cedric, I think a simpler solution will not just be cheaper but less complex - and with that more reliable: The speed problem of the EU users is probably just feeling. You should quantify it for both SMTP and IMAP. Collect that data for the scenarios that your users complain about (is it to a partner or inter-office?). Only then can you work on a solution that you will be able to prove to them, is better. This is paramount. My suggestions: * Server on the east cost is good for both NA and EU. * Good (better?) internet connection for the EU office, prioritize SMTP vs HTTP in router/firewall (fast internet is WAY cheaper than cluster setups plus administrators) * SMTP relay in EU _office_, so that _sending_ mails is with LAN speed for users Create a redundant setup for SMTP and IMAP together on the east cost. You'll get redundancy without the WAN problem. Setup a secondary MX in a different data center for uber-redundency. It will not enable your users to read their mail in case the 1st data center is on fire, but no client mails will get lost, as they will be queued on the 2nd MX - better read client mails late then never! Setup a clone of the primary server at the 2nd MX and sync mails & backup there on a hourly basis. If the 1st data center is not back in an hour, you can still switch DNS to the 2nd site and your users will have had a very short downtime. The result is not a top-notch 100% solution, but it is simple and everything is implemented on application layer. That gives you freedom to switch products, hardware, platform and administrators(!). Ask your customer/supervisor what uptime is necessary and how much they are willing to pay. The SLAs of MS/Google/etc offer up to 99.9% (~9 hours downtime per year). If that is the goal, then they should pay the price for their equipment and staff. For anything less my argument is less complexity for higher reliability. Greetings Daniel -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: <http://dovecot.org/pipermail/dovecot/attachments/20160215/11f1e694/attachment.sig>
Hi Daniel, I do already have a backup server where I replicate the mails every 4 hours using imapsync. Not the best, but this is workable. The one time I redirected to backup server, people where complaining that they were having duplicate mails ( this was pop access ) I also have mx backup with easydns, was part of the plan so I use it. We have three main offices, a few on the road managers and many dealers. Dealers run their own business under our trademark, and use the same domains for mails as ours (head offices). I mean every mailbox is on the same server. May be not the best. I have the mandate to get everything running smooth. There can be outages, but not too long. One hour is too long for some.... It will also be to educated people here and there. I understand that what they want costs money, but I'm not able to make them understand that what they want is not plug and play cheap solution. But, I'm going to test on VMs dovecot on glusterfs with dovecot director if I manage to get it working properly. Might be a challenge, but who knows, it might work :) Best regards, Cedric 2016-02-15 4:16 GMT-05:00 Daniel Tr?der <troeder at univention.de>:> On 02/13/2016 04:00 AM, Cedric Malitte wrote: > > Hi, > > > > I use dovecot for a long time now, but only as a single isolated server > > each time. > > > > I joined a company a few years back. We had trouble with compagnies > hosting > > our mail, supposedly full redundant and so on. > > > > The company is small, but we have many dealers around the world, and it's > > growing. > > > > Mail became the fist choice for clients to contact the dealers. > > No mail, and we loose sales. > > > > For now we have a single server ( with a backup ) on east coast. > > And sometimes peoples from EU complain about speed.... ah users :) > > > > What I'd like to implement is a redundant system with 2 servers, one in > NA, > > one in EU. > > And I'd also like to be able to add another server if needed on the west > > coast. > > > > Idea is, that if a server goes down, the users will be able to still > > receive and send mails, and never loose mails. > > > > For geobalacing and failover, I read that I can do it with DNS ( I'm with > > easydns ). > > > > I'm at the first stage where I collect informations that I try to > > understand and foresee a solution. > > > > First idea is to set up servers with a mysql master, slaves and a > glusterfs > > in replica mode on the servers. > > I tried glusterfs on FreeBSD and OMG, it's slow as hell ! ( well maybe > it's > > a trouble on the VMs nics ) > > On centos it's way better. > > But I read there might be trouble/index corruption for the mail storage > on > > "shared" space using maildir. > > > > I also had a look at dsync, but I wonder if it can be used on more than 2 > > servers. > > > > I found many pages on dovecot clusters using shared storage NFS mounted, > > but I feel it's not really what I need as the servers will be in > different > > datacenters. > > > > So any guide, clue hint would be really appreciated for me to do my > > homework ! > > > > Regards. > > > > Cedric > > Hi Cedric, > > I think a simpler solution will not just be cheaper but less complex - > and with that more reliable: > > The speed problem of the EU users is probably just feeling. You should > quantify it for both SMTP and IMAP. Collect that data for the scenarios > that your users complain about (is it to a partner or inter-office?). > Only then can you work on a solution that you will be able to prove to > them, is better. This is paramount. > > My suggestions: > * Server on the east cost is good for both NA and EU. > * Good (better?) internet connection for the EU office, prioritize SMTP > vs HTTP in router/firewall (fast internet is WAY cheaper than cluster > setups plus administrators) > * SMTP relay in EU _office_, so that _sending_ mails is with LAN speed > for users > > Create a redundant setup for SMTP and IMAP together on the east cost. > You'll get redundancy without the WAN problem. > > Setup a secondary MX in a different data center for uber-redundency. It > will not enable your users to read their mail in case the 1st data > center is on fire, but no client mails will get lost, as they will be > queued on the 2nd MX - better read client mails late then never! > > Setup a clone of the primary server at the 2nd MX and sync mails & > backup there on a hourly basis. If the 1st data center is not back in an > hour, you can still switch DNS to the 2nd site and your users will have > had a very short downtime. > > The result is not a top-notch 100% solution, but it is simple and > everything is implemented on application layer. That gives you freedom > to switch products, hardware, platform and administrators(!). > > Ask your customer/supervisor what uptime is necessary and how much they > are willing to pay. The SLAs of MS/Google/etc offer up to 99.9% (~9 > hours downtime per year). If that is the goal, then they should pay the > price for their equipment and staff. For anything less my argument is > less complexity for higher reliability. > > Greetings > Daniel > >