dovecot - Nov 2019 - Using dovecot Replication in a medium to large enterprise.

Good day Guys

Just wanted to pick the communities brain and experience(s) for a second.

At $CORP where I work. My team has inherited a single server mail solution.

We need to look to building a standby / replicated solution.

One of the things we were looking at is https://wiki.dovecot.org/Replication

We have in excess of +/- 500 mail boxes and using just under 1TB of disk 
space. That is a lot of small files.

Reading about this plugin / module, things that stick out and is 
concerning is the comments:

"The replication in general is a bit resource intensive, so it's not 
recommended to be used in multi-million user installations."

"WARNING: Shared folder replication doesn't work correctly right now
?"

I would like to ask the community (and anyone else in a similar 
enterprise) how and if this solution works and scales.

In short, I am asking, is anyone else using Dovecot Replication in a 
medium to large enterprise?
Can you please share your experiences, pitfalls and if you have any tips 
managing mail replication.

Kindest regards and thanks in advance.
Brent Clark

Hi,

Replication - no, never used it, but we do use Dovecot dsync a lot.
Over a few hundred servers that is, in a "multi-million user 
installations".

There are some issues and pitfalls that are solvable, but in general it 
works.

In the sense of resource intensity I guess dsync is as intense as 
replication,
and IT IS resource intensive. Which is also solvable by process priority 
etc.

For example we are tracking the state of every mailbox dsync,
as it sometimes fails and dsync needs to be re-tried.

One of the pitfalls is that I have never managed to make Sieve 
replication to work,
so rsync-over-ssh is used for that.

In general it was pretty hard to make a RELIABLE dsync solution that works.

~500 mailboxes is not a lot, and it will require some effort to make 
everything tick,
I'm not sure if it is worth it in your case.

The PRO version has S3 driver, consider this solution too.

On 28/11/2019 09:35, Brent Clark via dovecot wrote:
> Good day Guys
>
> Just wanted to pick the communities brain and experience(s) for a second.
>
> At $CORP where I work. My team has inherited a single server mail 
> solution.
>
> We need to look to building a standby / replicated solution.
>
> One of the things we were looking at is 
> https://wiki.dovecot.org/Replication
>
> We have in excess of +/- 500 mail boxes and using just under 1TB of 
> disk space. That is a lot of small files.
>
> Reading about this plugin / module, things that stick out and is 
> concerning is the comments:
>
> "The replication in general is a bit resource intensive, so it's
not
> recommended to be used in multi-million user installations."
>
> "WARNING: Shared folder replication doesn't work correctly right
now ?"
>
> I would like to ask the community (and anyone else in a similar 
> enterprise) how and if this solution works and scales.
>
> In short, I am asking, is anyone else using Dovecot Replication in a 
> medium to large enterprise?
> Can you please share your experiences, pitfalls and if you have any 
> tips managing mail replication.
>
> Kindest regards and thanks in advance.
> Brent Clark

On 2019-11-28 18:35, Brent Clark via dovecot wrote:
> Good day Guys
> 
> Just wanted to pick the communities brain and experience(s) for a 
> second.
> 
> At $CORP where I work. My team has inherited a single server mail 
> solution.
> 
> We need to look to building a standby / replicated solution.
> 
> One of the things we were looking at is 
> https://wiki.dovecot.org/Replication
You may want to look into a block device replication solution, like 
DRBD, integrated in a Pacemaker cluster. Can offer you a reliable, 
tested and resource friendly solution even in the simplest approaches 
consisting of a dual node, active-passive cluster.

-- 
Adi Pircalabu

On 28 Nov 2019, at 00:35, Brent Clark <brentgclarklist at gmail.com>
wrote:
> We have in excess of +/- 500 mail boxes and using just under 1TB of disk
space. That is a lot of small files.
That?s honestly pretty small. When it says "it's not recommended to be
used in multi-million user installations?
You are several orders of magnitude away from that. Well, three if you want to
be technical about it.

If you want a backup server ready to go, rsync will probably work just fine for
you, or as Adi have mentioned, a block-level replicator (though I suspect that
is overkill) and dsync is likely to do exactly what you want. Yes, you have to
keep an eye on it, but at your level It?s probably going to be solid (based on
reading other?s experiences with it).

Of course, it is also easy to do it wrong, so careful reading of the
documentation is critical, and testing your solution a coupe thousands times
before you start relying on it.



-- 
MS Word still hasn't caught up -- it has more bells and whistles, but
	not as many pistons and cylinders. -- Steve Hayes