Thank you for the reply. I have done the sync manually with "doveadm sync" command. But, I have not configured the replication yet. I am looking at the below webpage for the replication. https://wiki.dovecot.org/Replication I am using the dovecot version "2.2.36". I am confused with what needs to be done after reading that page. 1. They are talking about v2.3.1 and v2.2+. Which one do I need to follow? Could you please give me more details on this? Providing some sample settings will be more helpful for me, please. 2. Also, do I need to set the replication on both of my servers the same and as it is? Please help me in understanding this? Thanks, Kishore Potnuru On Thu, Jul 16, 2020 at 6:20 PM Gerald Galster <list+dovecot at gcore.biz> wrote:> I have 2 test servers with the below configuration. > > =========================> Linux OS - Red Hat Enterprise Linux Server release 7.7 (Maipo) > Dovecot version - 2.2.36 (1f10bfa63) > Postfix version - 2.10.1 > =========================> > Trying to create High Availability. > > I have added both of the above servers behind a F5 load balancer. I have > got a Load Balancer FQDN "intl-dev-imaptest.testorg.com". I have > enabled/opened the ports (25/110/143/993/995) on the above " > intl-dev-imaptest.testorg.com". > > When I send 10 emails to "intl-dev-imaptest.testorg.com", then those 10 > emails are getting distributed between the above 2 backend servers (5 > emails to each server). I see those 5 emails each in both the servers. > > > You should see 10 emails on each server if replication is working: 5 > emails that were directly delivered via loadbalancer and 5 emails from the > other server via replication. > > From Outlook I have configured the email address using "POP and IMAP", > when I gave the IMAP server as "intl-dev-imaptest.testorg.com" ,then it > shows only 5 emails from server1 in outlook and after a few > seconds/minutes, automatically it shows/refreshes the other 5 emails from > server2. But I am not seeing all the 10 emails at the same time. why? > > > The loadbalancer does its job, sometimes the Outlook connection is > forwarded to server A sometimes to server B. So you just see the mails on > the respective server. This is very bad. Your Mailclient is probably > syncing and deleting emails everytime the connection is moved to the other > server. As I suggested in the other thread you should at least configure > some kind of ip stickyness when using a loadbalancer, so that your > mailclient reaches the same backend. > > The purpose of replication is that two servers, operating independently, > have the same dataset. Your servers seem to have completely distinct > datasets, which indicates replication is not working. Did you configure > replication? > > > So I tried the sync command. When I execute sync command like below from > server1, it reflects the same emails in other server2 also. Then I see the > same number of emails in both the servers. Is it not possible to access the > both servers emails at one time with the "sync" command? Do we need to run > this on all the email boxes on both servers? don't we miss/lose any emails > during this sync process multiple times? > > "doveadm sync -f -u kishore at test.testorg.com > remote:vmail at bal3200dev002.testorg.com" > > Is "replication" and "sync" are same? > > > Think of replication as a continous sync. This has to be done every time > an email is delivered, which dovecot does automatically when replication is > configured. > > You don't lose any emails because the replication/sync is bidirectional, > it copies from the respective other server what's missing. Of course this > is not instant but usually happens within seconds. > > > Why are we not able to see all the emails at one time without the "sync" > command? > > > Probably because you did not configure replication? > > > What is the best and easiest way to create High Availability with just 2 > servers, like emails should travel to both servers equally and if one > server goes down also, another server should take care of the > emails/functionality. This is my requirement. > > > It seems you just have to configure replication. > > My current real time environment: I have around 10 email domains and each > domain is having 10 imap emails. In total around 100 email boxes/addresses. > We receive around 50K emails in a day to those email addresses. We are > using the "Maildir" format in our environment. Want to move to the High > Availability option with 2 servers. > > > See my other mail, it may be better to use mdbox instead of maildir. > > Best regards > Gerald >-------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20200716/a970f9bf/attachment.html>
> I have done the sync manually with "doveadm sync" command. But, I have not configured the replication yet.If you don't tell dovecot where to replicate, nothing gets replicated.> I am looking at the below webpage for the replication. > > https://wiki.dovecot.org/Replication <https://wiki.dovecot.org/Replication> > > I am using the dovecot version "2.2.36". I am confused with what needs to be done after reading that page. > > 1. They are talking about v2.3.1 and v2.2+. Which one do I need to follow? Could you please give me more details on this? Providing some sample settings will be more helpful for me, please.I don't understand your confusion. You are using 2.2.36, which is v2.2+ (meaning a version greater than 2.2). The documentation states you need at least 2.3.1 if you want to use the noreplicate feature. So you can't use that with 2.2.36, but as your goal is to replicate everything you don't need "noreplicate". Besides that I can't see any difference in configuring replication for 2.2/2.3. If you want to replicate emails with ssh you just have to follow the first section, the sample settings are right on that page. It's basically copying everything from "mail_plugins = $mail_plugins notify replication" to "replication_max_conns = 10" into a config file like /etc/dovecot/conf.d/12-replication.conf You only have to change the following line to match your server/ssh setup: mail_replica = remote:vmail at anotherhost.example.com Then generate and configure ssh keys for user vmail (passwordless authentication) on both servers.> 2. Also, do I need to set the replication on both of my servers the same and as it is?On server A) you should configure mail_replica = remote:vmail at server_B and on server B) you should configure mail_replica = remote:vmail at server_A If you skip B) and new mail arrives on B) it is not immediately synced to A) In that case you would have to wait until a mail gets synced from A) (you remember sync is bidirectional) Best regards Gerald -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20200717/96eaa719/attachment-0001.html>
Hi Gerald,
Thank you for the details. As per your suggestion, I have made the changes
to dovecot.conf file. Still I don't see any replication is happening.
Please see the dovecot.conf file.
I do not see "/etc/dovecot/conf.d/12-replication.conf" in my servers.
So I
had put everything in the dovecot.conf file only. Please see the complete
data in it below. The below data is in server A. In other server (server B)
Also I have the same configuration, except mail_replica line and it is
pointing to the other server like, " mail_replica remote:vmail at
bal3200dev001.testorg.com ".
I have generated/configured the ssh keys also for vmail user in both
servers. Now When i manually ssh to the server, it is not asking for a
password.
========================================disable_plaintext_auth = no
listen = *
log_path = /var/log/dovecot.log
#mail_location = maildir:/z1nfs/mail/virtual/%d/%n/Maildir/
mail_location = maildir:/z1devenv/mail/virtual/%d/%n/Maildir/
passdb {
args = /etc/dovecot/passwd
driver = passwd-file
}
pop3_uidl_format = %g
protocols = pop3 imap
#ssl = yes
#ssl_cert = </etc/pki/dovecot/certs/dovecot.pem
#ssl_key = </etc/pki/dovecot/private/dovecot.pem
userdb {
# args = uid=vmail gid=vmail home=/z1nfs/mail/virtual/%d/%n
args = uid=vmail gid=vmail home=/z1devenv/mail/virtual/%d/%n
driver = static
}
mail_debug = yes
verbose_ssl = no
# Enable the replication plugin globally
mail_plugins = $mail_plugins notify replication
service replicator {
process_min_avail = 1
}
dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u
plugin {
mail_replica = remote:vmail at bal3200dev002.testorg.com
}
service aggregator {
fifo_listener replication-notify-fifo {
user = vmail
}
unix_listener replication-notify {
user = vmail
}
}
service replicator {
unix_listener replicator-doveadm {
mode = 0600
user = vmail
}
}
replication_max_conns = 10
========================================
Am i missing anything here? Please guide me.
Thanks,
Kishore Potnuru
On Fri, Jul 17, 2020 at 12:36 AM Gerald Galster <list+dovecot at
gcore.biz>
wrote:
>
> I have done the sync manually with "doveadm sync" command. But, I
have not
> configured the replication yet.
>
>
> If you don't tell dovecot where to replicate, nothing gets replicated.
>
> I am looking at the below webpage for the replication.
>
> https://wiki.dovecot.org/Replication
>
> I am using the dovecot version "2.2.36". I am confused with what
needs to
> be done after reading that page.
>
> 1. They are talking about v2.3.1 and v2.2+. Which one do I need to follow?
> Could you please give me more details on this? Providing some sample
> settings will be more helpful for me, please.
>
>
> I don't understand your confusion. You are using 2.2.36, which is v2.2+
> (meaning a version greater than 2.2).
> The documentation states you need at least 2.3.1 if you want to use the
> noreplicate feature.
> So you can't use that with 2.2.36, but as your goal is to replicate
> everything you don't need "noreplicate".
> Besides that I can't see any difference in configuring replication for
> 2.2/2.3.
>
> If you want to replicate emails with ssh you just have to follow the first
> section, the sample settings are
> right on that page. It's basically copying everything from
"mail_plugins > $mail_plugins notify replication"
> to "replication_max_conns = 10" into a config file like
> /etc/dovecot/conf.d/12-replication.conf
>
> You only have to change the following line to match your server/ssh setup:
> mail_replica = remote:vmail at anotherhost.example.com
>
> Then generate and configure ssh keys for user vmail (passwordless
> authentication) on both servers.
>
> 2. Also, do I need to set the replication on both of my servers the same
> and as it is?
>
>
> On server A) you should configure mail_replica = remote:vmail at server_B
and
> on server B) you should configure mail_replica = remote:vmail at server_A
>
> If you skip B) and new mail arrives on B) it is not immediately synced to
> A)
> In that case you would have to wait until a mail gets synced from A)
> (you remember sync is bidirectional)
>
> Best regards
> Gerald
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20200717/020d3cc1/attachment.html>