Hi, I've observed some odd behaviour with dsync replication between two hosts, specifically to do with sieve script replication. In short, I have two hosts which replicate in a master-master type setup where almost all of the reads and writes happen to just one of the two hosts. They are both running 2.2.devel (9dc6403), which is close to the latest 2.2 -git . Pigeonhole is running master-0.4 . This is on x86_64 Gentoo. Normal mail replication between Maildir's for all users works fine, however it appears that something recently committed to the code has broken sieve script replication between the two. I am sure this did once work. Replication is via tcps: . Sieve scripts on the lesser-used host are not up to date by an order of days/weeks with the main host and they don't seem to re-replicate - even if the rules don't exist at all on the replica. The symptoms and effects look to be the same as this (unanswered) post from December: http://dovecot.org/list/dovecot/2015-December/102690.html I am not sure how to view the transaction log files, but I am seeing the same symptoms, ie no live replication, and on the lesser-used host almost all the scripts were old and some had the 1970 date on them. Even after forcing a [dsync replication replicate '*'] the scripts are not replicated. As it stands now there are no sieve scripts on one of the two members and the system seems unable to replicate by itself. Secondly, I am also seeing some doubled up outputs if I run 'doveadm sieve list -A': thunderstorm reuben # doveadm sieve list -A reuben rules ACTIVE liam rules ACTIVE kaylene rules ACTIVE reuben rules ACTIVE liam rules ACTIVE kaylene rules ACTIVE ... Has anyone else experienced the replication problem? Are sieve scripts actually replicating in live time for other 2.2.24/2.2.25 users as well? For me I didn't notice this till I went looking so I wonder if other people are experiencing this but just not aware of it yet...? Reuben
Reuben Farrelly <reuben-dovecot at reub.net> wrote:> I've observed some odd behaviour with dsync replication between two hosts, specifically to do with sieve script replication.[?]> Has anyone else experienced the replication problem? Are sieve scripts actually replicating in live time for other 2.2.24/2.2.25 users as well? For me I didn't notice this till I went looking so I wonder if other people are experiencing this but just not aware of it yet...?Welcome to the club: http://dovecot.org/list/dovecot/2016-July/105014.html If I am not mistaken you are the fourth now reporting this issue. No solution yet. Regards, Michael
Op 7/31/2016 om 4:27 AM schreef Reuben Farrelly:> Hi, > > I've observed some odd behaviour with dsync replication between two > hosts, specifically to do with sieve script replication. > > In short, I have two hosts which replicate in a master-master type > setup where almost all of the reads and writes happen to just one of > the two hosts. > > They are both running 2.2.devel (9dc6403), which is close to the > latest 2.2 -git . Pigeonhole is running master-0.4 . This is on > x86_64 Gentoo. > > Normal mail replication between Maildir's for all users works fine, > however it appears that something recently committed to the code has > broken sieve script replication between the two. I am sure this did > once work. Replication is via tcps: . > > Sieve scripts on the lesser-used host are not up to date by an order > of days/weeks with the main host and they don't seem to re-replicate - > even if the rules don't exist at all on the replica. > > The symptoms and effects look to be the same as this (unanswered) post > from December: > > http://dovecot.org/list/dovecot/2015-December/102690.html > > I am not sure how to view the transaction log files, but I am seeing > the same symptoms, ie no live replication, and on the lesser-used host > almost all the scripts were old and some had the 1970 date on them. > > Even after forcing a [dsync replication replicate '*'] the scripts are > not replicated. As it stands now there are no sieve scripts on one of > the two members and the system seems unable to replicate by itself. > > Secondly, I am also seeing some doubled up outputs if I run 'doveadm > sieve list -A': > > thunderstorm reuben # doveadm sieve list -A > reuben rules ACTIVE > liam rules ACTIVE > kaylene rules ACTIVE > reuben rules ACTIVE > liam rules ACTIVE > kaylene rules ACTIVE > ... > > Has anyone else experienced the replication problem? Are sieve > scripts actually replicating in live time for other 2.2.24/2.2.25 > users as well? For me I didn't notice this till I went looking so I > wonder if other people are experiencing this but just not aware of it > yet...?I will look at this more soon. Regards, Stephan.
On 1/08/2016 10:01 AM, Stephan Bosch wrote:> Op 7/31/2016 om 4:27 AM schreef Reuben Farrelly: >> Hi, >> >> I've observed some odd behaviour with dsync replication between two >> hosts, specifically to do with sieve script replication.>> Has anyone else experienced the replication problem? Are sieve >> scripts actually replicating in live time for other 2.2.24/2.2.25 >> users as well? For me I didn't notice this till I went looking so I >> wonder if other people are experiencing this but just not aware of it >> yet...? > > I will look at this more soon. > > Regards, > > Stephan.Some further information. On the primary host: thunderstorm home # ls -al */sieve/rules.sieve -rw------- 1 user1 user1 3570 Jul 31 11:45 user1/sieve/rules.sieve -rw------- 1 user2 user2 175 Mar 15 2014 user2/sieve/rules.sieve -rw------- 1 user3 user3 725 Jul 31 09:32 user3/sieve/rules.sieve -rw------- 1 user4 user4 0 Jan 1 1970 user4/sieve/rules.sieve -rw------- 1 user5 user5 0 Jan 1 1970 user5/sieve/rules.sieve -rw-r--r-- 1 user6 user6 3719 Jul 31 11:24 user6/sieve/rules.sieve thunderstorm home # On the secondary host: lightning home # ls -al */sieve/rules.sieve -rw------- 1 user1 user1 3570 Jan 1 1970 user1/sieve/rules.sieve -rw------- 1 user2 user2 175 Mar 14 2014 user2/sieve/rules.sieve -rw------- 1 user3 user3 725 Jul 31 07:32 user3/sieve/rules.sieve -rw------- 1 user4 user4 0 Jan 1 1970 user4/sieve/rules.sieve -rw-r--r-- 1 user5 user5 0 Jan 1 1970 user5/sieve/rules.sieve -rw-r--r-- 1 user6 user6 3719 Jan 1 1970 user6/sieve/rules.sieve lightning home # In other words, the rules did eventually get propagated across, and based on the file sizes they are complete. But there is obviously something amiss with handling of dates (which in turn may relate to how the system determines that the file on each server is up to date or not, I guess). In this case the two systems are in different timezones - the primary is GMT+10 and the secondary GMT+8. Also the status of active users is not always replicated either. On one host the output of 'doveadm sieve list -A' shows my own account as ACTIVE but the other host shows all users - except for my account - as being active, and the sieve script for my account is not being replicated. The other interesting thing is the output of: dovecot sieve list -A While (as I said above) the output of this command is doubled up on the main host, it is not doubled up on the secondary host. Reuben
Op 7/31/2016 om 4:27 AM schreef Reuben Farrelly:> Hi, > > I've observed some odd behaviour with dsync replication between two > hosts, specifically to do with sieve script replication. > > In short, I have two hosts which replicate in a master-master type > setup where almost all of the reads and writes happen to just one of > the two hosts. > > They are both running 2.2.devel (9dc6403), which is close to the > latest 2.2 -git . Pigeonhole is running master-0.4 . This is on > x86_64 Gentoo. > > Normal mail replication between Maildir's for all users works fine, > however it appears that something recently committed to the code has > broken sieve script replication between the two. I am sure this did > once work. Replication is via tcps: . > > Sieve scripts on the lesser-used host are not up to date by an order > of days/weeks with the main host and they don't seem to re-replicate - > even if the rules don't exist at all on the replica. > > The symptoms and effects look to be the same as this (unanswered) post > from December: > > http://dovecot.org/list/dovecot/2015-December/102690.html > > I am not sure how to view the transaction log files, but I am seeing > the same symptoms, ie no live replication, and on the lesser-used host > almost all the scripts were old and some had the 1970 date on them. > > Even after forcing a [dsync replication replicate '*'] the scripts are > not replicated. As it stands now there are no sieve scripts on one of > the two members and the system seems unable to replicate by itself.The following bugs were fixed recently: https://github.com/dovecot/core/commit/b4adb461ce12bf578d2d70806b205cf3cbf1a51d https://github.com/dovecot/core/commit/27ccbb0f36e07141785db94557afb63a2aa9eeba I wonder whether this also applies to your problem. Regards, Stephan.