dovecot - Sep 2013 - dovecot 2.2.x and replication with dsync and shared / public namespace

Hi,

I'm trying to configure replication with dsync on Dovecot 2.2.5 (and 
latest nightly).

I did followed the guide at http://wiki2.dovecot.org/Replication

But I can get dsync-server working only if my users do not have access 
to shared mailboxes and / or public mailboxes which is problematic for 
our setup.

dsync-server is trying to create folders in public/ namespace :
-------------------------------------
2013-09-23 20:34:04 dsync-server(adupont): Panic: file 
dsync-brain-mailbox-tree.c: line 384 
(dsync_brain_mailbox_tree_add_delete): assertion failed: (other_node->ns 
== NULL || other_node->ns == node->ns)
2013-09-23 20:34:04 dsync-server(adupont): Error: Raw backtrace: 
/usr/local/lib/dovecot/libdovecot.so.0(+0x64faa) [0x7f7bf42dcfaa] -> 
/usr/local/lib/dovecot/libdovecot.so.0(+0x64fee) [0x7f7bf42dcfee] -> 
/usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f7bf4299299] -> 
dovecot/doveadm-server() [0x41ba52] -> 
dovecot/doveadm-server(dsync_brain_recv_mailbox_tree_deletes+0xbb) 
[0x41c39b] -> dovecot/doveadm-server(dsync_brain_run+0x37c) [0x41972c] 
-> dovecot/doveadm-server() [0x419bf8] -> dovecot/doveadm-server() 
[0x42ba20] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) 
[0x7f7bf42ed3d6] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xaf) 
[0x7f7bf42ee23f] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
[0x7f7bf42ecf18] -> dovecot/doveadm-server() [0x416e59] -> 
dovecot/doveadm-server() [0x40cc77] -> dovecot/doveadm-server() 
[0x415ddd] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) 
[0x7f7bf42ed3d6] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xaf) 
[0x7f7bf42ee23f] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
[0x7f7bf42ecf18] -> 
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7f7bf429e553] -> dovecot/doveadm-server(main+0x11b) [0x40c9eb] -> 
/lib64/libc.so.6(__libc_start_main+0xed) [0x7f7bf3f0932d] -> 
dovecot/doveadm-server() [0x40ca59]
2013-09-23 20:34:04 dsync-server(adupont): Fatal: master: 
service(doveadm): child 28131 killed with signal 6 (core dumped)
-------------------------------------
dsync-server is trying to create folders in shared/ namespace :
-------------------------------------
2013-09-23 20:34:05 dsync-server(s.durant): Panic: file 
dsync-mailbox-tree-sync.c: line 1029 (sync_create_mailboxes): assertion 
failed: (node->ns == other_node->ns)
2013-09-23 20:34:05 dsync-server(s.durant): Error: Raw backtrace: 
/usr/local/lib/dovecot/libdovecot.so.0(+0x64faa) [0x7ffb6fd74faa] -> 
/usr/local/lib/dovecot/libdovecot.so.0(+0x64fee) [0x7ffb6fd74fee] -> 
/usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7ffb6fd31299] -> 
dovecot/doveadm-server() [0x426025] -> 
dovecot/doveadm-server(dsync_mailbox_trees_sync_init+0x139) [0x427789] 
-> dovecot/doveadm-server(dsync_brain_recv_mailbox_tree_deletes+0xdd) 
[0x41c3bd] -> dovecot/doveadm-server(dsync_brain_run+0x37c) [0x41972c] 
-> dovecot/doveadm-server() [0x419bf8] -> dovecot/doveadm-server() 
[0x42ba20] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) 
[0x7ffb6fd853d6] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xaf) 
[0x7ffb6fd8623f] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
[0x7ffb6fd84f18] -> dovecot/doveadm-server() [0x416e59] -> 
dovecot/doveadm-server() [0x40cc77] -> dovecot/doveadm-server() 
[0x415ddd] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) 
[0x7ffb6fd853d6] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xaf) 
[0x7ffb6fd8623f] -> 
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
[0x7ffb6fd84f18] -> 
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7ffb6fd36553] -> dovecot/doveadm-server(main+0x11b) [0x40c9eb] -> 
/lib64/libc.so.6(__libc_start_main+0xed) [0x7ffb6f9a132d] -> 
dovecot/doveadm-server() [0x40ca59]
2013-09-23 20:34:05 dsync-server(s.durant): Fatal: master: 
service(doveadm): child 28137 killed with signal 6 (core dumped)
-------------------------------------

If I run the command "doveadm -v sync -u my.user -f
tcp:10.20.0.2:12345"
mailboxes are sync correctly. No problem with public and shared mailboxes.

If i run the command doveadm replicator replicate 's.durant' or doveadm 
replicator replicate '*', I get plenty of Panic / Error for all users 
that can access Public mailboxes or Shared mailboxes.

Regards,
MARIADASSOU Romain

Replying to my self :

Dovecot 2.2 seems to have problem to replicate shared folders / public 
folders.

To workaround, I had to modify source file 
"src/replication/replicator/dsync-client.c" and change line 210 :
   str_printfa(cmd, "\tsync\t-d\t-N\t-l\t%u",
DSYNC_LOCK_TIMEOUT_SECS);
To :
   str_printfa(cmd, "\tsync\t-d\t-l\t%u", DSYNC_LOCK_TIMEOUT_SECS);

This way, replicator is using command :
doveadm sync -u USERNAME -d -l 30 -U -s
instead of
doveadm sync -u USERNAME -d -N -l 30 -U -s
(note : -N enable to sync all Namespace)
Latest command would sync all user's Namespace (private / shared / 
public) and will failed, and user's mailboxes won't be replicated.

An option in replicator plugin's could be usefull to enable / disable 
sync of shared folders, sync of public folders, and sync of user's 
mailboxes.

MARIADASSOU Romain

Le 23/09/2013 22:40, romain a ?crit :
> Hi,
>
> I'm trying to configure replication with dsync on Dovecot 2.2.5 (and 
> latest nightly).
>
> I did followed the guide at http://wiki2.dovecot.org/Replication
>
> But I can get dsync-server working only if my users do not have access 
> to shared mailboxes and / or public mailboxes which is problematic for 
> our setup.
>
> dsync-server is trying to create folders in public/ namespace :
> -------------------------------------
> 2013-09-23 20:34:04 dsync-server(adupont): Panic: file 
> dsync-brain-mailbox-tree.c: line 384 
> (dsync_brain_mailbox_tree_add_delete): assertion failed: 
> (other_node->ns == NULL || other_node->ns == node->ns)
> 2013-09-23 20:34:04 dsync-server(adupont): Error: Raw backtrace: 
> /usr/local/lib/dovecot/libdovecot.so.0(+0x64faa) [0x7f7bf42dcfaa] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(+0x64fee) [0x7f7bf42dcfee] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f7bf4299299] -> 
> dovecot/doveadm-server() [0x41ba52] -> 
> dovecot/doveadm-server(dsync_brain_recv_mailbox_tree_deletes+0xbb) 
> [0x41c39b] -> dovecot/doveadm-server(dsync_brain_run+0x37c) [0x41972c] 
> -> dovecot/doveadm-server() [0x419bf8] -> dovecot/doveadm-server() 
> [0x42ba20] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) 
> [0x7f7bf42ed3d6] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xaf) 
> [0x7f7bf42ee23f] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
> [0x7f7bf42ecf18] -> dovecot/doveadm-server() [0x416e59] -> 
> dovecot/doveadm-server() [0x40cc77] -> dovecot/doveadm-server() 
> [0x415ddd] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) 
> [0x7f7bf42ed3d6] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xaf) 
> [0x7f7bf42ee23f] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
> [0x7f7bf42ecf18] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
> [0x7f7bf429e553] -> dovecot/doveadm-server(main+0x11b) [0x40c9eb] -> 
> /lib64/libc.so.6(__libc_start_main+0xed) [0x7f7bf3f0932d] -> 
> dovecot/doveadm-server() [0x40ca59]
> 2013-09-23 20:34:04 dsync-server(adupont): Fatal: master: 
> service(doveadm): child 28131 killed with signal 6 (core dumped)
> -------------------------------------
> dsync-server is trying to create folders in shared/ namespace :
> -------------------------------------
> 2013-09-23 20:34:05 dsync-server(s.durant): Panic: file 
> dsync-mailbox-tree-sync.c: line 1029 (sync_create_mailboxes): 
> assertion failed: (node->ns == other_node->ns)
> 2013-09-23 20:34:05 dsync-server(s.durant): Error: Raw backtrace: 
> /usr/local/lib/dovecot/libdovecot.so.0(+0x64faa) [0x7ffb6fd74faa] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(+0x64fee) [0x7ffb6fd74fee] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7ffb6fd31299] -> 
> dovecot/doveadm-server() [0x426025] -> 
> dovecot/doveadm-server(dsync_mailbox_trees_sync_init+0x139) [0x427789] 
> -> dovecot/doveadm-server(dsync_brain_recv_mailbox_tree_deletes+0xdd) 
> [0x41c3bd] -> dovecot/doveadm-server(dsync_brain_run+0x37c) [0x41972c] 
> -> dovecot/doveadm-server() [0x419bf8] -> dovecot/doveadm-server() 
> [0x42ba20] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) 
> [0x7ffb6fd853d6] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xaf) 
> [0x7ffb6fd8623f] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
> [0x7ffb6fd84f18] -> dovecot/doveadm-server() [0x416e59] -> 
> dovecot/doveadm-server() [0x40cc77] -> dovecot/doveadm-server() 
> [0x415ddd] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) 
> [0x7ffb6fd853d6] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xaf) 
> [0x7ffb6fd8623f] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
> [0x7ffb6fd84f18] -> 
> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
> [0x7ffb6fd36553] -> dovecot/doveadm-server(main+0x11b) [0x40c9eb] -> 
> /lib64/libc.so.6(__libc_start_main+0xed) [0x7ffb6f9a132d] -> 
> dovecot/doveadm-server() [0x40ca59]
> 2013-09-23 20:34:05 dsync-server(s.durant): Fatal: master: 
> service(doveadm): child 28137 killed with signal 6 (core dumped)
> -------------------------------------
>
> If I run the command "doveadm -v sync -u my.user -f 
> tcp:10.20.0.2:12345" mailboxes are sync correctly. No problem with 
> public and shared mailboxes.
>
> If i run the command doveadm replicator replicate 's.durant' or 
> doveadm replicator replicate '*', I get plenty of Panic / Error for
> all users that can access Public mailboxes or Shared mailboxes.
>
> Regards,
> MARIADASSOU Romain