samba - Dec 2016 - Is it fine to store the tdb files in the shared directory to solve the ctdb data synchronization problem?

Hi folks
   I face a problem about the data synchronization when using the ctdb. When one
ctdb node fails, other ctdb nodes may not recover the data of the failed node. I
think it may be due to the fact that ctdb stores its tdb files in the local
storage. So, is it fine to store the tdb files in the shared directory of the
backend storage? When one node fails, other nodes may recover the data from the
shared directory. Could this solve the ctdb data synchronization problem? Are
there any problems with this solution?
   Thanks.
-------------------------------------------------------------------------------------------------------------------------------------
本邮件及其附件含有杭州华三通信技术有限公司的保密信息，仅限于发送给上面地址中列出
的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、
或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本
邮件！
This e-mail and its attachments contain confidential information from H3C, which
is
intended only for the person or entity whose address is listed above. Any use of
the
information contained herein in any way (including, but not limited to, total or
partial
disclosure, reproduction, or dissemination) by persons other than the intended
recipient(s) is prohibited. If you receive this e-mail in error, please notify
the sender
by phone or email immediately and delete it!

On Fri, Dec 02, 2016 at 10:45:53AM +0000, Chenyehua via samba
wrote:
>    I face a problem about the data synchronization when using the
>    ctdb. When one ctdb node fails, other ctdb nodes may not recover
>    the data of the failed node. I think it may be due to the fact
>    that ctdb stores its tdb files in the local storage. So, is it
>    fine to store the tdb files in the shared directory of the
>    backend storage? When one node fails, other nodes may recover the
>    data from the shared directory. Could this solve the ctdb data
>    synchronization problem? Are there any problems with this
>    solution?
The architecture of ctdb is such that it is supposed to lose the data
that is stored on the local node in the tdb. For example information
on open files is worthless if the node crashes, the files are closed
by definition.

ctdb was explicitly designed to not store the tdb data on shared
storage. Shared storage is not made for this workload: Heavy locking,
tons of small writes all around in the file, high consistency
requirements. This is nothing you want remote.

What data do you need to synchronize?

With best regards,

Volker Lendecke

Thanks very much, Volker.

I want to synchronize three tdb files(listed as below) which is relevant to the
opened file:
1) brlock.tdb
2) locking.tdb
3) smbXsrv_open_global.tdb
These three tdb files are used to realize the durable file handle feature.

As what you have said, the performance will be affected if storing these three
files in the shared storage.
So can I realize the synchronization with the solutions below:
1) set these three tdb files as persistent tdb files to realize synchronization
(Is this way even worse than storing in the shared storage when concerning the
performance affecting?)
2) Real-time backing up the modification of the normal tdb to other ctdb
nodes(just a simple guess, not test yet)
Do you have any other solutions or suggestions about this data synchronization
problem?

Looking forward to your reply, thanks.
Best regards

-----邮件原件-----
发件人: vlendec at samba.org [mailto:vlendec at samba.org] 代表 Volker Lendecke
发送时间: 2016年12月2日 18:55
收件人: chenyehua 11692 (RD)
抄送: samba at lists.samba.org
主题: Re: [Samba] Is it fine to store the tdb files in the shared directory to
solve the ctdb data synchronization problem?

On Fri, Dec 02, 2016 at 10:45:53AM +0000, Chenyehua via samba
wrote:
>    I face a problem about the data synchronization when using the
>    ctdb. When one ctdb node fails, other ctdb nodes may not recover
>    the data of the failed node. I think it may be due to the fact
>    that ctdb stores its tdb files in the local storage. So, is it
>    fine to store the tdb files in the shared directory of the
>    backend storage? When one node fails, other nodes may recover the
>    data from the shared directory. Could this solve the ctdb data
>    synchronization problem? Are there any problems with this
>    solution?
The architecture of ctdb is such that it is supposed to lose the data that is
stored on the local node in the tdb. For example information on open files is
worthless if the node crashes, the files are closed by definition.

ctdb was explicitly designed to not store the tdb data on shared storage. Shared
storage is not made for this workload: Heavy locking, tons of small writes all
around in the file, high consistency requirements. This is nothing you want
remote.

What data do you need to synchronize?

With best regards,

Volker Lendecke
-------------------------------------------------------------------------------------------------------------------------------------
本邮件及其附件含有杭州华三通信技术有限公司的保密信息，仅限于发送给上面地址中列出
的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、
或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本
邮件！
This e-mail and its attachments contain confidential information from H3C, which
is
intended only for the person or entity whose address is listed above. Any use of
the
information contained herein in any way (including, but not limited to, total or
partial
disclosure, reproduction, or dissemination) by persons other than the intended
recipient(s) is prohibited. If you receive this e-mail in error, please notify
the sender
by phone or email immediately and delete it!