Thanks Tim,
You are correct the /mnt/lustre is the lustre share, and /share/mnt is
an NFS share where the CTDB_RECOVERY_LOCK file is stored for the 3 ctdb
services running on 3 clients.
I think that the problem could have something to do with the
--with-privatedir
I wasn''t sure what ''private dir'' should point to in a
cluster environment.
My ctdb tdb''s are located at /usr/local/var/ctdb/
My Samba secret.tdb is located at /etc/samba/ so should the
--with-privatedir point to /etc/samba ?
And am I correct in using the following when configuring samba
--with-lockdir=/usr/local/var/ctdb ?
Regards
Darren George
Linux Server Administrator
Patersons Asia PTE Ltd.20 Science Park Road, #01-04/05 Teletech Park,
Singapore Science Park II, Singapore 117674. http://www.patersons.net/
Tel: +65 6403 5900
CONFIDENTIALITY: This e-mail and any attachment is confidential. If you
are not the intended recipient or you receive this e-mail in error,
please notify the sender immediately and do not disclose the contents to
another person or take or print copies or use it for any other purpose,
but delete it immediately.
VIRUSES: We offer no guarantees that this e-mail or any attachment will
be received error or virus free and accept no responsibility whatever in
this regard.
SECURITY: This transmission is not guaranteed as secure. We do not
accept liability for any errors or omissions in the contents of this
e-mail which results from this transmission.
Registered in England under number 3579805. Registered address as above
Tim Wickberg wrote:> It''s a little hard to tell, but I''m assuming that
/mnt/lustre is the
> lustre mount, and /share/mnt is either kept to each machine (but the
> name seems to imply that it isn''t) or is shared storage that all
three
> machines are seeing (but that isn''t the lustre pool?).
>
> Either way, I''m guessing that either:
>
> - Your --private-dir isn''t very private (and is on shared storage
at
> /share/mnt/ )
> - Or, your CTDB_RECOVERY_LOCK is too private.
>
> The --with-privatedir directory needs to be local to the machine - if
> all three ctdb''s are trying to maintain their local state on a
shared
> FS things would probably not behave normally.
>
> CTDB_RECOVERY_LOCK need to be on shared storage (preferably on the
> lustre pool) to have the three ctdbd''s communicate. (Is ctdbd
running
> on each? Any debug messages from it?)
>
> You may also want to watch out for
>
> https://bugzilla.lustre.org/show_bug.cgi?id=5135
>
> Hope this helps,
> - Tim
>
> Darren George wrote:
>> Hi,
>>
>> I''m having an issue with file locking whist using
lustre+ctdb+samba
>>
>> My Lustre Setup
>> mds/mgs
>> 3 oss
>> 3 Clients each with ctdb and samba services running.
>>
>> All servers and clients running Centos 5 - Kernal 2.6.19-92
>> All servers and clients with Lustre 1.6.6
>> ctdb 3.0
>> Samba 3.2.1
>>
>> CTDB Configuration:
>>
>> ./configure --bindir=/usr/bin --sbindir=/usr/sbin
--libdir=/usr/lib/ctdb
>>
>> /etc/sysconfig/ctdb
>>
>> CTDB_RECOVERY_LOCK="/share/mnt/recovery"
>> CTDB_PUBLIC_ADDRESSES=/etc/ctdb/public_addresses
>> CTDB_MANAGES_SAMBA=yes
>> CTDB_LOGFILE=/var/log/log.ctdb
>>
>> /etc/ctdb/public_addresses
>>
>> 192.168.3.200/24 eth0
>>
>> /etc/ctdb/nodes
>>
>> 192.168.3.173
>> 192.168.2.3
>> 192.168.1.3
>>
>> SAMBA Configuration
>>
>> ./configure --with-ctdb=/etc/ctdb --with-cluster-support
>> --enable-pie=no --bindir=/usr/bin --sbindir=/usr/sbin
>> --libdir=/usr/lib/samba --with-privatedir=/share/mnt/ctdb
>> --with-lockdir=/usr/local/var/ctdb --with-piddir=/var/run
>> --with-swatdir=/usr/share/swat --with-logfilebase=/var/log/samba
>> --with-configdir=/etc/samba
>>
>> Load smb config files from /etc/samba/smb.conf
>> Processing section "[homes]"
>> Processing section "[filepool]"
>> Processing section "[printers]"
>> Loaded services file OK.
>> Server role: ROLE_STANDALONE
>> Press enter to see a dump of your service definitions
>>
>> [global]
>> workgroup = MYGROUP
>> server string = Samba Server Version %v
>> passdb backend = tdbsam
>> use mmap = No
>> clustering = Yes
>> idmap backend = tdb2
>> fileid:mapping = fsname
>> ea support = Yes
>> cups options = raw
>>
>> [filepool]
>> comment = The Shared Area
>> path = /mnt/lustre
>> read only = No
>> create mask = 0777
>> directory mask = 0777
>>
>> All lustre clients have been mounted with the ''-o
flock'' option.
>>
>> Everything from a linux prospective is working fine, my linux clients
>> can read/write to files on the file share and file locking is active.
>>
>> >From a window share prospective clients can read/write to files on
>> the file share but file locking is not active.
>>
>> The only errors I can find from samba logs are:
>>
>> [2009/01/11 02:50:25, 3]
>> locking/locking.c:fetch_share_mode_unlocked(857)
>> fill_share_mode_lock failed
>> [2009/01/11 02:50:25, 10] smbd/statcache.c:stat_cache_lookup(235)
>> stat_cache_lookup: lookup failed for name [SG]
>> [2009/01/11 02:50:33, 5] lib/util_sock.c:read_socket_with_timeout(928)
>> read_socket_with_timeout: blocking read. EOF from client.
>> [2009/01/11 02:50:33, 3] smbd/process.c:smbd_process(2027)
>> receive_message_or_smb failed: NT_STATUS_END_OF_FILE, exiting
>> Has anyone dealt with a similar set-up and can assist with the file
>> locking from a ctdb+samba environment ?
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090116/e4d5c3b7/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 6185 bytes
Desc: not available
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090116/e4d5c3b7/attachment-0001.gif