Hi Experts I''m trying to determine whether SQLite will properely work on Lustre. From a quick search it seems that some people use it successfully using flock/localflock mount option. But I have doubts. I''m interested in 2 scenarios: - SQLite works properely for single Lustre node. - SQLite files are accessed simultanously by processes on more than one node. In SQLite documentation (http://www.sqlite.org/lockingv3.html) there is statement that this db engine uses advisory locks to protect shared date. I know Lustre doesn''t implement this yet. So my questions are: - Am I right suspecting that for current Lustre version (1.6.6) there is real risk of SQLite files corruption (in a sense of their integrity - not from fs point of view) even if using flock option? - Are advisory locks still planned to come with 1.8 version (as it is indicated in current Lustre Operation Manual, but there is no such feature in November Roadmap slide on wiki.lustre.org site)? Thanks in advance. Best Regards Michal Wesolowski
Hello! On Feb 11, 2009, at 4:19 PM, Michal Wesolowski wrote:> In SQLite documentation (http://www.sqlite.org/lockingv3.html) there > is > statement that this db engine uses advisory locks to protect shared > date. I know > Lustre doesn''t implement this yet. So my questions are: > - Am I right suspecting that for current Lustre version (1.6.6) > there is real > risk of SQLite files corruption (in a sense of their integrity - not > from fs > point of view) even if using flock option? > - Are advisory locks still planned to come with 1.8 version (as it > is indicated > in current Lustre Operation Manual, but there is no such feature in > November > Roadmap slide on wiki.lustre.org site)?Lustre supports posix advisory locks. You can enable it with -o flock mount option (on all clients). -o localflock is posix advisory locks with no cluster coherency, so it is unsafe for the same database used across several nodes. Bye, Oleg
Oleg Drokin wrote:> Hello! > > On Feb 11, 2009, at 4:19 PM, Michal Wesolowski wrote: >> In SQLite documentation (http://www.sqlite.org/lockingv3.html) there is >> statement that this db engine uses advisory locks to protect shared >> date. I know >> Lustre doesn''t implement this yet. So my questions are: >> - Am I right suspecting that for current Lustre version (1.6.6) there >> is real >> risk of SQLite files corruption (in a sense of their integrity - not >> from fs >> point of view) even if using flock option? >> - Are advisory locks still planned to come with 1.8 version (as it is >> indicated >> in current Lustre Operation Manual, but there is no such feature in >> November >> Roadmap slide on wiki.lustre.org site)? > > Lustre supports posix advisory locks. You can enable it with -o flock > mount option > (on all clients). > -o localflock is posix advisory locks with no cluster coherency, so it > is unsafe > for the same database used across several nodes. > > Bye, > OlegThanks Oleg My interpretation of Lustre documentation was that flock/localflock implements only mandatory locks. So I assume statement in Lustre Manula (ver. 15, page 16-1): Note ? Advisory fcntl/flock/lockf locks will be available in Lustre 1.8. is a documentation bug, isn''t it? Thanks Michal
Hello! On Feb 12, 2009, at 2:10 AM, Michal Wesolowski wrote:> My interpretation of Lustre documentation was that flock/localflock > implements only mandatory locks.There is no such thing as posix mandatory locking, I think. The existing mandatory locking depends on a combination of advisory locking and kernel support in interpretation of a special file mode set.> So I assume statement in Lustre Manula (ver. 15, page 16-1): > Note ? Advisory fcntl/flock/lockf locks will be available in Lustre > 1.8. > is a documentation bug, isn''t it?Yes, this is somewhat outdated. 1.6.6 and above have posix advisory locking working (though performance is not ideal, especially if you have many clients doing a lot of locks on the same file at the same time). Bye, Oleg
On Feb 11, 2009 22:19 +0100, Michal Wesolowski wrote:> I''m trying to determine whether SQLite will properely work on Lustre. From a > quick search it seems that some people use it successfully using > flock/localflock mount option. But I have doubts. > I''m interested in 2 scenarios: > - SQLite works properely for single Lustre node. > - SQLite files are accessed simultanously by processes on more than one node.I think that is only between processes on the same node. I think accessing SQLite between different processes on different nodes might result in file corruption by SQLite itself, regardless of the locking by Lustre. If you want to test this, you definitely need to mount all clients with "-o flock" so the locking is coherent across all clients.> In SQLite documentation (http://www.sqlite.org/lockingv3.html) there is > statement that this db engine uses advisory locks to protect shared date.6.0 How To Corrupt Your Database Files SQLite uses POSIX advisory locks to implement locking on Unix... Your best defense is to not use SQLite for files on a network filesystem. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
On Thu, 2009-02-12 at 02:30 -0500, Andreas Dilger wrote:> > I think that is only between processes on the same node. I think accessing > SQLite between different processes on different nodes might result in file > corruption by SQLite itself, regardless of the locking by Lustre.Can you expand on why you think this might be the case Andreas? If the locking works, and the application obeys the locks, what avenues do you see for corruption?> 6.0 How To Corrupt Your Database Files > > SQLite uses POSIX advisory locks to implement locking on Unix... > Your best defense is to not use SQLite for files on a network filesystem.Andreas, do you think there is any scientific data behind this kind of statement or do you think it''s just FUD given NFS'' long history of broken file locking? b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090212/c4e693d8/attachment.bin
Oleg Drokin wrote:> Hello! > > On Feb 11, 2009, at 4:19 PM, Michal Wesolowski wrote: >> In SQLite documentation (http://www.sqlite.org/lockingv3.html) there is >> statement that this db engine uses advisory locks to protect shared >> date. I know >> Lustre doesn''t implement this yet. So my questions are: >> - Am I right suspecting that for current Lustre version (1.6.6) there >> is real >> risk of SQLite files corruption (in a sense of their integrity - not >> from fs >> point of view) even if using flock option? >> - Are advisory locks still planned to come with 1.8 version (as it is >> indicated >> in current Lustre Operation Manual, but there is no such feature in >> November >> Roadmap slide on wiki.lustre.org site)? > > Lustre supports posix advisory locks. You can enable it with -o flock > mount option > (on all clients). > -o localflock is posix advisory locks with no cluster coherency, so it > is unsafe > for the same database used across several nodes. > > Bye, > OlegThanks Oleg My interpretation of Lustre documentation was that flock/localflock implements only mandatory locks. So I assume statement in Lustre Manula (ver. 15, page 16-1): Note ? Advisory fcntl/flock/lockf locks will be available in Lustre 1.8. is a documentation bug, isn''t it? Thanks Michal
Andreas Dilger wrote:> On Feb 11, 2009 22:19 +0100, Michal Wesolowski wrote: >> I''m trying to determine whether SQLite will properely work on Lustre. From a >> quick search it seems that some people use it successfully using >> flock/localflock mount option. But I have doubts. >> I''m interested in 2 scenarios: >> - SQLite works properely for single Lustre node. >> - SQLite files are accessed simultanously by processes on more than one node. > > I think that is only between processes on the same node. I think accessing > SQLite between different processes on different nodes might result in file > corruption by SQLite itself, regardless of the locking by Lustre. > > If you want to test this, you definitely need to mount all clients with > "-o flock" so the locking is coherent across all clients. > >> In SQLite documentation (http://www.sqlite.org/lockingv3.html) there is >> statement that this db engine uses advisory locks to protect shared date. > > > 6.0 How To Corrupt Your Database Files > > SQLite uses POSIX advisory locks to implement locking on Unix... > Your best defense is to not use SQLite for files on a network filesystem.I saw this too, but my impression was that this remark apply to previous deliberations about buggy implementation of advisory locks on some NFS systems. regards michal
Thanks Oleg ! I have subquestion regarding locking performance, please see below. Oleg Drokin wrote:> Hello! > > On Feb 12, 2009, at 2:10 AM, Michal Wesolowski wrote: >> My interpretation of Lustre documentation was that flock/localflock >> implements only mandatory locks. > > There is no such thing as posix mandatory locking, I think. > The existing mandatory locking depends on a combination of advisory > locking and kernel support in > interpretation of a special file mode set. > >> So I assume statement in Lustre Manula (ver. 15, page 16-1): >> Note ? Advisory fcntl/flock/lockf locks will be available in Lustre 1.8. >> is a documentation bug, isn''t it? > > Yes, this is somewhat outdated. > 1.6.6 and above have posix advisory locking working (though performance > is not ideal, > especially if you have many clients doing a lot of locks on the same > file at the same > time).Do you mean file locking mechanism in Lustre doesn''t perform as efficiently as on local system or is not yet optimized? Or is there any general impact on Lustre when at least one node has flock option turned on? For example additional burden on MDS or OSSes which results in lower IO/s performance. Regards Michal
Hello! On Feb 12, 2009, at 3:29 PM, Michal Wesolowski wrote:>> Yes, this is somewhat outdated. >> 1.6.6 and above have posix advisory locking working (though >> performance is not ideal, >> especially if you have many clients doing a lot of locks on the >> same file at the same >> time). > Do you mean file locking mechanism in Lustre doesn''t perform as > efficiently as on local system or is not yet optimized? Or is there > any general impact on Lustre when at least one node has flock option > turned on? For example additional burden on MDS or OSSes which > results in lower IO/s performance.It is both not very optimized and slower than local system since it needs to send network rpcs for locking (Except for the localflock which is same speed as for local fs). The performance impact would only be realized if not only you mount with -o flock, but actually use posix locking, and mostly only in those operations. There is no impact on OSS operations at all. The only MDS impact is there is now a bit more processing to handle those locking RPCs. Bye, Oleg
Oleg, thanks for your support m. Oleg Drokin wrote:> Hello! > > On Feb 12, 2009, at 3:29 PM, Michal Wesolowski wrote: >>> Yes, this is somewhat outdated. >>> 1.6.6 and above have posix advisory locking working (though >>> performance is not ideal, >>> especially if you have many clients doing a lot of locks on the >>> same file at the same >>> time). >> Do you mean file locking mechanism in Lustre doesn''t perform as >> efficiently as on local system or is not yet optimized? Or is there >> any general impact on Lustre when at least one node has flock option >> turned on? For example additional burden on MDS or OSSes which >> results in lower IO/s performance. > > It is both not very optimized and slower than local system since it > needs to send network rpcs for locking (Except for the localflock > which is same speed as for local fs). > The performance impact would only be realized if not only you mount > with -o flock, but actually use posix locking, and mostly only in > those operations. > There is no impact on OSS operations at all. The only MDS impact is > there is now a bit more processing to handle those locking RPCs. > > Bye, > Oleg > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss