Locking of Quartz databases is currently done with a lockfile. This
works well from the actual locking side, but the downside is that
the lock isn't released if a process doesn't destroy the
Xapian::WritableDatabase object. This is made worse because some
of the bindings don't call destructors (or don't do it reliably).
The obvious alternative is to use actual locking APIs. On Unix, this
mean fcntl. Windows has a suitable API too.
But the problem is that on Unix, fcntl locks are per process. So if
we used fcntl locking, then if a process tried to open the same database
twice as a WritableDatabase, it would succeed. This is probably more of
a worry in a threaded application, but threads aren't actually needed to
hit this problem.
So that's why we've stuck with lock files so far. But I have a cunning
plan...
We fork() and then fcntl() lock from a different process. Since each
WritableDatabase now has its own locking process, fcntl() locking works!
Now we don't really want to carry round the VM weight of the whole
application in each lock process, as we'll end up eating up swap with
old copies of pages from the application if there are many long lived
WritableDatabase objects. So fork() and exec() makes sense.
I wondered about fork() fcntl() exec("/bin/true") and then just kill
the
child process to release the lock. But we want to be sure that the
child process dies when the parent does, so it might be better to have
our own little helper program. We could open a pipe to the helper,
which then blocks on read or select until the other end of the pipe is
closed either explicitly or by the main process terminating. Hmm, that
sounds a lot like "/bin/cat" actually!
We could perhaps even allow people to avoid the overhead of fork() and
exec() by allowing a hint to be set. So if you tell Xapian that you're
not opening the same database more than once for writing, it'll just
lock in process. It's worth checking if the overhead is actually enough
to be an issue first though.
And on Windows we can just use the locking API.
Cheers,
Olly
This makes a good deal of sense to me. Using "cat" as a helper sounds a reasonable plan - I first thought that it would cause problems if the helper was absent, but /bin/cat is pretty fundamental. Then I thought that some "cat"s might have a fair bit of junk in them which we don't want to bring into memory, but GNU cat is only 16k on my system, and probably being executed pretty often by other things anyway. And though the options to cat vary between systems, that won't be a problem because you won't need to pass any options anyway. So, no objections, if it works. -- Richard Boulton <richard at tartarus.org>
On Fri, Apr 08, 2005 at 01:27:14PM +0100, Olly Betts wrote:> So that's why we've stuck with lock files so far. But I have a cunning > plan... > > We fork() and then fcntl() lock from a different process. Since each > WritableDatabase now has its own locking process, fcntl() locking works!I've now got round to trying this, and it works pretty much as I'd hoped. It needed one slight tweak - we seem to require a bidirectional connection so that the child process can tell the parent if fcntl() failed or not. Without a back connection, the child process can die to say the lock failed but the parent won't be able to tell if the child has got the lock or is still trying. And without a forward connection, the child won't know when the parent exits without explicitly closing the lock. There may be a way around this, but a bidirectional connection isn't really a problem.> I wondered about fork() fcntl() exec("/bin/true") and then just kill the > child process to release the lock. But we want to be sure that the > child process dies when the parent does, so it might be better to have > our own little helper program. We could open a pipe to the helper, > which then blocks on read or select until the other end of the pipe is > closed either explicitly or by the main process terminating. Hmm, that > sounds a lot like "/bin/cat" actually!What I'm currently doing is execl("/bin/cat", NULL); and then if that fails I just enter a loop to read bytes from stdin until EOF.> We could perhaps even allow people to avoid the overhead of fork() and > exec() by allowing a hint to be set. So if you tell Xapian that you're > not opening the same database more than once for writing, it'll just > lock in process. It's worth checking if the overhead is actually enough > to be an issue first though.Here are some preliminary timings for a successful lock and unlock on x86_64 Linux (I timed 10000 iterations using "time"): 98.4-175.2 us Quartz style locking with a lock file 533.2-605.6 us fork/fcntl/exec locking 8.1-10.4 us fnctl alone ("us" being microseconds) Although the fork/fcntl/exec scheme is around 3-6 times slower than quartz, I suspect it's not actually a problem for WritableDatabase becase the lock/unlock still takes only half a millisecond. Now the locking works, I'll see if I can streamline it at all. And also get timings on other platforms. Cheers, Olly
Olly Betts <olly <at> survex.com> writes:> Locking of Quartz databases is currently done with a lockfile. This > works well from the actual locking side, but the downside is that > the lock isn't released if a process doesn't destroy the > Xapian::WritableDatabase object. This is made worse because some > of the bindings don't call destructors (or don't do it reliably). > > The obvious alternative is to use actual locking APIs. On Unix, this > mean fcntl. Windows has a suitable API too. > > But the problem is that on Unix, fcntl locks are per process. So if > we used fcntl locking, then if a process tried to open the same database > twice as a WritableDatabase, it would succeed. This is probably more of > a worry in a threaded application, but threads aren't actually needed to > hit this problem.Olly - I know nothing about the internals of Xapian, but I wonder whether this cunning plan is more complex and expensive than necessary. I wonder why Xapian doesn't apply the flock, then set a flag indicating that the database is locked. Now whenever Xapian goes to open a database, it would first check whether the flag is set. If so, Xapian knows that that the database is already open within this process. If the flag is not set, Xapian continues onward, probing the file with flock to see whether it's open within some other process. This should work in a threaded environment so long as the customary synchronization primitives are used to avoid race conditions.> So that's why we've stuck with lock files so far. But I have a cunning > plan... > > We fork() and then fcntl() lock from a different process. Since each > WritableDatabase now has its own locking process, fcntl() locking works! > > I wondered about fork() fcntl() exec("/bin/true") and then just kill the > child process to release the lock. But we want to be sure that the > child process dies when the parent does, so it might be better to have > our own little helper program. We could open a pipe to the helper, > which then blocks on read or select until the other end of the pipe is > closed either explicitly or by the main process terminating. Hmm, that > sounds a lot like "/bin/cat" actually!