I discovered, while trying to set up Tcl bindings for Notmuch (https://notmuchmail.org/), which uses Xapian, that flintlock was not being locked (I had lost updates). I then found that opening a Xapian database for writing directly via the Xapian Tcl bindings also silently fails to lock flintlock. I have taken a copy of flint_lock.cc to play with, and I find that it locks the file when called from a minimal C main program, but not when bound as a Tcl extension. What happens is that fcntl with F_SETLK fails to get the lock, but reports success. If anyone is using the Tcl bindings you might need to be aware of this. Has anyone seen this with the Tcl (or any other) bindings? (Or indeed in any other context?) Thanks, Eric -- ms fnd in a lbry
On Sun, 21 Feb 2016 14:15:25 +0100 (CET), Eric J <eric at deptj.eu> wrote:> I discovered, while trying to set up Tcl bindings for Notmuch > (https://notmuchmail.org/), which uses Xapian, that flintlock was not > being locked (I had lost updates).8>< --------> What happens is that fcntl with F_SETLK fails to get the lock, but > reports success.Actually I have now found that the fcntl works, it is the execl which loses the lock! Eric -- ms fnd in a lbry
On Sun, Feb 21, 2016 at 02:15:25PM +0100, Eric J wrote:> I discovered, while trying to set up Tcl bindings for Notmuch > (https://notmuchmail.org/), which uses Xapian, that flintlock was not > being locked (I had lost updates).It seems to work for me, testing with this: package require Tcl 8 package require xapian 1.0.0 xapian::WritableDatabase db "tmp.db" $xapian::DB_CREATE_OR_OPEN xapian::WritableDatabase db2 "tmp.db" $xapian::DB_CREATE_OR_OPEN I wonder if the problem is unrelated to locking, but instead it's that the Tcl database doesn't get explicitly destroyed in your script, so that the C++ object doesn't either, and the changes don't get committed. I would try calling close() on the WritableDatabase object before your script exits. There's some discussion of this in the Tcl bindings docs (section "Destructors"): https://xapian.org/docs/bindings/tcl8/ Cheers, Olly
On Sun, 21 Feb 2016 22:33:22 +0000, Olly Betts <olly at survex.com> wrote:> On Sun, Feb 21, 2016 at 02:15:25PM +0100, Eric J wrote: > > I discovered, while trying to set up Tcl bindings for Notmuch > > (https://notmuchmail.org/), which uses Xapian, that flintlock was not > > being locked (I had lost updates). > > It seems to work for me, testing with this: > > package require Tcl 8 > package require xapian 1.0.0 > xapian::WritableDatabase db "tmp.db" $xapian::DB_CREATE_OR_OPEN > xapian::WritableDatabase db2 "tmp.db" $xapian::DB_CREATE_OR_OPENeric at bruno [ ~ ]$ cat /proc/version Linux version 3.13.300 (root at bruno) (gcc version 4.8.2 (GCC) ) #2 SMP Tue Sep 16 21:01:43 BST 2014 eric at bruno [ ~ ]$ tclsh % info patchlevel 8.6.1 % package require Tcl 8 8.6.1 % package require xapian 1.0.0 1.2.18 % xapian::WritableDatabase db "tmp.db" $xapian::DB_CREATE_OR_OPEN _e0c4b00000000000_p_Xapian__WritableDatabase % xapian::WritableDatabase db2 "tmp.db" $xapian::DB_CREATE_OR_OPEN _f0d3b00000000000_p_Xapian__WritableDatabase % At which point eric at bruno [ ~ ]$ lsof tmp.db/flintlock COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME cat 13543 eric 5w REG 8,9 0 930437 tmp.db/flintlock cat 13552 eric 9w REG 8,9 0 930437 tmp.db/flintlock Blaming the execl is due to stepping though my copy of the lock code in gdb, and seeing, in lsof, 5w on the open, still 5w on the fork, 5ww on the fcntl, and 5w again on the execl.> I wonder if the problem is unrelated to locking, but instead it's that > the Tcl database doesn't get explicitly destroyed in your script, so > that the C++ object doesn't either, and the changes don't get committed. > > I would try calling close() on the WritableDatabase object before your > script exits.I did have a close in my script, and then added a destroy. No way of telling if it was first-to-close or last-to-close that was being lost.> There's some discussion of this in the Tcl bindings docs (section > "Destructors"): > > https://xapian.org/docs/bindings/tcl8/Thanx, Eric -- ms fnd in a lbry
cf: http://permalink.gmane.org/gmane.comp.search.xapian.general/9965> Eric J <eric <at> deptj.eu> wrote:...> Earlier 8.5.x are presumably the same as 8.5.18.If someone might post one or more code samples (incl. instructions for compiling, if relevant) and a manner of checking the following: 1) "database locks" with Tcl bindings aren't functioning 2) "database locks" with Tcl bindings function correctly if I find the time to test on GNU/Linux (32 bit) tcl8.5.11 the specified xapian-core version number (1.2.18), I will post the results.
cf: http://permalink.gmane.org/gmane.comp.search.xapian.general/9970 http://mirrors.slackware.com/slackware/slackware-iso/slackware-14.0-iso/ slackware-14.0-install-dvd.iso http://mirrors.slackware.com/slackware/slackware-14.0/source/tcl/tcl/tcl.SlackBuild FLAGS and configure arguments http://mirrors.slackware.com/slackware/slackware-current/source/l/xapian-core/xapian-core.SlackBuild FLAGS and configure arguments http://xapian.org/download xapian-bindings-1.2.22.tar.xz Terminal session 1: bash-4.2# tclsh % info patchlevel 8.5.11 % package require Thread can't find package Thread % package require xapian 1.0.0 1.2.22 % xapian::WritableDatabase db "tmp.db" $xapian::DB_CREATE_OR_OPEN _88a70f09_p_Xapian__WritableDatabase % Terminal session 2: (with the same working directory) bash-4.2# tclsh % info patchlevel 8.5.11 % package require Thread can't find package Thread % package require xapian 1.0.0 1.2.22 % xapian::WritableDatabase db2 "tmp.db" $xapian::DB_CREATE_OR_OPEN Unable to get write lock on tmp.db: already locked % Terminal session 3: (with the same working directory) bash-4.2# lsof -r 5 tmp.db/flintlock COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME cat 5630 root 5ww REG 8,3 0 3858547 tmp.db/flintlock ======COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME cat 5630 root 5ww REG 8,3 0 3858547 tmp.db/flintlock ======^C bash-4.2#