search for: mvcc

I'm one of the developers of Notmuch, an email client that makes extensive use of Xapian. For some time, full multi-version concurrency control (MVCC) has been on our wish list. Olly mentioned that brass now uses free lists---a key step toward full MVCC---and I offered to pitch in on putting the other pieces into place. Before I wade too deep into this, I wanted to propose my plan. Full MVCC would enable Xapian to keep any database revision...

Olly Betts <olly at survex.com> wrote: > On Fri, Aug 18, 2023 at 10:41:52AM +0000, Eric Wong wrote: > > Olly Betts <olly at survex.com> wrote: > > > While the match is running, get_mset(2000, 1000) needs to track > > > 3000 entries so this won't reduce your heap usage (at least not > > > peak usage). > > > > > > Is the heap

In other words, is it possible to avoid duplicates if new documents are inserted into the DB by another process in-between ->get_mset calls when reusing Xapian::Enquire objects? I do some expensive processing on each mset window, so I always limit the results to limit heap usage even if I'm planning on going through a big chunk of the DB: $mset = $enq->get_mset(0, 1000);

On Thu, Aug 17, 2023 at 09:28:26PM +0000, Eric Wong wrote: > In other words, is it possible to avoid duplicates if new > documents are inserted into the DB by another process in-between > ->get_mset calls when reusing Xapian::Enquire objects? The Database object itself effectively does (it works in a snapshot of the state of the database when you open it, or last called reopen() which

Hello all, Well, my local mirror now has CentOS 5.6! Yay! But, I cannot get it to install. A "yum update" and a "yum --skip-broken" yields me this: --> Processing Conflict: firefox conflicts xulrunner >= 1.9.2.14 --> Finished Dependency Resolution firefox-3.6.13-2.el5.centos.i386 from installed has depsolving problems --> firefox conflicts with xulrunner

...rformance penalties of that technology. And the index often encounter writes in the time it takes for the backup to copy off the files. Thus the index is potentially unsafe on tape. I dont know much about Xapian's internal versioning of data, but I suspect it is something like PostgreSQL's MVCC system. Would so my theory was: If no ->flush "has ended" while the backup was running, then the index would be safe, since the last thing to do is more or less to set the revision flag to the new version, which all have to co-exists with a fully functional version? Could we make a ho...

OK ... there has been an awful lot of attributing problems to the 4.4 upgrade that are not really upgrade problems. The best I can tell there is really only one major problem: 1. You need to install python-sqlite before sqlite (or at nearly the same time). To accomplish this you can do this: yum update python-sqlite sqlite Then after that, a normal yum update works fine. There are 2 other

Hi, I have read the concurrency webpage from the Xapian documentation: http://getting-started-with-xapian.readthedocs.io/en/latest/concepts/concurrency.html But it is still not clear to me how to ensure thread-safety when using libxapian (C++ API). Usually when doing multi-threading many threads can read the same variable concurrently without locking provided none of the threads modifies the

...tempting to open another writer on the same database will throw xapian.DatabaseLockError to indicate that it wasn’t possible to acquire a lock. Multiple concurrent readers are supported (in addition to the writer). Note the caution there about the number of versions that can exist in Xapian's MVCC, and how to deal with DatabaseModifiedError. J -- James Aylett, occasional troublemaker & project governance xapian.org

...in > distributed SQL databases (ideas which are applicable here) whereby the > communication profile between peers are minimized, and synchronization is > never necessary except in circumstances where a peer has the data resident > in memory and needs to perform an update (requiring an MVCC lock). Through > a catalog you can find out who is the chairman for any particular record, > thus be able to know who manages locks related to it, as well as handles > contended updates. In this way, communication tends to be segmented, and > lock management is localized. If you check...

Olly Betts <olly at survex.com> wrote: > On Wed, Aug 23, 2023 at 01:53:27PM +0000, Eric Wong wrote: > > I'm already retrying the ->get_mset operations; but now I'm > > wondering where I'd hit DatabaseModifiedErrors while inside a > > Xapian::MSetIterator loop. > > > > I assume ->get_document is a place where it gets thrown; > > but

Superb. I'll take a look. Thank you On Tue, Oct 6, 2020 at 1:46 AM Martin Schwenke <martin at meltin.net> wrote: > Hi Bob, > > On Mon, 5 Oct 2020 09:31:59 -0400, Robert Buck <robert.buck at som.com> > wrote: > > > It seems as though, when I go from `clustering = no` to `clustering = > yes`, > > if I do a domain join, it will fail. However, if I do a

...ances have been made in distributed SQL databases (ideas which are applicable here) whereby the communication profile between peers are minimized, and synchronization is never necessary except in circumstances where a peer has the data resident in memory and needs to perform an update (requiring an MVCC lock). Through a catalog you can find out who is the chairman for any particular record, thus be able to know who manages locks related to it, as well as handles contended updates. In this way, communication tends to be segmented, and lock management is localized. It seems to us, and we need to me...

Hi @ all Does anyone has some experience with PostgreSQL and RoR? Does it gives any disadvantages or problems with PostgreSQL? I have a webapplication, that manage (read/write) many records (no. 200''000 records) and I search a ideal database. Actually, I have two options: - MySQL / - PostgreSQL And now, I would like to ascertain a good solution... Thx, ribit -- Posted via

I have multiple puppetdb severs pointed at the same database. Would setting gc-interval to 0 disable it on the extra puppetdb nodes? -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/4DEyx0IXbLsJ. To post to this group, send email to

Is there any reason to avoid using postgresql for small web apps? That is, is it''s overhead so large that the lighter MySQL will work substantially better for small apps on small machines? Or are they very similar in performance and configuration? I''ve used MySQL but postgresql has its appeals and I''m wondering if I need to keep MySQL around at all. On the other

Hello, I have configured puppet and puppetdb (storeconfig) with postgresql and the service run and the collecting are good for example: I can make query with the next command and responds ok . [.........................] [root@master]# curl -X GET -H ''Accept: application/json'' http://127.0.0.1:8080/v2/facts --data-urlencode ''query=["=", "name",