Xapian discuss - Aug 2006 - Omega, flint, big stubs, big samples, missing dates, htdig_noindex

As I ponder how to get my poor computer out of the temporal
rift (maybe one of the flux capacitors is leaking?), it seems like
a good opportunity to ask some questions about Omega.
I read all the documentation I could find, including the last
six months of this list archive.

Does omindex have a super extra secret hidden option
for using flint backend?

Which is likely to be faster on a single machine, one gigantic
database or a few thousand smaller databases glued together
via a stub db?

Is there a prayer of Omega storing a larger (e.g. configurable size)
sample someday so that one can get a better summary result? Or
is that idea doomed due to backwards compatibility issues?

How can one successfully use END in an Omega query, but not
see document dates in the summary results or even the date
field at all in godmode?

Any thoughts on that earlier htdig_noindex question?

Whew.

On Mon, Aug 07, 2006 at 09:48:53PM -0700, Jeff Breidenbach wrote:
> Does omindex have a super extra secret hidden option
> for using flint backend?
I think it's still true that to use flint you just set:

XAPIAN_PREFER_FLINT="true"

in your environment before running whatever. Providing flint is
compiled in, that should work.
> Which is likely to be faster on a single machine, one gigantic
> database or a few thousand smaller databases glued together
> via a stub db?
Depends :-)

It depends on things like how your disks are laid out, what your data
looks like, and other concerns. How long does it take you to build
your db? If it's under a day, I'd recommend doing both and testing
each way.
> Is there a prayer of Omega storing a larger (e.g. configurable size)
> sample someday so that one can get a better summary result? Or
> is that idea doomed due to backwards compatibility issues?
Umm. Do we have an internal limit on the document data? I can quite
happily add data several times the size of the underlying database
block size, so I'm guessing no (effectively).

I think it's just that the number of words / characters / whatever is
hard coded in omindex.cc, so it's pretty easy to change. Note that
with scriptindex you have a configurable truncation limit.
> How can one successfully use END in an Omega query, but not
> see document dates in the summary results or even the date
> field at all in godmode?
Sorry, not sure I understand what you're asking...

James

-- 
/--------------------------------------------------------------------------\
  James Aylett                                                  xapian.org
  james@tartarus.org                               uncertaintydivision.org