Displaying 2 results from an estimated 2 matches for "xapian_flush_threshold_length".
2007 Feb 07
2
My new record: Indexing 20 millions docs = 79m9.378s
Gentoo Linux 2.6
8 AMD Opteron 64-bit Processors
32GB Memory
--------------------------------------------------------------------------------
Environment:
------------------
XAPIAN_FLUSH_THRESHOLD=21000000
XAPIAN_FLUSH_THRESHOLD_LENGTH=16000000
XAPIAN_PREFER_FLINT=True
Indexing 20 million documents:
--stemmer=none
-------------------------------------------
real 79m9.378s
user 77m28.696s
sys 1m36.654s
# delve /home/kevin/index
---------------------------------------
number of documents = 19999995
average document lengt...
2004 Oct 08
1
indexing performance
...%, but memory use mounted to VSZ 244M RSS180M. considering we have 2G RAM,
I wonder whether we have a way to utilize our machine more to get better
performance with indexing.
Question:
How can I expedite our indexer? Did I do sth wrong with my indexer?
BTW, I set the following env parameters:
XAPIAN_FLUSH_THRESHOLD_LENGTH=5000000
XAPIAN_FLUSH_THRESHOLD=10000
Many many thanks.
Hongyan Ma