Olly writes:> The .DB file is full of blocks, which are 8KB unless you've specified > otherwise explicitly.(Haven't.)> The format of the blocks is described in comments in > backends/chert/chert_table.cc and .h.Cool!> However, it occurs to me you could try xapian-compact on the bad > database (if you didn't already). That reads the entries in the table > sequentially for which the dividing keys don't matter, so it may give > you a good copy - I'd try that first.I hadn't tried that - I have now started a run on the copy of the index, to try it out.> Compacting would fix this count too, but it's stored in the base file > (baseA is the active one from the output above) and see comments in > backends/chert/chert_btreebase.cc for the format of that.I'm crossing my fingers for xapian-compact to save the day - thanks for the tip and pointers! Best regards, Adam -- "OK, I'll call you tomorrow. And by the way, I'll Adam Sj?gren see if I can twist some arms and get you an Oscar. asjo at koldfront.dk But I want a Nobel Prize in return."
Adam writes:>> However, it occurs to me you could try xapian-compact on the bad >> database (if you didn't already). That reads the entries in the table >> sequentially for which the dividing keys don't matter, so it may give >> you a good copy - I'd try that first. > > I hadn't tried that - I have now started a run on the copy of the index, > to try it out.Compacting went without any errors (run time around 10 hours): $ xapian-compact index index.compact-2011-11-18 postlist: Reduced by 47% 64119424K (136063368K -> 71943944K) record: Reduced by 35% 635936K (1772776K -> 1136840K) termlist: Reduced by 41% 33945728K (81677456K -> 47731728K) position: Reduced by 38% 368390992K (945391144K -> 577000152K) spelling: doesn't exist synonym: doesn't exist $ However the following xapian-check failed (after 16 minutes): $ xapian-check /z/orne/nzdb/index.compact-2011-11-18 record: baseB blocksize=8K items=10465475 lastblock=142104 revision=1 levels=2 root=601 B-tree checked okay record table structure checked OK termlist: baseB blocksize=8K items=20930950 lastblock=5966465 revision=1 levels=3 root=343098 B-tree checked okay xapian-check: DatabaseError: inflate failed (invalid distance too far back) $ Odd, or? Best regards, Adam -- "H?r nu lige her: N?r Bill Gates personligt siger, Adam Sj?gren at det er s?dan, s? m? du alts? tro p? det !" asjo at koldfront.dk