When I index many documents, I don't want to store duplicate documents. How can I detect whether a same document has been stored in database and then discard it? Or is there a function that xapian can automatically detect it and avoid storing duplicate ducuments? 2009-11-03 Best wishes from ray.
Jarrod Roberson
2009-Nov-03 04:11 UTC
[Xapian-discuss] How to avoid storing duplicate documents?
On Mon, Nov 2, 2009 at 9:01 PM, ??? <ysj.ray at gmail.com> wrote:> > When I index many documents, I don't want to store duplicate documents. > How can I detect whether a same document has been stored in database and > then discard it? > Or is there a function that xapian can automatically detect it and avoid > storing duplicate ducuments? > > 2009-11-03 >If you have a unique KEY/ID you can add to each document then you can look up that key and if it found don't re-add the document. -- Jarrod Roberson 678.551.2852