search for: snippetgenerator

G'day, This is some code which have been using for some time now in our searching beta, forward ported from 1.2.12 to today's git. It adds a class Xapian::SnippetGenerator which can be used to generate human-readable snippet strings. By default, the snippets are HTML formatted, with the matched search term inside a <b> tag, 5 words of context around matched search terms, and ellipsis "..." between non-adjacent context. There is a certain amount of c...

Since the option of running -mode=inverse_throughput was added to llvm-exegesis the names of classes like UopsSnippetGenerator and UopsBenchmarkRunner, that this mode shares with uops, started to be less descriptive. Inverse_throughput doesn't use the uops counters, so for example, the instruction layout shared between these two modes is really connected to parallelism, not uops. It's doubly confusing for architec...

...n't > try to comment on that at all :) I've just opened a pull request for the CJK tokenizer: https://github.com/xapian/xapian/pull/114 > I wonder if we can arrange suitable defaults to use your > implementation with the older API, and come up with a newer API that > allows a SnippetGenerator class to be used from the MSet. The FastMail snippet generator has been written when MSet didn't create snippets. I'll first compare both implementations to see if there is a good reason for them to coexist, or might just as well merge any additional features into MSet. > A good start...

Hi, The Cyrus IMAP mail server uses Xapian as search engine. Recently, FastMail has sponsored implementation of two Xapian features: CJK word splitting and a generator for search snippets. I've been working on both features and we would be happy to get them merged into Xapian master. The CJK word tokenizer uses the word segmentation algorithms of the International Components for Unicode