Hi, On Fri, Jul 29, 2016, at 13:45, James Aylett wrote:> On Fri, Jul 29, 2016 at 12:12:25PM +0200, rsto at paranoia.at wrote: > > The FastMail snippet generator has been written when MSet didn't create > > snippets. I'll first compare both implementations to see if there is a > > good reason for them to coexist, or might just as well merge any > > additional features into MSet.I created pull request https://github.com/xapian/xapian/pull/117 which updates MSet to optionally calculate snippet relevances based on query term coverage. That's similar to how it is done in the latest version of the FastMail snippet generator. I'd be happy to migrate the remaining features of the FastMail snippet generator into MSet, but it might be good to do so feature by feature. Oddly enough, the pull request causes Travis to break for clang but not for gcc [1]. That's because the clang build process fails for the test 'querypairwise1' [2], which AFAIK I didn't touch at all. Is that a known issue or did I break anything? [1] https://travis-ci.org/xapian/xapian/builds/149512190 [2] https://travis-ci.org/xapian/xapian/jobs/149512191#L15051> > I've just opened a pull request for the CJK tokenizer: > > https://github.com/xapian/xapian/pull/114 > > Unfortunately, Travis breaks since pkg-config can't find libicu on the > > machine [1]. > You should be able to install it; if you add libicu-dev to the > packages stanza in .travis.yml it will put it in there. > However you seem to be using pkg-config, which Ubuntu 12.04 LTS (which > travis currently uses) doesn't provide for libicu. 14.04 LTS does, and > it's possible to use that as a beta with travis, I think by changing:I'm a bit a loss here. I can't get it to build on both 12.04 LTS and 14.04 LTS. On 12.04 LTS Travis breaks due to pkg-config missing [3]. On 14.04 LTS the build breaks due to Python missing the sphinx package [4]. That's probably a Python version 2 vs 3 environment issue. [3] https://travis-ci.org/xapian/xapian/jobs/149516398#L1530 [4] https://travis-ci.org/xapian/xapian/jobs/149516019#L4262 The only way is see is to get rid of the pkg-config checks in configure.ac and check in configure.ac for libicu headers and libraries. Or do you plan to use pkg-config in xapian-core and rather make this work with Travis? Cheers, Robert
On Wed, Aug 03, 2016 at 06:54:32PM +0200, rsto at paranoia.at wrote:> I created pull request https://github.com/xapian/xapian/pull/117 which > updates MSet to optionally calculate snippet relevances based on query > term coverage. That's similar to how it is done in the latest version of > the FastMail snippet generator. I'd be happy to migrate the remaining > features of the FastMail snippet generator into MSet, but it might be > good to do so feature by feature.I don't have time to look at this right now, but from a quick look this seems to be a promising way of merging the additional FastMail features :)> Oddly enough, the pull request causes Travis to break for clang but not > for gcc [1]. That's because the clang build process fails for the test > 'querypairwise1' [2], which AFAIK I didn't touch at all. Is that a > known issue or did I break anything? > > [1] https://travis-ci.org/xapian/xapian/builds/149512190 > [2] https://travis-ci.org/xapian/xapian/jobs/149512191#L15051Hmm, it's been running okay on recent PRs as far as I know. That's...weird :-(> I'm a bit a loss here. I can't get it to build on both 12.04 LTS and > 14.04 LTS. On 12.04 LTS Travis breaks due to pkg-config missing [3]. On > 14.04 LTS the build breaks due to Python missing the sphinx package [4]. > That's probably a Python version 2 vs 3 environment issue. > > [3] https://travis-ci.org/xapian/xapian/jobs/149516398#L1530 > [4] https://travis-ci.org/xapian/xapian/jobs/149516019#L4262 > > The only way is see is to get rid of the pkg-config checks in > configure.ac and check in configure.ac for libicu headers and libraries. > Or do you plan to use pkg-config in xapian-core and rather make this > work with Travis?As far as I'm aware, we're not planning on using pkg-config in xapian-core; the general aim is to keep dependencies to a bare minimum. Right now that's libuuid and zlib (and a little more for Omega), and although they both do provide pkgconfig, our existing autoconf detection of those has never caused problems that I'm aware of. Do you think it's safe to detect libicu using headers? I'm not convinced it's simple enough to do reliably (hence the old icuconfig system before moving to pkg-config). J -- James Aylett, occasional trouble-maker xapian.org
On Wed, Aug 3, 2016, at 19:26, James Aylett wrote:> On Wed, Aug 03, 2016 at 06:54:32PM +0200, rsto at paranoia.at wrote: > > Oddly enough, the pull request causes Travis to break for clang but not > > for gcc [1]. That's because the clang build process fails for the test > > 'querypairwise1' [2], which AFAIK I didn't touch at all. Is that a > > known issue or did I break anything? > > > > [1] https://travis-ci.org/xapian/xapian/builds/149512190 > > [2] https://travis-ci.org/xapian/xapian/jobs/149512191#L15051 > > Hmm, it's been running okay on recent PRs as far as I > know. That's...weird :-(I see that querypairwise1 tests for the running time of a test case. Could there be a glitch with high loads on the Travis build machine causing inconsistent runtime performance? The error message for clang reads: Running test: querypairwise1... FAILED Test with 50 repetitions took 0.003521 secs Test with 500 repetitions took 0.055811 secs Whereas the same test succeeded for the g++ build. I'll retry to build tomorrow.> Do you think it's safe to detect libicu using headers? I'm not > convinced it's simple enough to do reliably (hence the old icuconfig > system before moving to pkg-config).Thanks, I'll see how autoconf checks for libuuid and zlib. Use of icuconfig is indeed discouraged by the ICU maintainers . If I can't get the PR build properly without pkgconfig, I'll make both libicu and pkg-config optional. To do so, I'll try to get Travis build with 14.04 LTS, which might require a few build runs. I'll notify you when the CJK pull request passes Travis. Cheers, Robert