Xapian discuss - Sep 2011 - New scws patch for xapian-core based on svn trunk

Hi, I have already re-make the patch file, based on the trunk codes in SVN
respo.

But I can not build the xapian, because there are many errors when building
'languages/' for stemmer. Therefore, I could not test the new patching
code.

Patching code can be accessed from:

http://www.xunsearch.com/download/xapian-scws-1.3.x-trunk.patch

Need to install SCWS first as below steps:

1. compile/install the scws library: 
    http://www.ftphp.com/scws/down/scws-1.1.8.tar.bz2
2. Extracting dictionary files into  'etc/' of scws installed directory
    http://www.ftphp.com/scws/down/scws-dict-chs-utf8.tar.bz2

I will be very pleased, if this can be integrated into xapian releases.

BTW: Whether there is the latest source package  can be compiled directly?

Sorry, I missed this mail before.

On Mon, Sep 19, 2011 at 05:32:37PM +0800, hightman
wrote:
> Hi, I have already re-make the patch file, based on the trunk codes in
> SVN respo.
Cool, thanks.
> But I can not build the xapian, because there are many errors when
> building 'languages/' for stemmer. Therefore, I could not test the
new
> patching code.
Hmm, what are the errors?
> Patching code can be accessed from:
> 
> http://www.xunsearch.com/download/xapian-scws-1.3.x-trunk.patch
> 
> Need to install SCWS first as below steps:
> 
> 1. compile/install the scws library: 
>     http://www.ftphp.com/scws/down/scws-1.1.8.tar.bz2
> 2. Extracting dictionary files into  'etc/' of scws installed
directory
>     http://www.ftphp.com/scws/down/scws-dict-chs-utf8.tar.bz2
> 
> I will be very pleased, if this can be integrated into xapian releases.
I've opened a ticket to track this:

http://trac.xapian.org/ticket/594
> BTW: Whether there is the latest source package  can be compiled directly?
There are snapshot source downloads, which have all the generated files
in - see the "Snapshots" section here:

http://xapian.org/bleeding

Cheers,
    Olly

Hello, thanks.

I have created new patch file based on the snapshot package of
xapian-core-1.3.x. it works well with scws or without scws.

Here also includes some optimization for CJK terms, such as removal of the
stemmed record, conversion of multi-segmentation as synonym queries.

The patch file download URL: 
http://www.xunsearch.com/download/xapian-scws-1.3.x-snap.patch

1. Compile/install the scws library: 
    http://www.ftphp.com/scws/down/scws-1.2.0.tar.bz2
2. Extracting dictionary files into  'etc/' of scws install directory
    http://www.ftphp.com/scws/down/scws-dict-chs-utf8.tar.bz2
3. Patch & re-configure the xapian-core:
    patch -p1 < xapian-scws-1.3.x-snap.patch
    autoreconf
    rm -f queryparser/queryparser_internal.cc
    ./configure --with-scws=/usr/local/scws
    make

----- Simple test result -----

localhost:examples hightman$ ./simpleindex ./db
???????????????????
Hello, I am hightman and come from china.

localhost:examples hightman$ ./simpleindex ./db
????T???????????

localhost:examples hightman$ ./simplesearch ./db ??
Parsed query is: Query(??@1)
1 results found.
Matches 1-1:

1: 0.569074 docid=2 [????T???????????]

localhost:examples hightman$ ./simplesearch ./db chinas
Parsed query is: Query(Zchina at 1)
1 results found.
Matches 1-1:

1: 0.377177 docid=1 [??????????????????? Hello, I am hightman and come from
china.]

localhost:examples hightman$ ./simplesearch ./db ????
Parsed query is: Query((????@1 SYNONYM (??@89 OR ??@90)))
1 results found.
Matches 1-1:

1: 0.121029 docid=1 [??????????????????? Hello, I am hightman and come from
china.]

localhost:examples hightman$ ./simplesearch ./db ?T?
Parsed query is: Query((?@1 OR t?@2))
1 results found.
Matches 1-1:

1: 0.876681 docid=2 [????T???????????]

? 2012-3-30???12:00? Olly Betts ???
> Sorry, I missed this mail before.
>