Displaying 20 results from an estimated 23 matches for "set_stemmer".
2015 Jul 26
1
Get term from document by position
...ar microbe";
int
main(int argc, char **argv)
{
// indexing
Xapian::WritableDatabase db_w(database_dir, Xapian::DB_CREATE_OR_OVERWRITE);
MyText text_to_index;
text_to_index.set_string();
Xapian::TermGenerator indexer;
Xapian::Stem stemmer("english");
indexer.set_stemmer(stemmer);
Xapian::Document doc;
indexer.set_document(doc);
indexer.set_stemming_strategy(Xapian::TermGenerator::STEM_ALL_Z);
indexer.index_text(text_to_index.text_str);
db_w.add_document(doc);
db_w.commit();
db_w.close();
//searching
Xapian::Database db(da...
2013 Sep 02
2
having trouble with prefixes
...:") + path);
Problem is, I can't seem to run a query that returns the document using
any of the terms. Here is the outline of the code that runs the queries
I'm trying to run:
Database db(db_path.string());
QueryParser queryparser;
Stem stemmer("english");
//queryparser.set_stemmer(stemmer);
queryparser.set_database(db);
queryparser.add_prefix("type", "T");
queryparser.add_prefix("md5sum", "Q");
queryparser.add_prefix("path", "P");
queryparser.add_prefix("extension", "E");
//maybe set stemmi...
2018 Nov 30
1
Xapian Benchmark results
...----------
#define NUM_THREADS 4
void parallel_index(int tid, int num_threads,vector<string> documents,Xapian::WritableDatabase *db )
{
int i =0;
char line[2048];
string line_string;
Xapian::TermGenerator indexer;
Xapian::Stem stemmer("english");
indexer.set_stemmer(stemmer);
// doc.set_data("content");
indexer.set_stemming_strategy(Xapian::TermGenerator::STEM_SOME);
Xapian::WritableDatabase database = *db;
try{
for(i=0;i<documents.size();++i)
{
ifstream file;
file.open(documents[i]);
line_string...
2008 Mar 12
1
how can i use stopwords?
...tion...
I've set the termgenerator like this:
$self->{'Stemmer'} = new Search::Xapian::Stem(german2);
$self->{'Stopper'} = new Search::Xapian::SimpleStopper();
$self->{'TermGenerator'} = new Search::Xapian::TermGenerator;
$self->{'TermGenerator'}->set_stemmer( $self->{'Stemmer'} );
$self->{'TermGenerator'}->set_stopper( $self->{'Stopper'} );
I've thought that xapian now exclude the stopwords automatic by the stemming-language.
Could someone explain the stopword function?
Thanks.
Greetings
Sascha
2012 Jun 04
1
Search not finding queries with stop words.
I have a search in perl that looks a bit like:
my $qp = new Search::Xapian::QueryParser();
$qp->set_stemmer(new Search::Xapian::Stem("english"));
$qp->set_stemming_strategy(STEM_SOME);
$qp->set_default_op($defaultop);
...
my $par = $qp->parse_query($query);
my $enq = $xDatabase->enquire( $par );
and in the db create script:
my $stopper = Search::Xapian::SimpleS...
2013 Sep 22
2
How to filter search result with query with has white space.
...std::string content;
std::string url;};
void indexData(document d) {
try {
Xapian::WritableDatabase db("/Users/ramesh/Desktop/xapian",
Xapian::DB_CREATE_OR_OPEN);
Xapian::TermGenerator indexer;
Xapian::Stem stemmer("english");
indexer.set_stemmer(stemmer);
Xapian::Document doc;
doc.set_data(d.title);
indexer.set_document(doc);
indexer.index_text(d.title,1,"title");
indexer.index_text(d.content,1,"content");
indexer.index_text(d.url,1,"url");
doc.add_boolea...
2013 Sep 22
2
How to filter search result with query with has white space.
...std::string content;
std::string url;};
void indexData(document d) {
try {
Xapian::WritableDatabase db("/Users/ramesh/Desktop/xapian",
Xapian::DB_CREATE_OR_OPEN);
Xapian::TermGenerator indexer;
Xapian::Stem stemmer("english");
indexer.set_stemmer(stemmer);
Xapian::Document doc;
doc.set_data(d.title);
indexer.set_document(doc);
indexer.index_text(d.title,1,"title");
indexer.index_text(d.content,1,"content");
indexer.index_text(d.url,1,"url");
doc.add_boolea...
2010 Jun 09
1
TermGenerator incorrectly tokenizes German text which contains special characters
...?dlich is indexed as gesundheitssch? and Zgesundheitssch?
(stemmed).
All character encodings are set to UTF-8, the MySql database is also in
UTF-8 encoding.
*
#1 $lIndexer = new XapianTermGenerator();
#2 $lStemmer = new XapianStem(XapianHelper::GetStemmer($pLanguage)); //
?german?
#3 $lIndexer->set_stemmer($lStemmer);
#4 $lDoc = new XapianDocument();
#5 $lDoc->add_term($lObj->Id);
#6 $lIndexer->set_document($lDoc);
#7 $lIndexer->index_text("Nahrungserg?nzungsmittel Ausrei?er");
#8 $lIndexer->index_text($lSomeStringFromDb);*
In the code example just above here the problem onl...
2014 Jan 27
4
Perl Search::Xapian
...;
my $csvfile = $ARGV[0];
my $dbpath = $ARGV[1];
my $db = Search::Xapian::WritableDatabase->new(
path => $dbpath,
mode => "DB_CREATE_OR_OPEN",
);
# setup TermGenerator that'll be used in indexing.
my $tg = Search::Xapian::TermGenerator->new();
$tg->set_stemmer(Search::Xapian::Stem->new('en'));
# here is a for to loop thru all the csv? file.
my $fh = Tie::Handle::CSV->new($csvfile, header => 1);
while (my $csvline = <$fh>) {
my $description = $csvline->{DESCRIPTION};
my $title = $csvline->{TITLE};
my $identifier =...
2005 Jun 09
1
Query parser and stemming of norwegian letters
Hello, can I get an explanation of the following.
Running the following code:
....
pqp=new QueryParser();
Stem stem("norwegian");
cout << "DEBUG " << stem.stem_word(_sXapian)<< endl;
pqp->set_stemmer(stem);
pqp->set_database(*_pdatabase);
pqp->set_default_op(Query::OP_AND);
//Set the enquire
Query p=pqp->parse_query(_sXapian);
cout << " Query " << string(bufSL) << p.get_description() << endl;
---
gives the foll...
2007 Dec 29
3
Term-Flags
Hi,
Is it necessary to set the down below flag to the TermGenerator,
if I want the "Did you mean ..." spelling corrections?
Xapian::TermGenerator::flags::FLAG_SPELLING
Thank you very much
Markus
2009 Mar 02
0
Xapian, PHP bindings and
...shell
// level.
$query_string = xapianEncode( $_GET['q'] );
$rset = new XapianRSet();
$qp = new XapianQueryParser();
$stemmer = new XapianStem( 'french' );
$qp->set_stemmer($stemmer);
$qp->set_database($database);
$qp->set_stemming_strategy(XapianQueryParser::STEM_SOME);
$query = $qp->parse_query( $query_string ); //,
XapianQueryParser::FLAG_SPELLING_CORRECTION);
print 'Parsed query is: &...
2009 Aug 17
1
Xapian DatabaseError
...f search function.inc
<?php
require_once 'xapian.php';
function simpleSearch($query,$path){
try {
$database = new XapianDatabase($path);
$enquire = new XapianEnquire($database);
$qp = new XapianQueryParser();
$stemmer = new XapianStem("italian");
$qp->set_stemmer($stemmer);
$qp->set_database($database);
$qp->set_stemming_strategy(XapianQueryParser::STEM_SOME);
$query = $qp->parse_query($query);
echo "<br>Parsed query is: {$query->get_description()}\n";
$enquire->set_query($que...
2010 Mar 31
1
Hyphen search with parse_query()
...from "peter" & "bengtsson" and is different from "peterbengtsson") and
find it.
To start with I'm trying to use a basic python script to get to grips
with it. When I do this::
qp = xapian.QueryParser()
stemmer = xapian.Stem("english")
qp.set_stemmer(stemmer)
qp.set_database(database)
qp.set_stemming_strategy(xapian.QueryParser.STEM_SOME)
print "Query string is:", repr(query_string)
query = qp.parse_query(query_string)
print "Parsed query is: %s" % str(query)
I get the following output::
Query string...
2013 Aug 21
2
Perl interface isn't working in 1.2.x
At least it isn't working the way it used to.
Code:
$db = Search::Xapian::Database->new( $dx );
my $qp = Search::Xapian::QueryParser->new();
my $dbSize=$db->get_doccount();
# $qp->set_stemmer(new Search::Xapian::Stem("english"));
# $qp->set_stemming_strategy(STEM_SOME);
# $qp->set_default_op($defaultop);
my $par = $qp->parse_query($query);
my $enq = $db->enquire( $par );
my @matches = $enq->matches($nstart,$nrecords);
my $mset = $enq-&...
2007 May 30
1
QueryParser prefixing terms when stemming?
...wever, something seems to have changed during the upgrade and I
need help figuring out how my code should be written.
In version 0.9.9.1 of Search::Xapian, the following code results in
this output "Xapian::Query(pet:(pos=1))".
my $qp = new Search::Xapian::QueryParser;
$qp->set_stemmer(new Search::Xapian::Stem('english'));
$qp->set_default_op(OP_AND);
$qp->set_stemming_strategy(STEM_SOME);
warn $qp->parse_query($search_term);
In version 1.0.0.0, the same code results in "Xapian::Query(Zpet:
(pos=1))". The result is no matches, even though t...
2010 Oct 28
1
hypens in words + NEAR + 3 terms + AND_MAYBE => crash
...AR test NEAR x-y"
will.
Stemming language was "english", strategy STEM_SOME. Default op was OP_AND_MAYBE.
This is the offending Perl code:
[...]
my $qp = Search::Xapian::QueryParser->new();
my $stemmer = Search::Xapian::Stem->new("english");
$qp->set_stemmer($stemmer);
$qp->set_database($database);
$qp->set_stemming_strategy(STEM_SOME);
$qp->set_default_op(OP_AND_MAYBE);
$query_string = " x-y NEAR test NEAR test ";
my $query = $qp->parse_query($query_string, Search::Xapian::FLAG_DEFAULT);
[...]
Here'...
2018 Jul 19
1
choosing between probabilistic and boolean prefixes for terms
...= 'english';
sub stemmer { Search::Xapian::Stem->new($LANG) }
sub qp {
my ($self) = @_;
my $qp = $self->{query_parser};
return $qp if $qp;
# new parser
$qp = Search::Xapian::QueryParser->new;
$qp->set_default_op(OP_AND);
$qp->set_database($self->{xdb});
$qp->set_stemmer($self->stemmer);
$qp->set_stemming_strategy(STEM_SOME);
$qp->set_max_wildcard_expansion(100);
$qp->add_valuerangeprocessor(
Search::Xapian::NumberValueRangeProcessor->new(YYYYMMDD, 'd:'));
$qp->add_valuerangeprocessor(
Search::Xapian::NumberValueRangeProcessor->...
2014 Jan 21
2
seg fault on search
...base db;
db.add_database(Database(fullDB));
Enquire enquire(db);
try {
Xapian::QueryParser qp;
Xapian::Stem stemmer("english");
qp.set_database(db);
qp.set_default_op(Query::OP_OR);
qp.set_stemmer(stemmer);
qp.set_stemming_strategy(QueryParser::STEM_SOME);
Xapian::Query q=qp.parse_query(queryString);
...
As soon as it hits the q=qp.parse_query line it gets the seg fault.
The function is called like so:
int main (int argc, char* const argv[]) {
int...
2006 Nov 30
1
PHP / XapianQueryParser
...d I'm not sure if it is actually my mistake (although I'm pretty sure it is :))
I'm trying to use the XapianQueryParser (in PHP5), but I get an error everytime.
$odb = new XapianDatabase($db);
$ostem = new XapianStem("en");
$oqparser = new XapianQueryParser();$oqparser->set_stemmer($ostem);$oqparser->set_database($odb);$oqparser->set_stemming_strategy(1);$oqparser->add_boolean_prefix("Q",1);$oquery = $oqparser->parse_query($query, 1);
I found some perl examples and 'ported' it over to the PHP version. Varying the flags (like XapianQueryParser::...