Xapian devel - Apr 2009 - Test of Search::Xapian::Writable Database, and how to send a patch

Hello Xapian developers,

I'm Kosei MORIYAMA. I'm a student application gsoc 2009, Xapian and
SWIG project.

I wrote a test, search-xapian/t/writable database.t , which tests
methods of Search::Xapian::Writable Database, replace_document,
replace_document_by_term, delete_document and delete_document_by_term.

Then I thought to send a patch of this test to be reviewed. Following
xapian-core/HACKING file's 'Submitting Patches' section, run
following
command,

$ svn diff > writable database.diff

and attach this file to this mail.

Is this correct way of sending patch? And Do I need to send any other
information about this patch?

Cheers,
Kosei MORIYAMA
-------------- next part --------------
Index: writabledatabase.t
==================================================================---
writabledatabase.t	(revision 0)
+++ writabledatabase.t	(revision 0)
@@ -0,0 +1,203 @@
+#!/usr/bin/perl -W
+# Before `make install' is performed this script should be runnable with
+# `make test'. After `make install' it should work as `perl
test.pl'
+
+#########################
+
+use Test::More;
+if ($] < 5.008007) {
+    plan skip_all => 'Test requires Perl >= 5.8.7';
+} else {
+    # Number of test cases to run - increase this if you add more testcases.
+    plan tests => 34;
+}
+
+use Search::Xapian qw(:standard);
+
+
+# first create database dir, if it doesn't exist;
+my $db_dir = 'testdb';
+
+if( (! -e $db_dir) or (! -d $db_dir) ) {
+  mkdir( $db_dir );
+}
+
+opendir( DB_DIR, $db_dir );
+while( defined( my $file = readdir( DB_DIR ) ) ) {
+  next if $file =~ /^\.+$/;
+  unlink( "$db_dir/$file" ) or die "Could not delete
'$db_dir/$file': $!";
+}
+closedir( DB_DIR );
+
+my $create = Search::Xapian::WritableDatabase->new( $db_dir,
Search::Xapian::DB_CREATE );
+
+$create = undef;
+
+my $write = Search::Xapian::WritableDatabase->new( $db_dir,
Search::Xapian::DB_CREATE_OR_OPEN );
+
+# Let's try to index something.
+my $term = 'test';
+
+my $docid;
+for my $num (1..1000) {
+  my $doc = Search::Xapian::Document->new();
+
+  $doc->set_data( "$term $num" );
+
+  $doc->add_posting( $term, 0 );
+  $doc->add_posting( $num, 1 );
+
+  $doc->add_value(0, $num);
+  $write->add_document( $doc );
+} 
+$write->flush();
+
+for my $num qw (three four five) {
+  my $doc = Search::Xapian::Document->new();
+
+  $doc->set_data( "$term $num" );
+
+  $doc->add_posting( $term, 0 );
+  $doc->add_posting( $num, 1 );
+
+  $doc->add_value(0, $num);
+  $write->add_document( $doc );
+  $write->flush();
+}
+$write->flush();
+
+my $doccount = $write->get_doccount();
+is($doccount, 1003, "check number of documents in WritableDatabase");
+
+# replace document by docidb
+my $repdoc = Search::Xapian::Document->new();
+my $num = "six";
+$term = "test";
+$docid = 500;
+$repdoc->set_data( "$term $num" );
+$repdoc->add_posting( $term, 0 );
+$repdoc->add_posting( $num, 1 );
+$repdoc->add_value(0, $num);
+
+ok(!$write->term_exists($num), "check term exists");
+is($write->get_document($docid)->get_data(), "$term $docid",
"check document data");
+
+$write->replace_document($docid, $repdoc);
+$write->flush();
+
+ok($write->term_exists($num), "check term exists");
+is($write->get_document($docid)->get_data(), "$term $num",
"check document data");
+
+# replace document by term
+$repdoc = Search::Xapian::Document->new();
+$term = "test";
+$num = "seven";
+$repdoc->set_data( "$term $num" );
+$repdoc->add_posting( $term, 0 );
+$repdoc->add_posting( $num, 1 );
+$repdoc->add_value(0, $num);
+$repterm = "five";
+
+ok(!$write->term_exists($num), "check term exists");
+ok($write->term_exists($repterm), "check term exists");
+is($write->get_termfreq($num), 0, "check term frequency");
+is($write->get_termfreq($repterm), 1, "check term frequency");
+
+$write->replace_document_by_term($repterm, $repdoc);
+$write->flush();
+
+ok($write->term_exists($num), "check term exists");
+ok(!$write->term_exists($repterm), "check term exists");
+is($write->get_termfreq($num), 1, "check term frequency");
+is($write->get_termfreq($repterm), 0, "check term frequency");
+
+# replace document by term, if term is new
+$repdoc = Search::Xapian::Document->new();
+$term = "test";
+$num = "eight";
+$repdoc->set_data( "$term $num" );
+$repdoc->add_posting( $term, 0 );
+$repdoc->add_posting( $num, 1 );
+$repdoc->add_value(0, $num);
+
+is($write->get_termfreq($term), $doccount, "check term
frequency");
+is($write->get_termfreq($num), 0, "check term frequency");
+
+$write->replace_document_by_term($num, $repdoc);
+$write->flush();
+
+is(++$doccount, 1004, "check doccount");
+is($write->get_termfreq($term), $doccount, "check term
frequency");
+is($write->get_termfreq($num), 1, "check term frequency");
+
+# replace document by term.
+# if there are any documents which has same term, the document which has
smallest id is replaced.
+$repdoc = Search::Xapian::Document->new();
+$term = "test";
+$num = "nine";
+$repdoc->set_data( "$term $num" );
+$repdoc->add_posting( $term, 0 );
+$repdoc->add_posting( $num, 1 );
+$repdoc->add_value(0, $num);
+
+$write->replace_document_by_term($term, $repdoc);
+$write->flush();
+my $doc = $write->get_document(1);
+
+is($write->get_doccount(), 1, "check document count");
+is($doc->get_data(), "$term $num", "check document
data");
+
+# add documents for following tests
+for my $num qw (one two three four five) {
+  my $doc = Search::Xapian::Document->new();
+
+  $doc->set_data( "$term $num" );
+
+  $doc->add_posting( $term, 0 );
+  $doc->add_posting( $num, 1 );
+
+  $doc->add_value(0, $num);
+  $write->add_document( $doc );
+  $write->flush();
+}
+$write->flush();
+
+$doccount = $write->get_doccount();
+is($doccount, 6, "check number of documents in WritableDatabase");
+
+# delete document by docid
+my $lastdocid = $write->get_lastdocid();
+my $lastdocterm = $write->get_document($lastdocid)->get_value(0);
+ok($write->term_exists($lastdocterm), "check term exists");
+
+$write->delete_document($lastdocid);
+$write->flush();
+
+is($write->get_doccount(), $doccount - 1, "check number of documents in
WritableDatabase");
+ok(!$write->term_exists($lastdocterm), "check term exists");
+
+# delete document by term
+my $delterm = 'three';
+ok($write->term_exists($delterm), 'check term exists before deleting a
document');
+is($write->get_termfreq($delterm), 1, 'check term frequency before
deleting a document');
+
+$write->delete_document_by_term($delterm);
+$write->flush();
+
+is($write->get_doccount(), $doccount - 2, 'check WritableDatabase after
deleting a document');
+ok(!$write->term_exists($delterm), 'check term exists after deleting a
document');
+is($write->get_termfreq($delterm), 0, 'check term frequency after
deleting a document');
+
+# delete documents by term
+$delterm = 'test';
+ok($write->term_exists($delterm), 'check term exists of documents which
has term "test"');
+is($write->get_termfreq($delterm), $doccount - 2, 'check term frequency
of term "test"');
+
+$write->delete_document_by_term($delterm);
+$write->flush();
+
+is($write->get_doccount(), 0, 'check WritableDatabase after deleting all
documents');
+ok(!$write->term_exists($delterm), 'check term exists after deleting all
documents');
+is($write->get_termfreq($delterm), 0, 'check term frequency after
deleting all documents');
+
+1;

On Tue, Apr 14, 2009 at 05:21:36AM +0900, Kosei MORIYAMA
wrote:
> I'm Kosei MORIYAMA. I'm a student application gsoc 2009, Xapian and
> SWIG project.
> 
> I wrote a test, search-xapian/t/writable database.t , which tests
> methods of Search::Xapian::Writable Database, replace_document,
> replace_document_by_term, delete_document and delete_document_by_term.
> 
> Then I thought to send a patch of this test to be reviewed. Following
> xapian-core/HACKING file's 'Submitting Patches' section, run
following
> command,
> 
> $ svn diff > writable database.diff
> 
> and attach this file to this mail.
Thanks for the patch.
> Is this correct way of sending patch? And Do I need to send any other
> information about this patch?
The best thing to do with patches is to make a ticket for them in our trac
instance (at http://trac.xapian.org/).  This way, they don't get lost and
forgotten.  Also, large patches may not be accepted by the mailing list
software (it has a size limit, though I can't remember how large that limit
is off-hand).

Generally, with a patch you should include an explanation of why the patch
is a good idea.  In this case, I don't think there's anything obvious
extra
that needs to be included.

-- 
Richard