Xapian discuss - Sep 2006 - Datefield problems. Indexing or configuration problems?

Hello,

We use omage/xapian with php, and are having troubles with our date-fields and
our date-range searches.

Our scriptindex is formatted as follows:
    topicday  : boolean=DTD field=topicdate
    topicmonth: boolean=DTM
    topicyear : boolean=DTY
    replyday  : boolean=DRD field=replydate
    replymonth: boolean=DRM
    replyyear : boolean=DRY    
    topicdate : value=1
    topicdatei: value=2
    replydate : value=3
    replydatei: value=4

When we perform searches with date ranges limit, we get unexpected results:
Our first query on "omega" in year 2005, returns 9 results. 8 in
january, 1 on 11/aug/2005
DB=forum 'P=omega' DEFAULTOP=AND B=XF1 B=XF2 B=DTY2005

However when we narrow the search to use the DTM field and focus on august, we
get no results at all
DB=forum 'P=omega' DEFAULTOP=AND B=XF1 B=XF2 B=DTM200508

Narrowing further down, using the DTD fields, also doesn't yield any results
DB=forum 'P=omega' DEFAULTOP=AND B=XF1 B=XF2 B=DTD20050811

Doing a search on "surrounding" dates, doesn't help a bit:
DB=forum 'P=omega' DEFAULTOP=AND B=XF1 B=XF2 B=DTD20050810 B=DTD20050811
B=DTD20050812

Currently, I am quite lost as to where to start looking.
Is this a problem about having indexed our topics incorrectly. Or should I look
for configuration problems in the scriptindex or other parts of our setup?

When indexing, we use the following php script for the date fields:
 $recordTxt.= 'topicday=' 
.date('Ymd',$topicData['time'])."\n";
 $recordTxt.= 'topicmonth='.date('Ym'
,$topicData['time'])."\n";
 $recordTxt.= 'topicyear=' .date('Y', 
$topicData['time'])."\n";
 $recordTxt.= 'replyday=' 
.date('Ymd',$topicData['lastreply'])."\n";
 $recordTxt.= 'replymonth='.date('Ym',
$topicData['lastreply'])."\n";
 $recordTxt.= 'replyyear=' .date('Y', 
$topicData['lastreply'])."\n";
 $recordTxt.= 'topicdate=' .($topicData['time'])."\n";
 $recordTxt.=
'topicdatei='.($year2020-$topicData['time'])."\n";
 $recordTxt.= 'replydate='
.($topicData['lastreply'])."\n";
 $recordTxt.=
'replydatei='.($year2020-$topicData['lastreply'])."\n";

Any help appreciated,

-r-

On Mon, Sep 25, 2006 at 12:56:25PM +0200, Arnoud ten Hoedt (Ro?na?n
Webdevelopment) wrote:
> Currently, I am quite lost as to where to start looking.
You want to check which terms actually index the documents in question.

You can either use the "delve" tool (in the examples subdirectory of
xapian-core), or Omega's "godmode" - assuming you've installed
all
the default templates, just run omega with "FMT=godmode" added on
to the parameter list - this adds a "#<documentid>" link to the
left
of each hit.  Click on this and you can see all the terms which
index that document.  Click on a term to see all the documents which
match that term, etc.
> Is this a problem about having indexed our topics incorrectly. Or
> should I look for configuration problems in the scriptindex or other
> parts of our setup?
I suspect a problem with the data you're feeding into scriptindex
though the snippet of PHP you posted looked correct.

Cheers,
    Olly