Ferret talk - Oct 2006 - Ferret 0.10.11 & AAF: sorting Time fields doesn''t work

Ferret 0.10.11 & AAF: the time seems to be stored in a format that
can''t
be sorted, the order doesn''t make any sense. Workaround: use to_i on
the
Time object before putting it into the index.

-- 
Posted via http://www.ruby-forum.com/.

On 10/14/06, Andreas Schwarz <f at andreas-s.net>
wrote:
> Ferret 0.10.11 & AAF: the time seems to be stored in a format that
can''t
> be sorted, the order doesn''t make any sense. Workaround: use to_i
on the
> Time object before putting it into the index.
This is fine for sorting but it won''t work when you want to do range
queries. If you want to run range queries against the field you will
need to pad the integer to a fixed width.

I usually add dates in YYYYMMDD format so that they are also human
readable. Another thing to take into account is the precision you
want. The higher the precision you store the longer indexing and
searching will take. For small indexes however the difference will be
negligible.

Cheers,
Dave



Cheers,
Dave

David Balmain wrote:
> On 10/14/06, Andreas Schwarz <f at andreas-s.net> wrote:
>> Ferret 0.10.11 & AAF: the time seems to be stored in a format that
can''t
>> be sorted, the order doesn''t make any sense. Workaround: use
to_i on the
>> Time object before putting it into the index.
> 
> This is fine for sorting but it won''t work when you want to do
range
> queries. If you want to run range queries against the field you will
> need to pad the integer to a fixed width.
Thanks, I didn''t think of that. The width of a unix timestamp
isn''t
going to change for the next few years, though.
> I usually add dates in YYYYMMDD format so that they are also human
> readable.
I need at least seconds in this case.
> Another thing to take into account is the precision you
> want. The higher the precision you store the longer indexing and
> searching will take. For small indexes however the difference will be
> negligible.
Will this also affect :sort performance? I have another index with 450k 
documents that I also would like to sort by time, that''s probably no 
longer a small index. Is the sorting capable of :sorting and :limiting a 
large numer of results (e.g. 100k), or is this something that should 
only be done with small result sets?

Andreas

-- 
Posted via http://www.ruby-forum.com/.

On 10/14/06, Andreas Schwarz <f at andreas-s.net>
wrote:
> David Balmain wrote:
> > On 10/14/06, Andreas Schwarz <f at andreas-s.net> wrote:
> >> Ferret 0.10.11 & AAF: the time seems to be stored in a format
that can''t
> >> be sorted, the order doesn''t make any sense. Workaround:
use to_i on the
> >> Time object before putting it into the index.
> >
> > This is fine for sorting but it won''t work when you want to
do range
> > queries. If you want to run range queries against the field you will
> > need to pad the integer to a fixed width.
>
> Thanks, I didn''t think of that. The width of a unix timestamp
isn''t
> going to change for the next few years, though.
>
> > I usually add dates in YYYYMMDD format so that they are also human
> > readable.
>
> I need at least seconds in this case.
>
> > Another thing to take into account is the precision you
> > want. The higher the precision you store the longer indexing and
> > searching will take. For small indexes however the difference will be
> > negligible.
>
> Will this also affect :sort performance? I have another index with 450k
> documents that I also would like to sort by time, that''s probably
no
> longer a small index. Is the sorting capable of :sorting and :limiting a
> large numer of results (e.g. 100k), or is this something that should
> only be done with small result sets?
It should sort the first time in under one second. After that the sort
will be cached so it will be even quicker. To use sort caching though,
be sure to use a sort object and not a sort_field or plain old string.
Also, for best performance, sort by byte instead of integer (which
will only work if you have a fixed width integer field). ie

    :sort => Sort.new(SortField.new(:field_name, :type => :byte))

Cheers,
Dave