Ferret talk - Sep 2006 - A few questions about numbers and dates

Hi,

I just noticed that Ferret seems to convert every field to a string
[ruby code appended for those interested], which has thwarted my attempt
to format Dates (to "dd/mm/yyyy") and Floats (to "n.nn") for
consumption
further down the line based on the class of the field stored.

I considered pre-formatting Dates and Floats prior to indexing, which
would store the field exactly as I need for presentation purposes,
however I would lose sorting and range searching.

I''m wondering what approaches people are using to manage formatting
post
retrieve from Ferret. 

Any pointers appreciated,

Kind Regards

Neville
=========================
require ''rubygems''
require ''ferret''
require ''date''

p Ferret::VERSION

@dir = Ferret::Store::RAMDirectory.new
@index = Ferret::Index::Index.new(:dir => @dir)

invoice = {:invoice_date => Date.new(2006,9,20), :invoice_value =>
44.50, :invoice_no => 45656, :invoice_to => ''Nev''}

@index << invoice

doc = @index[0].load

doc.fields.each do |f|
  p f
  p doc[f].class
  p doc[f]
end

=========================
>ruby test_format.rb
"0.10.6"
:invoice_date
String
"2006-09-20"
:invoice_value
String
"44.5"
:invoice_no
String
"45656"
:invoice_to
String
"Nev"

On 9/28/06, Neville Burnell <Neville.Burnell at bmsoft.com.au>
wrote:
> Hi,
>
> I just noticed that Ferret seems to convert every field to a string
> [ruby code appended for those interested], which has thwarted my attempt
> to format Dates (to "dd/mm/yyyy") and Floats (to
"n.nn") for consumption
> further down the line based on the class of the field stored.
>
> I considered pre-formatting Dates and Floats prior to indexing, which
> would store the field exactly as I need for presentation purposes,
> however I would lose sorting and range searching.
>
> I''m wondering what approaches people are using to manage
formatting post
> retrieve from Ferret.
>
> Any pointers appreciated,
>
> Kind Regards
>
> Neville
<code snipped>

Hi Neville,

One possible solution is to add those fields twice, once in a
stored/unindexed field in the correct format for output and another in
an unstored/indexed field with the correct format for sorting and
range queries.

Actually, sorting will already work on those fields above, it is just
range queries you have to worry about, and even they will work on your
date field. I would certainly be amenable to developing a
FloatRangeQuery and IntegerRangeQuery with the right motivation. ;-)

Finally, I''ve mentioned before that I''m very interested in
building an
object database based on Ferret. I mentioned it here:

  http://www.ruby-forum.com/topic/82086#new

By turning documents into objects and giving fields types it would
solve the problems you have just mentioned plus many more.

Cheers,
Dave

Hi David,
 
> One possible solution is to add those fields twice
> once in a stored/unindexed field in the correct format
> for output and another in an unstored/indexed field 
> with the correct format for sorting and range queries.
Yes, I''ll likely do this, in combination with some form of hungarian
notation to indicate the "native" type of the field.

This would allow all dates to be stored as is by Ferret, but allow the
presentation layer to reformat the date for display to dd/mm/yyyy
format.

For numbers I am planning to pad the string with zeros which will
likewise support ranges/sorting, and then strip the zeros in the
presentation layer.
> Actually, sorting will already work on those fields above
I don''t see how sorting could work for dates stored as
"dd/mm/yyyy" ...
Would you elaborate?
> Finally, I''ve mentioned before that I''m very interested
in
> building an object database based on Ferret. I mentioned it here:
> http://www.ruby-forum.com/topic/82086#new
> By turning documents into objects and giving fields types it 
> would solve the problems you have just mentioned plus many more.
Yes, I think this is a very interesting idea, very useful and
reminiscent of WinFS for objects!

Kind Regards

Neville

On 9/29/06, Neville Burnell <Neville.Burnell at bmsoft.com.au>
wrote:
> > Actually, sorting will already work on those fields above
>
> I don''t see how sorting could work for dates stored as
"dd/mm/yyyy" ...
> Would you elaborate?
Sorry, I meant, the date 2006-09-20 in you example would sort
correctly. You are quite right. "dd/mm/yyyy" won''t sort
correctly.