Rails - Apr 2005 - Ruby/Odeum Full Text Inverted Index Extension

Hello Everyone,

Announcing the Ruby/Odeum library--a full text inverted indexer for
Ruby:

http://www.zedshaw.com/projects/ruby_odeum/

Ruby/Odeum is an extension that wraps Mikio Hirabayashi’s QDBM Odeum
library for fast reverse indexing of documents.  It supports indexing
documents by their words, breaking the text into words, normalizing the
words, searching, and attaching meta-data to each document.  It is very
fast and the Ruby/Odeum extension is nice and small with only two
classes needed.

This release features:

1.  Much improved project layout with a Rake and setup.rb build.
2.  Full RubyDoc documentation for every function.
3.  A more complete example script odeum_mgr which uses Ruby/Odeum to
index some documents and let you search it.
4.  Some new functions to make the library even easier and help reduce
the amount of Ruby to C interfunction calls.
5.  Bug fixes related to memory management.  There seems to be an issue
with the garbage collector not collecting Document objects inside an
each block, so there''s a new Document.close function to do it manually.

Ruby/Odeum is licensed under the same license as QDBM (LGPL).

Feel free to download and send me your comments.  I''m already in the
process of removing all C99 code in order to get it to compile on old
2.95 C (that way you people using old version of *BSD can avoid
upgrading for one more year :-)

Zed A. Shaw

P.S. I also posted this to ruby-talk, but it seems there''s a lot of
rails folks that aren''t on ruby-talk.  What''s the etiquete
here?  Do
people care about cross posting?

I care!
Where do I go to subscribe to Ruby-talk ?

On 4/14/05, Zed A. Shaw <zedshaw-dd7LMGGEL7NBDgjK7y7TUQ@public.gmane.org>
wrote:
> 
> Hello Everyone,
> 
> Announcing the Ruby/Odeum library--a full text inverted indexer for
> Ruby:
> 
> http://www.zedshaw.com/projects/ruby_odeum/
> 
> Ruby/Odeum is an extension that wraps Mikio Hirabayashiÿs QDBM Odeum
> library for fast reverse indexing of documents. It supports indexing
> documents by their words, breaking the text into words, normalizing the
> words, searching, and attaching meta-data to each document. It is very
> fast and the Ruby/Odeum extension is nice and small with only two
> classes needed.
> 
> This release features:
> 
> 1. Much improved project layout with a Rake and setup.rb build.
> 2. Full RubyDoc documentation for every function.
> 3. A more complete example script odeum_mgr which uses Ruby/Odeum to
> index some documents and let you search it.
> 4. Some new functions to make the library even easier and help reduce
> the amount of Ruby to C interfunction calls.
> 5. Bug fixes related to memory management. There seems to be an issue
> with the garbage collector not collecting Document objects inside an
> each block, so there''s a new Document.close function to do it
manually.
> 
> Ruby/Odeum is licensed under the same license as QDBM (LGPL).
> 
> Feel free to download and send me your comments. I''m already in
the
> process of removing all C99 code in order to get it to compile on old
> 2.95 C (that way you people using old version of *BSD can avoid
> upgrading for one more year :-)
> 
> Zed A. Shaw
> 
> P.S. I also posted this to ruby-talk, but it seems there''s a lot
of
> rails folks that aren''t on ruby-talk. What''s the etiquete
here? Do
> people care about cross posting?
> 
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
> 


-- 
- Ramin
http://www.getintothis.com/blog


_______________________________________________
Rails mailing list
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
http://lists.rubyonrails.org/mailman/listinfo/rails

http://www.ruby-lang.org/en/20020104.html

On Apr 14, 2005, at 8:11 AM, Zed A. Shaw wrote:
> Hello Everyone,
>
> Announcing the Ruby/Odeum library--a full text inverted indexer for
> Ruby:
>
> http://www.zedshaw.com/projects/ruby_odeum/
>
> Ruby/Odeum is an extension that wraps Mikio Hirabayashi´s QDBM Odeum
> library for fast reverse indexing of documents.  It supports indexing
> documents by their words, breaking the text into words, normalizing the
> words, searching, and attaching meta-data to each document.  It is very
> fast and the Ruby/Odeum extension is nice and small with only two
> classes needed.
>
> This release features:
>
> 1.  Much improved project layout with a Rake and setup.rb build.
> 2.  Full RubyDoc documentation for every function.
> 3.  A more complete example script odeum_mgr which uses Ruby/Odeum to
> index some documents and let you search it.
> 4.  Some new functions to make the library even easier and help reduce
> the amount of Ruby to C interfunction calls.
> 5.  Bug fixes related to memory management.  There seems to be an issue
> with the garbage collector not collecting Document objects inside an
> each block, so there''s a new Document.close function to do it
manually.
>
> Ruby/Odeum is licensed under the same license as QDBM (LGPL).
>
> Feel free to download and send me your comments.  I''m already in
the
> process of removing all C99 code in order to get it to compile on old
> 2.95 C (that way you people using old version of *BSD can avoid
> upgrading for one more year :-)
>
> Zed A. Shaw
>
> P.S. I also posted this to ruby-talk, but it seems there''s a lot
of
> rails folks that aren''t on ruby-talk.  What''s the
etiquete here?  Do
> people care about cross posting?
>
I sure don''t mind.  Mostly because I only read ruby-talk every now and 
then.  At any rate, I just checked this out and it''s pretty awesome, 
great work Zed!  Anyone writing a Rails app that needs document 
searching should definitely check this out.  My only beef is that it''s 
GPL, but I suppose I can live with that ;)  Has anyone started to make 
a gem out of this?  If not, I''ll give it a stab tonight.

-Scott


_______________________________________________
Rails mailing list
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
http://lists.rubyonrails.org/mailman/listinfo/rails

Hi Barron,

Scott Barron wrote:
> On Apr 14, 2005, at 8:11 AM, Zed A. Shaw wrote:
> 
>> Hello Everyone,
>>
>> Announcing the Ruby/Odeum library--a full text inverted indexer for
>> Ruby:
>>
>> http://www.zedshaw.com/projects/ruby_odeum/
>>
>> .
>> .
>> .
>>
>> P.S. I also posted this to ruby-talk, but it seems there''s a
lot of
>> rails folks that aren''t on ruby-talk.  What''s the
etiquete here?  Do
>> people care about cross posting?
>>
> 
> I sure don''t mind.  Mostly because I only read ruby-talk every now
and
> then.  At any rate, I just checked this out and it''s pretty
awesome,
> great work Zed!  Anyone writing a Rails app that needs document 
> searching should definitely check this out.  My only beef is that
it''s
> GPL, but I suppose I can live with that ;)  Has anyone started to make a 
Just curiosity, what''s wrong with GPL(eled) licenses ?
> gem out of this?  If not, I''ll give it a stab tonight.
> 
> -Scott
> 
regards,

eduardo.

On Apr 15, 2005, at 4:25 PM, Eduardo Manso wrote:
>
> Hi Barron,
>
> Scott Barron wrote:
>> On Apr 14, 2005, at 8:11 AM, Zed A. Shaw wrote:
>>> Hello Everyone,
>>>
>>> Announcing the Ruby/Odeum library--a full text inverted indexer for
>>> Ruby:
>>>
>>> http://www.zedshaw.com/projects/ruby_odeum/
>>>
>>> .
>>> .
>>> .
>>>
>>> P.S. I also posted this to ruby-talk, but it seems there''s
a lot of
>>> rails folks that aren''t on ruby-talk.  What''s the
etiquete here?  Do
>>> people care about cross posting?
>>>
>> I sure don''t mind.  Mostly because I only read ruby-talk every
now
>> and then.  At any rate, I just checked this out and it''s
pretty
>> awesome, great work Zed!  Anyone writing a Rails app that needs 
>> document searching should definitely check this out.  My only beef is 
>> that it''s GPL, but I suppose I can live with that ;)  Has
anyone
>> started to make a
>
> Just curiosity, what''s wrong with GPL(eled) licenses ?
Nothing is *wrong* per se, I just prefer MIT license.  I think it 
offers more freedom, and I''ll leave it at that (this is not the place 
to debate licenses and in retrospect I probably shouldn''t have posted 
my preference as such).

-Scott


_______________________________________________
Rails mailing list
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
http://lists.rubyonrails.org/mailman/listinfo/rails

Fantastic library. 

What would be the main strategy to integrate this in a rails app.

a dedicated DRb service for the app? 
a namespaced TCP based server for multiple apps?
a In process version, perhaps persisted using a relational DB?
> > Just curiosity, what''s wrong with GPL(eled) licenses ?
> 
> Nothing is *wrong* per se, I just prefer MIT license.  I think it
> offers more freedom, and I''ll leave it at that (this is not the
place
> to debate licenses and in retrospect I probably shouldn''t have
posted
> my preference as such).

Not getting into license wars here, the code is luckily LGPL and not
GPL and thats a fair license for a library because it will allow the
majority of people to use it.
That being said I have found that people tend to contribute more code
back to me when I use a compleatly free license like MIT.

Rails is MIT and ruby itself is basically MIT as well. 

-- 
Tobi
http://www.snowdevil.ca - Snowboards that don''t suck
http://www.hieraki.org   - Open source book authoring
http://blog.leetsoft.com - Technical weblog