search for: indexwriter

I''m playing with "updating" docs in my index, and I think I''ve found bug with IndexWriter counting deleted docs. Script and output follow: ===== require ''rubygems'' require ''ferret'' p Ferret::VERSION @doc = {:id => ''44'', :name => ''fred'', :email => ''abc at ozemail.com.au''} @dir = Ferret:...

...39;' class SearchEngine include Ferret include Ferret::Document # Creates or returns an existing index for an organization def self.get_index(type = ''writer'') index_dir = "/var/search/index" if type == ''writer'' index = Index::IndexWriter.new(index_dir, :create_if_missing => true) elsif type == ''reader'' index = Index::IndexReader.open(index_dir, false) end return index end end Thanks!! - Shanti -- Posted via http://www.ruby-forum.com/.

Hi David, > Deleted documents don''t get deleted until commit is called Ok, but FYI, my experiments show that #commit doesn''t affect #doc_count, even across ruby sessions. On a different note, I''d like to request a variation of #add_document which returns the doc_id of the document added, as opposed to self. I''m trying to track down an issue with a large

Hi All, Hope all is going well. I''m having trouble with the following code creating an in memory index reader - it seems to be attempting to read from a file regardless. Here''s the simple code: require ''rubygems'' require ''ferret'' a = Ferret::Index::Index.new r = Ferret::Index::IndexReader.new(nil) Running the code on my OS X machine

Just a quick note for future reference - at least for me, ferret won''t work on Ruby 1.8.4. gem install ferret Successfully installed ferret-0.11.4-mswin32 ruby -v ruby 1.8.4 (2005-12-24) [i386-mswin32] irb irb(main):001:0> require ''ferret'' A windows error message box appears - ruby.exe - Entry Point Not Found The procedure entry point rb_w32_write could not be

...ocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:698) at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:712) at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1507) at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpd...

> I should also mention the reason I wouldn''t want > to return the document ID from any IndexWriter method > is that the document ID could become invalid when the > next document is added (if a segment merge is triggered > and deletes exist). At least when using an IndexReader, > the document ID is valid for the life of the reader. Thanks for your detail Dave! Regards, Neville...

Is there a fast way to determine how many documents a term occurs in, besides iterating through every document with TermDocEnum? -- Best regards, Stian Gryt?yr

I implemented a "reindex" command which simply creates an IndexWriter with :create => true for a prexisting index. The "reindexing" seems to start out ok, with several thousand docs added, then Ferret throws an exception: IO Error occured: couldn''t rename file "index\_0.tmp" to "index\_0.cfs": <File exists> I guess...

...>: Error: fts_solr: received invalid uid '0' or 11 03:36:03 indexer-worker(jom at grosjo.net)<2093><icWMJaIwD1znEgAA0thIag:GPBOCKMwD1wtCAAA0thIag>: Error: fts_solr: Indexing failed: 500 Server Error Solr: CAUSED BY: ORG.APACHE.LUCENE.STORE.ALREADYCLOSEDEXCEPTION: THIS INDEXWRITER IS CLOSED Dec 11 06:00:14 gjserver solr[16761]: at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:679) Dec 11 06:00:14 gjserver solr[16761]: at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:693) Dec 11 06:00:14 gjserver solr[16761]: at org.apache.lucene.index.Inde...

In the documentation, it says that optimize "should only be called when the index will no longer be updated very often, but will be read a lot". Does this mean it actually has a detrimental impact on updates and inserts? In my project there will be many more reads than updates, but there will still be a lot of updates. So should I be calling Optimize once a day or something like that,

Firstly, thanks Dave for all your hard work. Ferret Rocks!, I am just testing 0.11.4.win32 and it seems to work just fine, however the index creation phase of my app is perhaps 3x slower under 0.11.4 vs 0.10.9 Details follow: System: windows xp sp2, index on local hard disk, Ruby 1.8.6 Run #1, Ferret 0.10.9 - Reboot - Build index, 35,000 rows added in 297 seconds - Run #2, Ferret 0.11.4 -

...ted this analyzer: class DescriptionAnalyzer < Ferret::Analysis::Analyzer def token_stream(field, string) if field == "code" return CodeTokenStream.new(string) else return Ferret::Analysis::Analyzer.new.token_stream(field,string) end end end and created an IndexWriter with it: Ferret::Index::IndexWriter.new(get_index_path, :create => true, :analyzer => DescriptionAnalyzer.new) get_index_path is a method to calculate the path to use. The problem is that my token_stream method is never called....

...n you open an IndexReader on the index it is opened up on > that particular version (or state) of the index. So any > operations on the IndexReader (like searches) will only show > what was in the index at the time you opened it. Any modifications > to the index (usually through and IndexWriter) that occur after > you open the IndexReader will not appear in your searches. > So to keep searches up to date you need to close and reopen your > IndexReader every time you commit changes to the index. Would it be possible to enhance IndexReader to report the "version" of...

...ut> ??????? reclaim = 0 ??????? i = <optimized out> #2? 0x00007fbd98ae7e29 in ?? () from /usr/lib/libclucene-core.so.1 No symbol table info available. #3? 0x00007fbd98ae87ee in ?? () from /usr/lib/libclucene-core.so.1 No symbol table info available. #4? 0x00007fbd98b3639d in lucene::index::IndexWriter::init(lucene::store::Directory*, lucene::analysis::Analyzer*, bool, bool, lucene::index::IndexDeletionPolicy*, bool) () from /usr/lib/libclucene-core.so.1 No symbol table info available. #5? 0x00007fbd98b36883 in lucene::index::IndexWriter::IndexWriter(char const*, lucene::analysis::Analyzer*, bool...

...a thread for each request. Should I create a new IndexReader for each request also, or can I use one IndexReader concurrently? Index updates on the other hand are coordinated by a special Update Thread which runs every 10 minutes or so. I''m guessing that the best approach is to create an IndexWriter for each update run, which can be closed and discarded at the end of the update run. Or can I close and reuse a single IndexWriter? I searched http://ferret.davebalmain.com/api for details on the MultiReader, but I couldn''t find any details. If someone could post a link to point me in the...

...:allocator<lucene::index::IndexCommitPoint*> >&) () > ?? from /usr/lib/libclucene-core.so.1 > No symbol table info available. > #2? 0x00007f1653f15312 in ?? () from /usr/lib/libclucene-core.so.1 > No symbol table info available. > #3? 0x00007f1653f325e3 in lucene::index::IndexWriter::doFlush(bool) () > from /usr/lib/libclucene-core.so.1 > No symbol table info available. > #4? 0x00007f1653f32f9c in lucene::index::IndexWriter::flush(bool, bool) > () from /usr/lib/libclucene-core.so.1 > No symbol table info available. > #5? 0x00007f1653f33052 in > lucene::ind...

...Apr 05 19:30:53 indexer-worker(XXXXX): Fatal: master: service(indexer-worker): child 809 killed with signal 11 (core not dumped) Apr 05 19:30:53 imap(XXXXXX): Error: indexer failed to index mailbox INBOX Apr 05 19:30:56 indexer-worker(XXXXX): Error: lucene index /home/XXXXX/Maildir/lucene-indexes: IndexWriter() failed (#1): Lock obtain timed out When trussing the indexer-worker pid I can see it ends with: ...SNIP... 527: brk(0x005A7840) = 0 527: brk(0x005AD840) = 0 527: pread64(16, " I G E N E R I C I < /".., 8170,...

...8 = metadata !{metadata !"ISPDIL_MTHD_~IndexReader"} !2489 = metadata !{metadata !"ISPDIL_MTHD_indexExists"} !2490 = metadata !{metadata !"ISPDIL_MTHD_isLocked"} !2491 = metadata !{metadata !"ISPDIL_MTHD_unlock"} !2492 = metadata !{metadata !"ISPDIL_MTHD_IndexWriter"} !2493 = metadata !{metadata !"ISPDIL_MTHD_optimize"} !2494 = metadata !{metadata !"ISPDIL_MTHD_~IndexWriter"} !2495 = metadata !{metadata !"ISPDIL_MTHD_Init"} !2496 = metadata !{%"class.lucene::document::Document" zeroinitializer} .... And I have in...

...se end readers = [] readers << IndexReader.new("/tmp/0") readers << IndexReader.new("/tmp/1") readers << IndexReader.new("/tmp/2") readers << IndexReader.new("/tmp/3") readers << IndexReader.new("/tmp/4") index_writer = IndexWriter.new(:path => "/tmp/test") index_writer.add_readers(readers) index_writer.close() readers.each {|reader| reader.close()} i = Ferret::I.new(:path => ''/tmp/test'') res = i.search(''name*'') puts res.inspect # gives me: #<struct Ferret::Search::TopDoc...