search for: rdig

Hi all, Am having problems using RDig: With this rdig config... cfg.crawler.start_urls = [''http://www.defensetech.org''] cfg.crawler.include_hosts = [''www.defensetech.org''] cfg.index.path = ''/my/path/to/index'' cfg.verbose = true ...I get this output: $ rdig -c config/rdig_config....

Hi All, Hope it is ok to post RDig queries on this forum. Just trying to get RDig working (Ubuntu 6.06, RDig 0.3.0, ferret 0.9.4, rubyful_soup 1.0.4) Here is my output: sh:~/rdigtry$ rdig -c config/rdig_config.rb discovered content extractor class: RDig::ContentExtractors::PdfContentExtractor discovered content extractor class:...

I have a site with two indexes. Index A is created offline by RDig and queried from the web via RDig (specifically, RDig.searcher.search). Index B is managed by AAF with :remote => true. Simple enough. However, I need to query both indexes from RDig. Usually this is ok, as I modified RDig to accept an array of search_paths with an element for index A...

I''m developing locally on Windows and I have a remote dev box that runs Linux. I''m trying to use RDig just to index using urls, no files. Both use acts_as_ferret for an administrative search that works fine. On the Windows machine, I get no errors, but get no results. On the Linux machine, I get: File Not Found Error occured at <except.c>:93 in xraise Error occured in index.c:840 - sis_fi...

Hi All, I have to have a site wide search for my current application. By search I mean I have to search the static and the dynamic contents from the database. I have been searching on this for a while on the net and RDig seems to be a apt solution. While using it I have encountered a few problems. I know these might be very basic issues but I have not been able to figure out what is wrong with the code. I had the following lines in my /config/environment.rb 1. require ''rdig'' 2. require ...

...caches them locally for offline viewing. I want to do searches on this collection and associate extra data with each result (e.g date collected, reason for collection, perhaps a sequence number). Now all this data exists when the harvesting is done and could be stored in a database. I want to use RDig to index my collection of sites I also want to associate the index results with my extra data and display them along with search results. The index is built once and searched many times so I want searching to be as efficient as possible. The simplest way is to use e.g. the local URL as a key into...

Hi! RDig is a small tool to build a Ferret index for the contents of a website or intranet. It contains a simple HTTP crawler and some support for extracting textual content from the fetched pages. I built this to implement a site-wide search for a recent project that combined a Rails application with lot...

I got this root at linux:~# rdig -c configfile RDig version 0.3.4 using Ferret 0.10.14 added url file:///home/myaccount/documents/ waiting for threads to finish... root at linux:~# rdig -c configfile -q "Ruby" RDig version 0.3.4 using Ferret 0.10.14 executing query >Ruby< Query: total results: 0 root at linux:~#...

Hey there, I''m building a rails site using RDig as a site-wide search. I would like to be able to add just one URL (or possibly a list) to an existing index, so that when certain pages change I can update the index without reindexing the entire site. I looked through the documentation and didn''t see an example on how to do this so I a...

...9;'ve seen some people have similar issues but I haven''t seen any resolutions. The contents of the index directory follow the error. Has anyone seen anything like this and found a resolution? Many thanks. /mnt/apps/search/releases/20070622175637/script/../config/../vendor/ gems/rdig-0.3.4/lib/rdig/index.rb:48:in `optimize'': End-of-File Error occured at <except.c>:93 in xraise (EOFError) Error occured in store.c:216 - is_refill current pos = 0, file length = 0 from /mnt/apps/search/releases/20070622175637/script/../ config/../vendor/gems/rdi...

Directory of C:\search_app 01/21/2007 19:37 <DIR> . 01/21/2007 19:37 <DIR> .. 01/21/2007 19:36 427 008 ferret-0.10.13.gem 01/21/2007 19:07 148 992 rdig-0.3.4.gem 2 File(s) 576 000 bytes 2 Dir(s) 45 135 982 592 bytes free C:\search_app>gem install ferret Building native extensions. This could take a while... ERROR: Error installing gem ferret[.gem]: ERROR: Failed to build gem native ext ension. Gem files...

Lucene has simple wildcard syntax supporting ? and * thus ruby could be matched by rub? r*by etc. This doesn''t work using rdig on the command line e.g. rdig -c config.rb -q ''data:"ru?y"'' gives RDig version 0.3.4 using Ferret 0.10.14 executing query >data:"ru?y"< Query: data:"ru y"~1 which is something entirely different. The ferret docs seem to imply that support fo...

Lucene has a syntax "foo bar"~10 for finding foo within 10 words of bar. Does ferret support this feature? (the ~ is used for fuzzy queries) Does rdig? This could be a deal breaker for me ''cos I really need proximity searches -- Posted via http://www.ruby-forum.com/.

The following exception occurred while trying optimize a large index: vendor/gems/rdig-0.3.4/lib/rdig/index.rb:46:in `optimize'': End-of- File Error occured at <except.c>:93 in xraise (EOFError) Error occured in store.c:216 - is_refill current pos = 0, file length = 0 Now, I get the following error any time I try to create a new index on the directory that...

Hi, Is there any application where I can see the usage of Ferret engine(like example implementation). I have some difficulties in using it, sending query and getting the results. Thank you, Raj. -- Posted via http://www.ruby-forum.com/.

I was wondering if it is possible to search word documents using ferret. The actual text in a word document isn''t in a binary format - only the formatting. Surely it would be possible to parse that? -- Posted via http://www.ruby-forum.com/.

Hi, I am currently preparing a new version of my package papeR. When I run R CMD check using the development version of R I get the following note: Package in Depends field not imported from: ?nlme?, ?lme4?, ?survival? These packages needs to imported from for the case when this namespace is loaded but not attached. I now have problems to fix this issue. It is easy to get rid of two of the

I have a large index (~6GB, ~1 million docs) that was built by RDig. I wrote a script to iterate through the index to clear out some duplicate information to try to reduce the size of the index. clients.each {|client| docs = RDig.searcher.search("+supplier_id:#{client.id}") docs.each {|doc| data = doc[:data].dup #the contents of the web page...

I have a MySQL table with over 18 million records in it. We are indexing about 10 fields in this table with ferret. I am having problems with the initial building of the index. I created a rake task to run the "Model.rebuild_index" command in the background. That process ran fine for about 2.5 days before it just suddenly stopped. The log/ferret_index.log file says it got to about

Hi, One of the AAF features is to be able to index results of methods, but I haven''t seen anywhere how to do this. I have a method that returns the full text of a file and I''d like for this to be indexed. Can anyone out there help me out on this one? Tom -- Posted via http://www.ruby-forum.com/.