Displaying 1 result from an estimated 1 matches for "exclude_docu".
2007 Jan 23
3
Someone getting RDig work for Linux?
...# want to index. Useful to avoid indexing common elements like
navigation
# # and page footers for every page.
# :title_tag_selector => lambda { |tagsoup|
# tagsoup.html.head.title
# }
# )
)
# crawler options
# Notice: for file system crawling the include/exclude_document
patterns are
# applied to the full path of _files_ only (like /home/bob/test.pdf),
# for http to full URIs (like http://example.com/index.html).
# nil (include all documents) or an array of Regexps
# matching the URLs you want to index.
cfg.crawler.include_documents = nil
# nil (...