similar to: Question about the ticket #743 omindex: delay libmagic checks

> > I'd suggest to start with you just look at moving the libmagic check after > the filesize checks, so you don't need to get into whether libmagic or > the database check is cheaper on average. hi, Olly, I have moved the libmagic check after the filesize check directly, https://github.com/caiyulun/xapian/commit/3a97d9ee5397fa900a473aa9b3d8eeb720177a4e can you provide

Olly (looking at commit logs, I think this is your dept :-) For apps which re/index files frequently and need format conversion, I'd like to propose a patch for one of... Omindex library (thread safe): Omindex::init(options) // struct Omindex::options { ... } initialize mime_map, store default options session = new Omindex::Session(db_pathname) user threads use different sessions

Hi, all -- I want to do Plain Old Omindex'ing *but* the mapping between my documents' filenames and the URLs where I hope search users to find them is, uh..., strange. The simplest thing (to me) would be to run omindex for each document, e.g. omindex --no-delete -U /cool-url-1 /funky/doc/file-blah.pdf omindex --no-delete -U /cool-url-7 /doc/funky/ohmy/blah-file.txt ... and so on...

All, I'm having some issues compiling omega. Here are the particulars I'm on win7, using cygwin 4.9.2 64 bit. Here's the relevant output from make: libtool: link: g++ -fshow-column -Wall -W -Wredundant-decls -Wpointer-arith -Wca st-qual -Wcast-align -Wno-long-long -Wformat-security -fno-gnu-keywords -Wundef -Woverloaded-virtual -Wstrict-null-sentinel -Wshadow -Wstrict-overflow=1

I have around 550,000 files (4.7GB) that I need to index. It is a huge mix of file types. I don't need access to this via web. I just use for research locally. For now I do a grep and wait several minutes. omindex complains of Unknown extension: .... - skipping As I have many thousands of files that don't have extensions. (No Period.) Any way to use omindex to index regardless of

Hi, I'm trying to build 5-STABLE, I have cvsuped the latest source, cleared out /usr/obj and I still get this problem. Any idea what could be causing it? Mark ===> lib/libmagic cat /usr/src/lib/libmagic/../../contrib/file/Header /usr/src/lib/libmagic/../../contrib/file/Localstuff /usr/src/lib/libmagic/../../contrib/file/Magdir/zyxel /usr/src/lib/libmagic/../../contrib/file/Magdir/xdelta

If libmagic isn't installed then the guestfs_file_architecture API doesn't work. This means that inspection will always return <arch>unknown</arch> for every guest. This subtly breaks a few features. In particular it was reported that the virt-builder/virt-customize --install option did not work because the "unknown" architecture of the guest was not compatible

HI all, I'm just testing out the capabilities of xapian and omega. Environment - Fedora15. Disk to be indexed - 2GB? - FAT16 filesystem. Named "New Volume" When I add a text file to the disk, by right-clicking in Fedora and choosing Create New - > Text File The system creates the text file as expected. I added some content/words, however, xapian-omega will not index it:

I would really like to try out the features in the patch above. But I can't ever seem to get the resulting omindex.cc to "make". I tried updating to rev 10801 from the SVN then run /bootstrap but then I seem to get errors compiling everything when I try and do "make" (I'm using ubuntu 8.10). So I thought I'd try an apply the patch to the latest stable version

Given a directory tree like ... /foo | +-- A | +-- B | +-- C ... what is the best way to index A and C into a single Xapian database? AFAIK the alternatives are: omindex --db /my_db --no-delete /foo /foo/A omindex --db /my_db --no-delete /foo /foo/B or omindex --db /my_A_db /foo /foo/A omindex --db /my_B_db /foo /foo/B xapian-compact /my_A_db /my_B_db /my_db The first alternative does not

Hi. I am writing a python equivalent of omindex (we are using scriptindex currently - but I wanted to use omindex instead, and extend it to work with our internal file format.. BUT did not want to compile code if possible... so anyway). I have tried to keep the code as close to possible to the omindex native code, but am facing a bit of confusion: what exactly is the reason for omindex to take

Hi, I was researching indexing of text in omindex and scriptindex. While indexing text with omindex.cc possition of terms is saved with gap. This is not happening with scriptindex.cc While this is happening ? Another question is why in omindex.cc the term possition starts with 0 while in scriptindex it starts from 1 ? Code snippet from omindex.cc // Add postings for terms to the document

Hi all I need a little clarification with regard to Omega's behaviour with 'duplicate' files when running 'omindex'. How is a duplicate recognised? Is it simply by file path? How is an unmodified file detected, if at all? I would like to set up subversion post-commit hook to update my index. If possible I would like to just update the index with the newly commited files.

Hi all, I'm having a recurrent problem with Omega's indexing. When I run omindex, it sometimes misses to recognize the extension of some files (.doc, .pdf) and skips them. In the same run, omindex is otherwise perfectly able to index other files with same extensions. The reason is not clear but it should occur before it selects a content converter since for example, if I manually run

Hello, I was looking for a search engine for a small internal documentation site and found xapian and omega. Downloaded and compiled it using msys and ming on a german windows xp system. Finally installed apache on the same box. Following the omega example I copied the book to .../apache/htdocs and startet the omindex which hang up on the first document found. Even on very short doc with

hi there, is there an option to exclude child folders when running omindex? For example: omindex -p --db /var/blah/default --url /something /var/www --exclude /var/www/ignore Thanks, Jeff

James, That was exactly the issue. libmagic.dll.a was in /lib under cygwin. Adding a -L/lib took care of this. This was also an issue with -lpcre, which adding -L/lib fixed as well. Of course, I'm now running up against something else. from make libtool: link: g++ -fshow-column -Wall -W -Wredundant-decls -Wpointer-arith -Wca st-qual -Wcast-align -Wno-long-long -Wformat-security

Hi, I would like to add "text/x-mail" support to omega. I'm using mhonarc to export mail to HTML format and I'm using HTML parse to index mail content (largely inspired by "application/vnd.ms-outlook" format). The problem is that files attached to the mail are not indexing at all. I think it's not possible in "index_file" function to index 2 files as one

Hi guys: I was trying to index a large set of PDF documents using omindex and the system started to run out of forks (sh: fork temporarily unavailable) making the system unusable and probably skipping documents. I'm using MAC Osx Server 10.4.3 (Darwin/BSD) and GCC 4.0. The problem: On function stdout_to_string a popen is called, but is not closed properly (according the popen

This is a list with some libraries that I have been looking at. The idea is to discuss the advantages and disadvantages of adding some of these libraries to Xapian. If anyone knows another library that could be add to the list it would be great! Libfreexl: * For Excel (.xls) * Last release: 2018-02 * Info: gaia-gis.it/fossil/freexl/index * License: MPL tri-license