Hello All..n00bie alert. Right I have installed xapian 0.8.5 core, bindings (just for PHP as I might need that once i get out of n00b mode), examples and omega. All good so far... But I am now stuck, I can't find any documentation to explain how to do what I am about to explain... The reason for trying xapian out is because of your features page: http://www.xapian.org/features.php "You can also index data from any SQL or other RDBMS supported by the Perl DBI module. That includes MySQL, PostgreSQL, SQLite, Sybase, MS SQL, LDAP, and ODBC." I have a large MySQL database 488,446 records and its 304 meg in size. I currently have a full-text search on one column and I also limit the search query using "AND locationid = whatever etc" Fulltext using mysql is soo slow..can be 6 or so seconds..I have optimised mysql so its not that (could be a hardware issue but I don't have the cash just yet) Can someone point me in the right direction (the right documentation as I can't spot it) on how to index a mysql database and also tell me if its possible to use xapian/omega to do the limiting of the data like I would with mysql AND statments. Cheers John Wards
On Tue, Feb 08, 2005 at 04:46:52PM +0000, John Wards wrote:> Can someone point me in the right direction (the right documentation as > I can't spot it) on how to index a mysql database and also tell me if > its possible to use xapian/omega to do the limiting of the data like I > would with mysql AND statments.I'm a little pressed for time right now, so I can't actually point you in exactly the right direction (sorry :-), but yes you can do this. What I suggest you do is to look at the omega documentation for scriptindex, and then write a PHP CLI script that generates a scriptindex data file from your mysql database (it'll take a while to run, but will give you initial population of your database - once you're up and running you can integrate live changes to the Xapian database when you change the mysql db). For the AND stuff, you'll want to generate 'boolean' terms in the Xapian documents. I can't remember how good the documentation is at pointing in the right direction here, but basically what you want to do is define an all-capitals prefix such as XL for your location ids, and then generate terms that might be XL1, XL27 for location id = 1, 27 ... and so on. Then, when searching using omega, you can add in B=XL1 or whatever and the boolean terms (B for boolean) will be used to filter the documents returned by Xapian (it's a little more complex than that, but it should get you pointed in the right direction). (The thing about indexing using Perl DBI refers to a script that will do some of the above work for you, but I can't find it right now. Hence the way I've suggested above, which will do what you want without having to go near Perl :-) J -- /--------------------------------------------------------------------------\ James Aylett xapian.org james@tartarus.org uncertaintydivision.org
Sent to the list this time...stuipid email client..sorry richard ________________ Original message ________________ Subject: Re: [Xapian-discuss] Am I looking in the right place Author: "Richard Boulton" <richard@tartarus.org> Date: 08th February 2005 6:36:0 PM Take a look at "dbi2omega" in the omega sources directory. You'll probably want to fiddle a little with that, but once you've done that you can send the output from it directly to scriptindex (once you've set up a suitable indexing configuration for scriptindex). ------------------------------------------------------------------- Oooh my head. If some one has the time could they help me get my head around this.. I have a database called test. the table is called properties. i have the following colums id, address, locationid i need to search that database and return the ids. I am completly lost..i read the ir introduction..i did a module at uni but i am more confused at the techincal end. i am guessing that omega is the best/easiest way to go but i find the documentation confusing..i'm not usually confused by documentation...but it is late. please be gental with me.......