similar to: Xapian Search Websites Listings

I think this is a shining example of how well Xapian works with large document collections. I was just discussing this with my colleagues here and one of the issues that came up is that we'd love Xapian to become really lot more popular but have found that the documentation's a bit difficult to get into, as is the API. So I was wondering: do you have any thoughts on improving this and

> I think this is a shining example of how well Xapian works with large > document collections. I was just discussing this with my colleagues here > and one of the issues that came up is that we'd love Xapian to become > really lot more popular but have found that the documentation's a bit > difficult to get into, as is the API. I agree. There are a few gotchas, as well

Lately I was reading many articles about using boolean queries for search engine but I haven't seen any complete working demo. Therefore I put together very simple working demo of search engine using boolean query. Feel free to suggest any performance improvement or error while keeping it as simple as possible for understanding. Thanks, -Kevin Duraj http://myhealthcare.com

Xapian Index 253 million documents = 704G I just build my largest single Xapian index with 253 million unique documents on single server using single hard disk, less that 8G RAM and single processor 2.0 GHz. I do not see any search performance decreases in searching my indexes between 100 million and 250 million, which indicates a good scalability of Xapian and it looks like, I can push it easily

It took approximately five days, having single process using one core CPU and 6GB of memory to build this giant 607GB single Xapian index, containing 219 million of unique documents (web sites). So far I did not found any other implementation that would enable me to build such a single index containing over 200 million documents, while testing Lucene, Solr, MySQL, Hadoop and Oracle. Probably

Xapians, I am maintaining about two indexes for my search engines which approximately is each the same size. I would like to share this knowledge with you, since many of you have never seen Xapian index of this size. And of course you can search the index by yourself at - http://myhealthcare.com/ - http://find1friend.com/ I need 2 x 100 million more documents into each index, and I hope it will

Xapians! If tomorrow Xapian search engine would achieved the same performance and result in searches as Google we would not be able to beat Google, because we would create only a copy of the searches that already exists from Google search engine. However there is a way to beat anyone, and there is a way to beat Google successfully as well just do not give up. Some see it as implementing Ajax, or

Xapian 1.0.3 installation issues, I installed Xapian 1.0.3 and the search would not execute when run as Apache user. I could run the search fine inside ssh. I rolled Xapian to previous version 1.0.2 and the search still does not work even when I put back the old index made by Xapian 1.0.2 ... my search engine is out of work ... Kevin Duraj http://myhealthcare.com

The issue is that delve was renamed to xapian-delve but documentation is still saying that delve is delve. Who has access to update the documentation? http://www.linuxfromscratch.org/blfs/view/svn/general/xapian.html Installed Programs: copydatabase, delve, quest, simpleexpand, simpleindex, simplesearch, xapian-check, xapian-chert-update, xapian-compact, xapian-config, xapian-inspect,

Hi, While trying to setup xapian replication (initially for backup purposes), I'm encountering some errors. Our "fresh" index starts replication, and ends up with an index size that matches the replication master (4.5GB), but then throws : "Getting update for fresh from fresh xapian-replicate: NetworkError: Unable to fully synchronise: Database changing too fast" I

After couple of day of hacking my Fedora 6 server, finally I was able to install the new version of Xapian 1.0.3_svn9466 from trunk. Steps ----------- 1. Removed all old Xapian files and libraries from entire server. 2. Installed Xapian 1.0.3_svn9466. 3. libxapian.so.15 used to be in directory /usr/local/lib64/ however this time the library was in /usr/local/lib/ directory 4. cp

Social Networking Search Google and Bing launches social search that I invented and developed at http://Find1Friend.com/ before running on Xapian. They copy inventors ideas and never gave credit to the inventor, the exact same way when I wrote first social networking site in 1999 at http://MyWBS.com , Friendster, MySpace, Facebook and others copied the idea, made tons of money, held hundreds

On Tue, March 23, 2010 19:46, Kevin Duraj wrote: > I am merging 300 indexes at once, it takes less than a day for merge > to happen for 100 million documents, during merging I notice very heavy IO. That IO sounds pretty normal. To help with IO load, we have a dedicated index store cluster, dedicated source data cluster, dedicated indexing cluster, etc. Sigh. Each time I think we have

On Mon, Jul 11, 2016 at 02:02:56PM -0700, Kevin Duraj wrote: > You are saying that when I search for "delve Xapian 1.4" on Google, a > company worth of 491 Billion of Dollars and you saying that their top > of the search result has nothing to do with Xapian. > > https://www.google.com/search?q=xapian+delve&ie=utf-8&oe=utf-8#q=delve+xapian+1.4 Well, I'm not

Now imagine my situation and probably others too, when we are working with big data. I select 1 billion of YouTube videos, and then I index it with Xapian. Now a kid uploads Pokemon video and for some reason, the kid keeps pressing a single key on the keyboard until the term become 500 characters long (e.g., EEEEEEE). Xapian index is running and after it has indexed 500 million documents,

On Fri, Jul 08, 2016 at 06:42:23PM -0700, Kevin Duraj wrote: > The issue is that delve was renamed to xapian-delve but documentation > is still saying that delve is delve. Who has access to update the > documentation? > > http://www.linuxfromscratch.org/blfs/view/svn/general/xapian.html That website has nothing to do with Xapian, so you probably need to contact whoever runs it.

Olly, many thanks for suggesting htdig, you saved me a lot of time. Htdig looks better than my original idea - wget, you were right. Using htdig, I can crawl and search single website - but I need to integrate search of pages spread over 100+ sites. Learning, learning.... Htdig uses separate document database for every website (one database per URL to initiate crawling). Htdig also can merge

I've uploaded Xapian 1.0.7, which as usual you can download from: http://xapian.org/download This release fixes an assortment of bugs, and improves efficiency in a few cases. It's intended to be a relatively safe incremental update over 1.0.6. For a more detailed overview see: http://trac.xapian.org/wiki/ReleaseOverview/1.0.7 The full lists of user-visible changes are linked to from

James, I would like to propose to change the following code while indexing a term that is larger than 245 characters and then crashing and aborting the entire index, we could rather truncate the term to 245 characters and continue with indexing. if (tname.size() > MAX_SAFE_TERM_LENGTH) throw Xapian::InvalidArgumentError("Term too long (> " STRINGIZE(MAX_SAFE_TERM_LENGTH) "):

On 24 Jan 2016, at 13:48, James Aylett <james-xapian at tartarus.org> wrote: > Xapian?s websites (xapian.org, trac.xapian.org and lists.xapian.org) are now available via https: https://xapian.org, https://trac.xapian.org and https://lists.xapian.org/mailman/listinfo. (Currently https://lists.xapian.org/ redirects to non-https, because of limitations in mailman.) Update: lists.xapian.org