Geir Thomas Andersen | Runbox Solutions AS
2018-Jun-20 02:54 UTC
Xapian ported to WebAssembly for use in web browser
Hi all, First off, apologies if this list is not the correct place for this message. Although we have used Xapian with our email services for a long time (over 10 years) we haven't had the opportunity to contribute or participate in the Xapian community until now. We might therefore not be entirely familiar with acceptable protocol here, but we thought others might be interested in a new project of ours that integrates a fork of the Xapian project. A brief introduction: We are Runbox Solutions AS, a company based in Norway that provides secure and privacy-oriented email services. As mentioned we have for a long time used Xapian as part of our email services, to index large amounts of messages on our servers and provide search capabilities through our webmail interface. We have always been impressed with Xapian's processing speed, reliability, and adaptability, and we continue to be grateful for choosing Xapian for our webmail service back in 2008. Recently we have developed a client based webmail app utilizing a Xapian index that can be downloaded to and generated by the browser. We have done this by porting Xapian to WebAssembly using the C to WebAssembly compiler from emscripten.org. The Xapian database is stored in the browser using IndexedDB, which is available through the IDBFS file system of emscripten. Our fork can be found at https://github.com/runbox/xapian, and the diff can be seen at https://github.com/xapian/xapian/compare/master...runbox:master. The result is a very fast search experience, which can be demonstrated at https://runbox.com/app (please sign up a free trial account at https://runbox.com/signup first). We hope our fork can be of interest to others, and welcome any comments or questions. Best regards, Geir Thomas Andersen Managing Director Runbox Solutions AS
On Wed, Jun 20, 2018 at 04:54:40AM +0200, Geir Thomas Andersen | Runbox Solutions AS wrote:> Our fork can be found at https://github.com/runbox/xapian, and the > diff can be seen at > https://github.com/xapian/xapian/compare/master...runbox:master.That's an impressively small set of changes (fewer than some platforms and compilers have needed).> We hope our fork can be of interest to others, and welcome any > comments or questions.This seems to open up a lot of potentially interesting possibilities, and I think it probably makes sense to try to merge this in so it just works out of the box. It would be good to have a more satisfactory solution to the database locking though. Maybe that can be mapped to some sort of IndexedDB lock or named semaphore? What matters is that all writers agree on the locking mechanism, so there's quite a lot of flexibility here. Cheers, Olly