Xapian devel - Apr 2006 - Another PHP 5 wrapper...

Hi all ! , Bonjour ? tous !

n.b.
This is my first post to the list : please correct me if there is 
anything I don't do well.
Also, please note that English is not my native language, so forgive me 
(or correct me) for all the mistakes I will certainly make !

<introduction>

I started to play with Xapian some weeks ago. We are looking for a 
search solution to replace our own database fulltext search engine which 
is getting obsolete (I wrote it many years ago...)

I considered many libraries like Lucene, mnoGoSearch, Sphinx, Zebra, 
Mysql and PostgreSql  fulltext indices, and so on and I must say that 
Xapian is the best 'thing' that I found and it is progressively becoming
one of my best friend ;-)

What I definitely like in Xapian is :
- the ability to do real-time (re)-indexing on added/modified/deleted 
documents. This is something that products like lucene and others are 
missing (it is a frequent request in the lucene lists)
- the multiple bindings which make it usable from many-languages (our 
current target is PHP)
- the simple but very didactic documentation (e.g. intro_ir.html) and 
the comprehensive answers people like Olly Betts, James Aylett and 
others give to all questions posted on the lists
- the very nice OO api wich make it really simple to use
- the versatility of the product which allows its use in a wide field of 
applications

In fact, I don't know enough English words to say 'congratulations'
and
'thank you' to all who contributed to this great piece of software !

</introduction>

For the same reasons than Paul Dixon pointed out :

http://article.gmane.org/gmane.comp.search.xapian.general/2668
http://blog.dixo.net/2006/04/04/xapian-php5-wrapper/

we also started to make a PHP 5 wrapper around the Xapian php api 
generated by swig.

Ideally, swig will one day be able to generate OO bindings for PHP which 
will make our wrapper useless.
If not, perhaps we will start to write a native PHP extension...

Our goals were:
- target : PHP >= 5.1
- PHP api as friendly as the C++/Java ones
- ability to use native PHP interfaces and constructs like iterators, 
countable, arrayAccess, __toString and so on
- complete documentation for this API (phpdoc)
- ability to replace this wrapper by a native php extension with very 
few impact on client's code.

For now, we have an incomplete (it misses some iterators and some 
classes), not fully tested, but functional wrapper.
Naming conventions used are similar to those used in the java api (we 
use $doc->setData() and not $doc->set_data(), for example).

The wrapper is available here :
http://www.bdsp.tm.fr/aed/xapian/phpXapian-0.1.zip

and the documentation is available here :
http://www.bdsp.tm.fr/aed/xapian/documentation/index.html

Please consider this as very 'alpha' software : we are still refactoring
the api and the doc, we have not yet written unit tests (we use 
simpletest), and we did not do any performance test for now.

Nevertheless, we used it to create and search a 2,7 Gb database, and it 
runs very well.

I don't know how our wrapper compare with Paul's one. Paul's API is
very
similar to the C++ one, and he used a lot the PHP '__call' magic 
function. It's a brillant idea because most of methods get
'magically'
defined : size of his wrapper is half of our's and it's probably easier 
to add new method. But extensive use of '__call' probably have a cost, 
and it is not easy to have php documentation at the method level.

The fact that two so similar wrappers appear means that there is a real 
need for an OO API in PHP.
But having two PHP wrappers for Xapian would just add confusion and lost 
of time, so we will discard our wrapper if Paul's one become the 
'official' one.

All usefull comments are welcome !

Best regards,

-- 

Daniel M?nard

Banque de Donn?es Sant? Publique
Avenue du Professeur L?on Bernard
35043 Rennes C?dex

T?l. (+33) 2.99.02.29.42
Fax (+33) 2.99.02.26.28
E-mail Daniel.Menard@Bdsp.tm.fr
http://www.bdsp.tm.fr

Paul, it seems that you replied directly to me and not to the list, so I 
will quote your message here with my answers interceeded...

Paul Dixon a ?crit :
>>we also started to make a PHP 5 wrapper around the Xapian php api
>>generated by swig.
>>
>>Ideally, swig will one day be able to generate OO bindings for PHP which
>>will make our wrapper useless.
>>If not, perhaps we will start to write a native PHP extension...
>>    
>>
>
>Wow, nice work! That should make Xapian accessible to even more PHP
>developers
>
Yes, there is a real need, and I think that Xapian is a serious 
competitor for Zend_search, the php implementation of Lucene in the Zend 
Php framework.
>the one gripe I'd have is "why change the method names?"
>- I would think it best to stay close to the C++ API so that if a
>better "native" wrapper becomes available, code can be
transitioned
>more easily?
>
>Using the same method names also allows easier communication of one's
>ideas with people using Xapian on other platforms. Indeed, that was
>one my main motivations in making the wrapper.
>
>  
>
I agree that this is a discutable choice, but let me advocate :

- I started to develop this wrapper for our own needs, so I applied our 
own naming conventions.
- it's becoming a current practice in PHP developpement to use 
"lowercaseCamelCase" as a naming convention for methods (zend
framework,
symfony, ez components and many other use it).
- using the same naming conventions as major php libraries can be an 
advantage for spreading Xapian in php developpement (the "inclusion"
is
more natural)
- the changes I made are not so important and they are quasi
"automatic"
: the method names are the same, it is just the way they are written 
(set_data // setData). I'm not convinced that it can be a major problem 
for sharing ideas with other persons.
- a similar choice was done in the java wrapper for Xapian. In fact, my 
wrapper is very similar to the java one. For example, look at :
http://svn.xapian.org/trunk/xapian-bindings/java/org/xapian/Document.java?view=markup
- I don't know what will be the API for a "native" wrapper, but I 
suppose that swig can generate any method's name (can't it ?). So
it's
not impossible to imagine that an OO wrapper would follow the same 
naming conventions.
>>I don't know how our wrapper compare with Paul's one. Paul's
API is very
>>similar to the C++ one, and he used a lot the PHP '__call' magic
>>function. It's a brillant idea because most of methods get
'magically'
>>defined : size of his wrapper is half of our's and it's probably
easier
>>to add new method. But extensive use of '__call' probably have a
cost,
>>and it is not easy to have php documentation at the method level.
>>    
>>
>
>You're correct that my wrapper doesn't allow generation of
>documentation, but it does have the advantage that new methods are
>automatically supported without any wrapper changes. That's just a
>lucky side effect, I only did it that way to save some time :)
>  
>
It is really a brilliant idea, and I will certainly re-use it for rapid 
developpement.
However, apart from documentation, there are other drawbacks : no 
auto-completion nor quick help in IDEs, no possibility of reflexion on 
such classes, overloading methods is not intuitive, and so on.
>Performance-wise, I haven't done much benchmarking aside from timing
>the smoketest - my class-based version of the smoketest runs 2-3 times
>slower than the function based smoketest included in the
>xapian-bindings package. That doesn't mean an awful lot, so I'll do
>some more detailed benchmarking to give a clearer idea of the overhead
>in calling an API via the class is.
>  
>
Basically, I think that our two wrappers will have about the same 
overhead, except if using "__call" is many times slower than using a 
direct call. It is also probable that overhead of the wrapper will be 
negligible on a real corpus containing many documents.
So it will be great if you can do detailed benchmarking.
>It would be great to see a "native" PHP5 class extension though...
>  
>
Yes, we totally agree on that point!
(and I would use such an extension whatever naming conventions used !!!)

-- 

Daniel M?nard

Banque de Donn?es Sant? Publique
Avenue du Professeur L?on Bernard
35043 Rennes C?dex

T?l. (+33) 2.99.02.29.42
Fax (+33) 2.99.02.26.28
E-mail Daniel.Menard@Bdsp.tm.fr
http://www.bdsp.tm.fr