Xapian devel - May 2006 - Documentation for the PHP OO wrapper

I finally had a chance to spend some time on Olly's idea about 
generating documentation for the object-oriented PHP wrapper
http://www.oligarchy.co.uk/xapian/patches/xapian9.phps
>Another way to approach it might be to postprocess Doxygen's XML output
>(which would allow mechanical changes to match PHP syntax).
>  
>
I wrote an xslt script to test the idea (It is one of the first I write, 
so there are surely a lot of "bad things" in it, don't hesitate to
point
them !).
You can grab the xslt file from here (28 Kb) :
http://www.bdsp.tm.fr/aed/xapian/wrapperDoc/doc.xslt

It produces a all-in-one html file (331Kb) :
http://www.bdsp.tm.fr/aed/xapian/wrapperDoc/index.html

(Don't mind about disliking the look of the result : it is done with a 
css stylesheet included in the file and can easily be changed. ;-))

The only reason for having a standalone result file is that I don't know 
how to produce multiple outputs from a stylesheet, but I think that it 
can be an advantage : easy to download, to print, to mail and so on.

Generating this doc from a Xapian distribution requires three steps :
1. ask Doxygen to produce xml output (as a quick test, I directly 
modified "xapian-core/docs/doxygen_api_conf", setting GENERATE_XML to 
"YES" and ran "doxygen doxygen_api_conf", but the way to go
is probably
: change the ".in" file, ./configure --enable-maintainer-mode, make
docs
or something like that). The xml output is generated in
"docs/apidoc/xml/".
2. combine all xml files generated by Doxygen in a file. Doxygen gives 
all we need and generates an xslt for that, so we just have to run 
"xsltproc combine.xslt index.xml >all.xml" from the xml output
directory.
3. apply my xslt file on the file obtained to generate binding 
documentation : "xsltproc doc.xslt all.xml > somewhere/index.html"

The xslt file contains some configuration options (search for "Config"
in the file) like :
- generating table of contents for classes and methods
- specifying how methods signatures are written
- using camelCase for methods names, like in :
http://www.bdsp.tm.fr/aed/xapian/wrapperDoc/indexCamelCase.html
- adding links to php web site for basic php types,
- etc.

The xslt file try to solve many problems :
- it must decide if a class is to be documented or not (many classes 
which are in the public Xapian API are not in the wrapper).
The condition I set up is to keep only the public classes whose name 
does not contain the words 'Wrapper' , 'Internal',
'Error' and 'Decider'.
- it must decide if a method is to be documented (idem : many methods 
are in the API but not in the wrapper).
I exclude internal constructors, operators, destructors, deprecated 
methods and specific method names like 'clone'
- it must convert xapian types to the types used is the wrapper. Classes 
and enums are adjusted to the name they have in the wrapper and typedefs 
are converted to the equivalent php type.

I'm not very happy with the result because I had to hard code some 
Xapian names in the xslt file, which was not what I wanted (maintenance 
issue).

The generated documentation still have many problems that I was not able 
to resolve :
- some methods are documented but do not exist in the wrapper (e.g. 
empty(), max_size(), swap(), create()...)
- some forms of methods use arguments types which are not in the wrapper 
and should not be documented (e.g : get_mset with a MatchDecider, 
get_eset with an ExpandDecider, SimpleStopper::__construct() with 
iterators...)
- some methods exist in the wrapper but are not documented. This is 
everything which is specific to the bindings :
http://svn.xapian.org/trunk/xapian-bindings/php/docs/bindings.html?view=co
and some which are not documented like (Simple)Stopper::apply().
- some methods have different signatures in the api and in the wrapper 
(e.g. Enquire::_construct, Enquire_set_query...)

So this is a poor result : we have a doc which contains errors... :-( 
How usable is it ?

Generating a better doc require that the xslt file "knows" more about 
what is specific to the wrapper but adding more and more tests and 
conditions in the xslt file is probably not an option.

Perhaps we can use a hand-coded xml file which would be included in 
"all.xml" by using a modified version of combine.xslt.
It would allow to say things like :
- ignore methods xxx which have the signature yyy
- add documentation for method PositionIterator::get_termpos from what 
doxygen generated for "operator *"
- add documentation for a new method MSet::get_document_percentage (doc 
in Doxygen format would follow...)

I started to test that approach : the code is between comments in the 
xslt file (search for 'mappings'), and the xml file I used is here (to 
be copied in "all.xml") :
http://www.bdsp.tm.fr/aed/xapian/wrapperDoc/mappings.xml

I miss time for now to go on, and I am not sure this is the way to go : 
maintenance of this xml file is probably a problem...
On the other hand, the method used to generate the doc for the PHP 
wrapper can probably be re-used to generate specific doc for the other 
bindings. So a generic solution would be better. An xml version of 
xapian.i ?

Any ideas ?

PS : by working on this, I found some errors which can perhaps be 
corrected in the sources :
- MSetIterator::get_document documents an "it" parameter which does
not
exist
- some methods arguments are not documented and don't have a name in 
doxygen output (search for '$arg' in the html file)
- some methods of the wrapper can not be used from PHP 
(Enquire:register_match_decider for example)

Best regards,

-- 

Daniel M?nard

Banque de Donn?es Sant? Publique
Avenue du Professeur L?on Bernard
35043 Rennes C?dex

T?l. (+33) 2.99.02.29.42
Fax (+33) 2.99.02.26.28
E-mail Daniel.Menard@Bdsp.tm.fr
http://www.bdsp.tm.fr

On Wed, May 10, 2006 at 12:08:45PM +0200, Daniel M?nard
wrote:
> >Another way to approach it might be to postprocess Doxygen's XML
output
> >(which would allow mechanical changes to match PHP syntax).
>
> I wrote an xslt script to test the idea (It is one of the first I write, 
> so there are surely a lot of "bad things" in it, don't
hesitate to point
> them !).
I don't know xslt, so I can't comment on that really.
> It produces a all-in-one html file (331Kb) :
> http://www.bdsp.tm.fr/aed/xapian/wrapperDoc/index.html
It looks pretty good.  I suspect __construct should probably be
described as "new <CLASSNAME>" since that's how the user
will call it.
> I'm not very happy with the result because I had to hard code some 
> Xapian names in the xslt file, which was not what I wanted (maintenance 
> issue).
I think it's inevitably going to be somewhat inelegant because the
process of mapping the names has a number of special cases.  It should
be easier to maintain a mapping that to try to maintain parallel PHP
docs by hand - that's likely to result in more errors.  But it's a
problem that we're trying to duplicate knowledge contained in the SWIG
xapian.i and php/util.i files, as well as in SWIG itself to some extent.
> Generating a better doc require that the xslt file "knows" more
about
> what is specific to the wrapper but adding more and more tests and 
> conditions in the xslt file is probably not an option.
If it helps, we could generate a list of which classes and methods get
wrapped pretty easily I think.

Or perhaps we should approach this a different way and try to insert
this documentation into the generated PHP wrapper using SWIG magic (SWIG
can already add python docstrings for example).

If there a "PHP documentation comment" convention, we can use that.
Otherwise doxygen supports PHP "to some extent" apparently.
> I miss time for now to go on, and I am not sure this is the way to go : 
> maintenance of this xml file is probably a problem...
Perhaps, but it was definitely an instructive experiment even if the
conclusion might end up being that it's not the way to go.  So thanks
for trying it.
> On the other hand, the method used to generate the doc for the PHP 
> wrapper can probably be re-used to generate specific doc for the other 
> bindings. So a generic solution would be better. An xml version of 
> xapian.i ?
I would certainly be good to sort this out for all languages the same
way.
> PS : by working on this, I found some errors which can perhaps be 
> corrected in the sources :
> - MSetIterator::get_document documents an "it" parameter which
does not
> exist
Fixed.
> - some methods arguments are not documented and don't have a name in 
> doxygen output (search for '$arg' in the html file)
I had a quick look and I think most of these are cases where the
method is defined inline in the header and the parameter is unused.
We could perhaps include the parameter name and then use a cast
to void or self-assignment to suppress the "unused parameter" error.

I'll have a look...
> - some methods of the wrapper can not be used from PHP 
> (Enquire:register_match_decider for example)
SWIG/PHP doesn't currently support directors (i.e. you the ability to
subclass a wrapped C++ class and then pass an instance of that to a
wrapped C++ function/method which can then call virtual methods
implemented in PHP) so MatchDecider isn't usable from PHP currently.

I suspect it's probably not that hard to implement - if someone can
sketch out what the code would look like for an example, I can
easily sort out generating it.  Otherwise I think I'll try to
finish merging the PHP5 wrapper generation into SWIG first (they've
kindly given me commit access so this should happen fairly soon).

Cheers,
    Olly