Xapian discuss - May 2007 - Parallel installation of Python bindings

Thomas Waldmann asked me a question on IRC, about how to cope with 
upgrading his application (MoinMoin) to use the latest version of Xapian 
if available, but still work with the older releases (so that, for 
example, users can just use the version of Xapian available from their 
linux distribution), and, in particular, if other applications require a 
older Xapian to be installed, how to avoid conflicts.

I thought a bit about this over the weekend, but I can't see any easy 
way to allow multiple versions of the Xapian python bindings to be 
installed at once.  We could put the Xapian bindings in versioned 
directories (omitting the revision part of the version number probably, 
since the API should be backwards compatible within minor versions), but 
this would require some kind of magic "xapian.py" file which loaded
the
appropriate version, and allowed a specific version to be requested.

This seems like overkill for the problem, and I can't think of a simpler 
solution, so I suggest we don't attempt to allow parallel (system-wide) 
installation of Xapian.  A package could always include a Xapian 
installation inside it, of course, if it needs to use a particular 
version of Xapian.

Other than that, my best advice is to use only features which are 
present in 0.9 in your applications until 1.0 becomes widespread, and to 
only upgrade use of deprecated interfaces to the new interfaces, and use 
new features, once you're happy to require your users to have 1.0 installed.

Incidentally, Olly is working on debian packages for 1.0.0 (I'm not sure 
if he's quite finished, but he's certainly made progress), so
they'll be
available soon.

-- 
Richard

On Tue, May 29, 2007 at 12:23:43PM +0100, Richard Boulton wrote:
> Thomas Waldmann asked me a question on IRC, about how to cope with 
> upgrading his application (MoinMoin) to use the latest version of Xapian 
> if available, but still work with the older releases (so that, for 
> example, users can just use the version of Xapian available from their 
> linux distribution), and, in particular, if other applications require a 
> older Xapian to be installed, how to avoid conflicts.
I agree that this isn't practical to do. We're not intending to make
backwards incompatible changes often enough to really need it (I hope
:-).

Incidentally, for MoinMoin although they're reasonably careful about
getting dependencies with BC issues right, the entire thing still has
a significant upgrade headache, because each time a new version comes
out the processing and memory requirements go up, sometimes by
incredible amounts. From memory, 1.1 to 1.3 (I think - it occurs to me
that they are the version numbers of the Big SWIG Rewrite, so they
might be wrong here) took us from a usable system to something that
took minutes to load any page, on a machine that quite frankly should
have been able to cope. (Upgrades since then have been easier, but
we've been upgrading our hardware faster since then for other
reasons.)

Other projects don't even tend to worry about this sort of thing.

J

-- 
/--------------------------------------------------------------------------\
  James Aylett                                                  xapian.org
  james@tartarus.org                               uncertaintydivision.org

On Tue, May 29, 2007 at 12:23:43PM +0100, Richard Boulton
wrote:
> Thomas Waldmann asked me a question on IRC, about how to cope with 
> upgrading his application (MoinMoin) to use the latest version of Xapian 
> if available, but still work with the older releases (so that, for 
> example, users can just use the version of Xapian available from their 
> linux distribution), and, in particular, if other applications require a 
> older Xapian to be installed, how to avoid conflicts.
I don't think this should be necessary - we've tried where possible to
ensure that code that worked with 0.9.x just works with 1.0.0.  If it
used deprecated features which have been removed, it will need updating
but we only removed features which have been deprecated since 0.9.0 or
earlier, so it's easy to update code to work with both 0.9.x and 1.0.0.
That was the main aim of the deprecation policy in my mind.

There are a few exceptions though, such as the return type of
MatchDecider::operator(), but we've typically done this where there's
an easy workaround (in this case, it only affects C++ and C#, and in
both cases the preprocessor can be used to conditionally compile code
with the correct return type).

Is there a particular change that is problematic?  It may be too late to
fix now, but we should certainly try to learn from any such mistakes.
And perhaps we can add a compatibility shim for 1.0.1.
> I thought a bit about this over the weekend, but I can't see any easy 
> way to allow multiple versions of the Xapian python bindings to be 
> installed at once.
Well, you can just follow the "Installing without root access" section
of xapian-bindings' "INSTALL" document.  It's not especially
elegant
but it works (it's essentially how we run the "smoketest" for each
of
the languages).
> This seems like overkill for the problem, and I can't think of a
simpler
> solution, so I suggest we don't attempt to allow parallel (system-wide)
> installation of Xapian.
For the bindings I assume you mean?  The library versioning and
libtool's rpath handling mean this should just work for C++.

I think it would be a mistake to add a lot of magic here.  The
"Installing without root access" tips do provide a way for users to
make
this work if they really need it; if there are small tweaks to make
those work better, we should probably consider making them.
> Incidentally, Olly is working on debian packages for 1.0.0 (I'm not
sure
> if he's quite finished, but he's certainly made progress), so
they'll be
> available soon.
I'm making progress, but it's taking a lot of iterations because a lot
has changed.  And there's quite a lot of binary packages to build too
which takes time.  Hopefully I should be done this week though.

Cheers,
    Olly