Xen devel - Jun 2006 - Why do we use /usr/lib/python to install python libraries?

This seems exceptionally evil and is definitely a PITA.  What was the 
original problem that led us to do this?  I''d like to take a stab at 
correcting it so we can have a more sane installation.

Regards,

Anthony Liguori

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> This seems exceptionally evil and is definitely a PITA.  What was the
> original problem that led us to do this?  I''d like to take a stab
at
> correcting it so we can have a more sane installation.
I thought it involved some pain early one with supporting different Python 
versions in different distros?  Maybe with the binary installer?

Of course I could be talking bunk - it''s been like that for a loooooong
time
and my memory was better when I was younger ;-)

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Mark Williamson wrote:
>> This seems exceptionally evil and is definitely a PITA.  What was the
>> original problem that led us to do this?  I''d like to take a
stab at
>> correcting it so we can have a more sane installation.
>>     
>
> I thought it involved some pain early one with supporting different Python 
> versions in different distros?  Maybe with the binary installer?
>
> Of course I could be talking bunk - it''s been like that for a
loooooong time
> and my memory was better when I was younger ;-)
>   
Yeah, this is why I asked.  I have vague recollections of this 
discussion too but can''t remember the exact reasons.  I''m
hoping that
we''ve gotten smarter since then and can find a better solution :-)

Regards,

Anthony Liguori
> Cheers,
> Mark
>
>   

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 9 Jun 2006, at 03:00, Anthony Liguori wrote:
> Mark Williamson wrote:
>>> This seems exceptionally evil and is definitely a PITA.  What was
the
>>> original problem that led us to do this?  I''d like to take
a stab at
>>> correcting it so we can have a more sane installation.
>>>
>>
>> I thought it involved some pain early one with supporting different 
>> Python versions in different distros?  Maybe with the binary 
>> installer?
>>
>> Of course I could be talking bunk - it''s been like that for a 
>> loooooong time and my memory was better when I was younger ;-)
>>
>
> Yeah, this is why I asked.  I have vague recollections of this 
> discussion too but can''t remember the exact reasons.  I''m
hoping that
> we''ve gotten smarter since then and can find a better solution :-)
To be able to install on a machine different from the one we built on. 
Because our distribution method is to ''python install'' into a
dist/
subdirectory and then basically blat that directory into / on the 
target machine. It would probably be smarter to include python scripts 
to be able to ''python install'' out of dist/ -- it''d
require us to have
python-devel and C compiler installed on the target box though.

Ultimately the argument was that distros should install in their usual 
place because they *know* what python version they ship, but that since 
we didn''t we''d just install in /usr/lib/python because that
required
least thought for maximum coverage. :-)

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser wrote:
>
> On 9 Jun 2006, at 03:00, Anthony Liguori wrote:
>
>> Mark Williamson wrote:
>>>> This seems exceptionally evil and is definitely a PITA.  What
was the
>>>> original problem that led us to do this?  I''d like to
take a stab at
>>>> correcting it so we can have a more sane installation.
>>>>
>>>
>>> I thought it involved some pain early one with supporting different
>>> Python versions in different distros?  Maybe with the binary
installer?
>>>
>>> Of course I could be talking bunk - it''s been like that
for a
>>> loooooong time and my memory was better when I was younger ;-)
>>>
>>
>> Yeah, this is why I asked.  I have vague recollections of this 
>> discussion too but can''t remember the exact reasons. 
I''m hoping that
>> we''ve gotten smarter since then and can find a better solution
:-)
>
> To be able to install on a machine different from the one we built on. 
> Because our distribution method is to ''python install''
into a dist/
> subdirectory and then basically blat that directory into / on the 
> target machine. It would probably be smarter to include python scripts 
> to be able to ''python install'' out of dist/ --
it''d require us to have
> python-devel and C compiler installed on the target box though.
Does anyone make use of the whole dist/ directory separation?  I''ve 
always found that to be very odd.

Regards,

Anthony Liguori
>
> Ultimately the argument was that distros should install in their usual 
> place because they *know* what python version they ship, but that 
> since we didn''t we''d just install in /usr/lib/python
because that
> required least thought for maximum coverage. :-)
>
>  -- Keir
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Anthony Liguori wrote:
> Keir Fraser wrote:
>>
>> On 9 Jun 2006, at 03:00, Anthony Liguori wrote:
>>
>>> Mark Williamson wrote:
>>>>> This seems exceptionally evil and is definitely a PITA. 
What was the
>>>>> original problem that led us to do this?  I''d like
to take a stab at
>>>>> correcting it so we can have a more sane installation.
>>>>>
>>>>
>>>> I thought it involved some pain early one with supporting
different
>>>> Python versions in different distros?  Maybe with the binary
installer?
>>>>
>>>> Of course I could be talking bunk - it''s been like
that for a
>>>> loooooong time and my memory was better when I was younger ;-)
>>>>
>>>
>>> Yeah, this is why I asked.  I have vague recollections of this 
>>> discussion too but can''t remember the exact reasons. 
I''m hoping that
>>> we''ve gotten smarter since then and can find a better
solution :-)
>>
>> To be able to install on a machine different from the one we built on. 
>> Because our distribution method is to ''python
install'' into a dist/
>> subdirectory and then basically blat that directory into / on the 
>> target machine. It would probably be smarter to include python scripts 
>> to be able to ''python install'' out of dist/ --
it''d require us to have
>> python-devel and C compiler installed on the target box though.
> 
> Does anyone make use of the whole dist/ directory separation? 
I''ve
> always found that to be very odd.
> 
/me waves.. I never use make install, only make dist. Then I use the 
dist directory to maintain updates across hosts.  The other main reason 
is because I never want to actually install Xen when I build it.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, Jun 08, 2006 at 08:09:06PM -0500, Anthony Liguori wrote:
> This seems exceptionally evil and is definitely a PITA.  What was the 
> original problem that led us to do this?  I''d like to take a stab
at
> correcting it so we can have a more sane installation.
If you''re going to fix this, then great.  Perhaps you could make it:

a) use the proper Python distutils if installed;

b) fall back to using /usr/lib{,64}/python if distutils is not installed
   (though I can''t imagine that there are many systems that are
shipping with
   Python but without distutils, and you do need Python to run Xend, after
   all);

c) make sure that we actually do install into /usr/lib64 on appropriate
   systems.  This is a current bug, though I''ve no idea when it got
   introduced.  We need to install into /usr/lib64 because xc.so and xs.so are
   compiled native code.

Thanks Anthony,

Ewan.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ewan Mellor wrote:
> On Thu, Jun 08, 2006 at 08:09:06PM -0500, Anthony Liguori wrote:
>
>   
>> This seems exceptionally evil and is definitely a PITA.  What was the 
>> original problem that led us to do this?  I''d like to take a
stab at
>> correcting it so we can have a more sane installation.
>>     
>
> If you''re going to fix this, then great.  Perhaps you could make
it:
>   
I would love to, if patches are willing to be taken.
> a) use the proper Python distutils if installed;
>   
I think we already are aren''t we?  I thought the crux of the problem
was
that people are relying on having a deterministic install path so that 
it can be deployed on many machines (regardless of Python version).  We 
would have to break this "feature".  I''m happy to do that of
course :-)

Regards,

Anthony Liguori
> b) fall back to using /usr/lib{,64}/python if distutils is not installed
>    (though I can''t imagine that there are many systems that are
shipping with
>    Python but without distutils, and you do need Python to run Xend, after
>    all);
>
> c) make sure that we actually do install into /usr/lib64 on appropriate
>    systems.  This is a current bug, though I''ve no idea when it
got
>    introduced.  We need to install into /usr/lib64 because xc.so and xs.so
are
>    compiled native code.
>
> Thanks Anthony,
>
> Ewan.
>   

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Jun 14, 2006 at 06:45:12AM -0500, Anthony Liguori wrote:
> Ewan Mellor wrote:
> >On Thu, Jun 08, 2006 at 08:09:06PM -0500, Anthony Liguori wrote:
> >
> >  
> >>This seems exceptionally evil and is definitely a PITA.  What was
the
> >>original problem that led us to do this?  I''d like to take
a stab at
> >>correcting it so we can have a more sane installation.
> >>    
> >
> >If you''re going to fix this, then great.  Perhaps you could
make it:
> >  
> 
> I would love to, if patches are willing to be taken.
Certainly we''ll take the patch.
> >a) use the proper Python distutils if installed;
> >  
> 
> I think we already are aren''t we?  I thought the crux of the
problem was
> that people are relying on having a deterministic install path so that 
> it can be deployed on many machines (regardless of Python version).  We 
> would have to break this "feature".  I''m happy to do
that of course :-)
I meant use distutils in the install script, i.e. on the destination machine.
What people want to be able to do is create a dist directory, copy it
wholesale over to a target host, and then run install.sh and have the right
thing happen.  If the install.sh was clever enough to drop things into the
correct Python directory, then that''s great and doesn''t break
anything.

In the case where people use "make install" directly, yes, distutils
is used
already.

Some people copy the dist directory straight onto the root directory on their
test box (many of the developers here, for example).  That''s not really
supported, because you end up with both the hotplug and udev scripts
installed, for example, but it is handy for developers, so it would be good to
preserve that behaviour.  In other words, you still need a
dist/usr/lib/python/ directory to be created, even if install.sh
doesn''t copy
those files directly to that directory when it runs.

Thanks,

Ewan.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Jun 14, 2006 at 08:32:39AM +0100, Ewan Mellor wrote:
> c) make sure that we actually do install into /usr/lib64 on appropriate
>    systems.  This is a current bug, though I''ve no idea when it
got
>    introduced.  We need to install into /usr/lib64 because xc.so and xs.so
are
>    compiled native code.
Could you clarify this? What happens on 64-bit Linux installations right
now?

Since the 64-bit cleanness isn''t quite there yet, we need to install
both a 64-bit and 32-bit on Solaris. Unfortunately Python doesn''t seem
to have much support for running like this (in particular, it''ll just
give up if it finds an extension .so of the wrong elf class rather than
looking in a 64-bit path or whatever).

thanks
john

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Jun 14, 2006 at 04:31:56PM +0100, John Levon wrote:
> On Wed, Jun 14, 2006 at 08:32:39AM +0100, Ewan Mellor wrote:
> 
> > c) make sure that we actually do install into /usr/lib64 on
appropriate
> >    systems.  This is a current bug, though I''ve no idea when
it got
> >    introduced.  We need to install into /usr/lib64 because xc.so and
xs.so are
> >    compiled native code.
> 
> Could you clarify this? What happens on 64-bit Linux installations right
> now?
> 
> Since the 64-bit cleanness isn''t quite there yet, we need to
install
> both a 64-bit and 32-bit on Solaris. Unfortunately Python doesn''t
seem
> to have much support for running like this (in particular, it''ll
just
> give up if it finds an extension .so of the wrong elf class rather than
> looking in a 64-bit path or whatever).
My belief was that, because we have native code, on those platforms that
normally use /usr/lib64, we ought to be putting our stuff in /usr/lib64.
Does anyone know whether this is the case?

Currently, all the Python stuff goes into /usr/lib/python, even on platforms
with a /usr/lib64 directory.  I was assuming that that was a bug, but I
don''t
know for sure.

Ewan.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel