Fedora xen - May 2006 - Heads-up: Requiring PAE for running Xen

As we move forward with Xen enablement, there''s a desire for
being able to access more than 4 gigs of RAM on 32-bit Xen hosts.  The
options for handling this are
1) Another kernel.  This is bad due to 
   a) we''re running out of CD space already
   b) keeping things matched up between the HV and the guest kernels
   c) migration is worlds of pain with two types of kernels
2) Switch the 32-bit xen kernels to require PAE.  For most "current"
non-laptop hardware, this is a non-issue.  It does mean that xen won''t
work a lot of earlier PentiumM laptops
3) Do nothing, tell people to use 64bit if they want more than 4 gigs of
RAM
4) Make the PAE code handled at runtime.  This is a pretty non-trivial
amount of work :)

Given these, we''re looking at going with #2 and thus only having Xen
work on PAE-capable hardware in the development tree.  And we''re
planning to try to execute this switchover the beginning of next week.
Note that this will not affect bare metal installs at all.

Jeremy

-- 
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

On 5/17/06, Jeremy Katz <katzj@redhat.com> wrote:
> As we move forward with Xen enablement, there''s a desire for
> being able to access more than 4 gigs of RAM on 32-bit Xen hosts.  The
> options for handling this are
> 1) Another kernel.  This is bad due to
>    a) we''re running out of CD space already
>    b) keeping things matched up between the HV and the guest kernels
>    c) migration is worlds of pain with two types of kernels
> 2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
> non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
> work a lot of earlier PentiumM laptops
> 3) Do nothing, tell people to use 64bit if they want more than 4 gigs of
> RAM
> 4) Make the PAE code handled at runtime.  This is a pretty non-trivial
> amount of work :)
>
> Given these, we''re looking at going with #2 and thus only having
Xen
> work on PAE-capable hardware in the development tree.  And we''re
> planning to try to execute this switchover the beginning of next week.
> Note that this will not affect bare metal installs at all.
>
> Jeremy
>
> --
> fedora-devel-list mailing list
> fedora-devel-list@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-devel-list
>
xen always needed p6 hardware (didnt run on k6-II for example),
and AFAIK p6 supports PAE always, so not any lose here i think

-- 
Greetz, Antonio Vargas aka winden of network

http://wind.codepixel.com/
windNOenSPAMntw@gmail.com
thesameasabove@amigascne.org

Every day, every year
you have to work
you have to study
you have to scene.

On Wed, May 17, 2006 at 02:07:59PM -0400, Jeremy Katz
wrote:
> As we move forward with Xen enablement, there''s a desire for
> being able to access more than 4 gigs of RAM on 32-bit Xen hosts.  The
> options for handling this are
> 1) Another kernel.  This is bad due to 
>    a) we''re running out of CD space already
>    b) keeping things matched up between the HV and the guest kernels
>    c) migration is worlds of pain with two types of kernels
> 2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
> non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
> work a lot of earlier PentiumM laptops
How do I know if my laptop if affected? :)
> 3) Do nothing, tell people to use 64bit if they want more than 4 gigs of
> RAM
> 4) Make the PAE code handled at runtime.  This is a pretty non-trivial
> amount of work :)
> 
> Given these, we''re looking at going with #2 and thus only having
Xen
> work on PAE-capable hardware in the development tree.  And we''re
> planning to try to execute this switchover the beginning of next week.
> Note that this will not affect bare metal installs at all.
> 
> Jeremy
> 
-- 
Axel.Thimm at ATrpms.net

On Wed, May 17, 2006 at 11:42:43PM +0200, Axel Thimm
wrote:
> On Wed, May 17, 2006 at 02:07:59PM -0400, Jeremy Katz wrote:
> > As we move forward with Xen enablement, there''s a desire for
> > being able to access more than 4 gigs of RAM on 32-bit Xen hosts.  The
> > options for handling this are
> > 1) Another kernel.  This is bad due to 
> >    a) we''re running out of CD space already
> >    b) keeping things matched up between the HV and the guest kernels
> >    c) migration is worlds of pain with two types of kernels
> > 2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
> > non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
> > work a lot of earlier PentiumM laptops
> 
> How do I know if my laptop if affected? :)
Take a look at the ''flags'' field in /proc/cpuinfo - look for a
flag
called ''pae'' in it - if your CPU is a Pentium Mobile the
chances are
you won''t see the ''pae'' flag there.

For example on my laptop which dosn''t have PAE:

  [berrange@localhost ~]$ grep pae /proc/cpuinfo 
  [berrange@localhost ~]$ 

While on my desktop, which does have PAE:

  [berrange@camden berrange]$  grep pae /proc/cpuinfo 
  flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
  flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
  flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
  flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
  [berrange@camden berrange]$ 



Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=|

On Wed, May 17, 2006 at 10:49:03PM +0200, Antonio Vargas wrote:

 > >2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
 > >non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
 > >work a lot of earlier PentiumM laptops
 > 
 > xen always needed p6 hardware (didnt run on k6-II for example),
 > and AFAIK p6 supports PAE always, so not any lose here i think

Sadly, not the case..
Celerons, and some Pentium-M''s (as Jeremy noted above) are
P6 class, yet lack PAE.

		Dave

-- 
http://www.codemonkey.org.uk

On Thu, 2006-05-18 at 03:41 +0200, Arjan van de Ven
wrote:
> > xen always needed p6 hardware (didnt run on k6-II for example),
> > and AFAIK p6 supports PAE always, so not any lose here i think
> 
> you lose pentium M''s.
Some of them.  I have a Pentium M that has PAE.  Though I just bought it
about a month ago.

josh

While I understand the development pain, I think it''s important to note
my
case - I currently use Xen in production service (for Wiki server) with an
old Dual PIII. I understand that this will imply that on this specific h/w I
will have to stay with the current kernel versions, right?

Eyal.



-----Original Message-----
From: fedora-xen-bounces@redhat.com [mailto:fedora-xen-bounces@redhat.com]
On Behalf Of Dave Jones
Sent: Wednesday, May 17, 2006 6:49 PM
To: Development discussions related to Fedora Core
Cc: fedora-xen@redhat.com
Subject: [Fedora-xen] Re: Heads-up: Requiring PAE for running Xen

On Wed, May 17, 2006 at 10:49:03PM +0200, Antonio Vargas wrote:

 > >2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
 > >non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
 > >work a lot of earlier PentiumM laptops
 > 
 > xen always needed p6 hardware (didnt run on k6-II for example),
 > and AFAIK p6 supports PAE always, so not any lose here i think

Sadly, not the case..
Celerons, and some Pentium-M''s (as Jeremy noted above) are
P6 class, yet lack PAE.

		Dave

-- 
http://www.codemonkey.org.uk

--
Fedora-xen mailing list
Fedora-xen@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-xen

On Wed, May 17, 2006 at 07:09:10PM -0700, Eyal Traitel wrote:
 > While I understand the development pain, I think it''s important
to note my
 > case - I currently use Xen in production service (for Wiki server) with an
 > old Dual PIII. I understand that this will imply that on this specific h/w
I
 > will have to stay with the current kernel versions, right?

They should have PAE, so you should be ok.

		Dave

-- 
http://www.codemonkey.org.uk

Thanks.
I missed that one - always thought that it''s a new thing.
>From my crappy old server:
==[root@ghostworld ~]# grep pae /proc/cpuinfo
flags           : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 mmx
fxsr sse
flags           : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 mmx
fxsr sse
[root@ghostworld ~]#
== 

-----Original Message-----
From: Dave Jones [mailto:davej@redhat.com] 
Sent: Wednesday, May 17, 2006 7:35 PM
To: Eyal Traitel
Cc: ''Development discussions related to Fedora Core'';
fedora-xen@redhat.com
Subject: Re: [Fedora-xen] Re: Heads-up: Requiring PAE for running Xen

On Wed, May 17, 2006 at 07:09:10PM -0700, Eyal Traitel wrote:
 > While I understand the development pain, I think it''s important
to note
my  > case - I currently use Xen in production service (for Wiki server)
with an  > old Dual PIII. I understand that this will imply that on this
specific h/w I  > will have to stay with the current kernel versions, right?

They should have PAE, so you should be ok.

		Dave

--
http://www.codemonkey.org.uk

On Wednesday 17 May 2006 05:57pm, Daniel P. Berrange
wrote:
> On Wed, May 17, 2006 at 11:42:43PM +0200, Axel Thimm wrote:
[snip]
> > How do I know if my laptop if affected? :)
Do you have a 32bit laptop that can take more than 4GB of RAM?  Not likely.  
Because of the fact that most laptops are not designed to plug that much RAM 
(most today top out at 2GB, even), it seems that most laptop makers do not 
use chips that can support PAE.
> Take a look at the ''flags'' field in /proc/cpuinfo - look
for a flag
> called ''pae'' in it - if your CPU is a Pentium Mobile the
chances are
> you won''t see the ''pae'' flag there.
>
> For example on my laptop which dosn''t have PAE:
>
>   [berrange@localhost ~]$ grep pae /proc/cpuinfo
>   [berrange@localhost ~]$
>
> While on my desktop, which does have PAE:
>
>   [berrange@camden berrange]$  grep pae /proc/cpuinfo
>   flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
I have on rare occasion, seen some Intel chips list pae in their flage even 
though the chip doesn''t support it.  I have also heard (but seen it
myself)
of some boards built for Intel chips which could support pae, but the
board''s
memory controller or number of slots or BIOS (or whatever) prevents it from 
being used (like, not being able to plug in more than 4GB anyway).
-- 
Lamont R. Peterson <lamont@gurulabs.com>
Senior Instructor
Guru Labs, L.C. [ http://www.GuruLabs.com/ ]
GPG Key fingerprint: F98C E31A 5C4C 834A BCAB  8CB3 F980 6C97 DC0D D409

On Wednesday 17 May 2006 12:07pm, Jeremy Katz wrote:
> As we move forward with Xen enablement, there''s a desire for
> being able to access more than 4 gigs of RAM on 32-bit Xen hosts.  The
> options for handling this are
> 1) Another kernel.  This is bad due to
>    a) we''re running out of CD space already
>    b) keeping things matched up between the HV and the guest kernels
>    c) migration is worlds of pain with two types of kernels
> 2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
> non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
> work a lot of earlier PentiumM laptops
> 3) Do nothing, tell people to use 64bit if they want more than 4 gigs of
> RAM
> 4) Make the PAE code handled at runtime.  This is a pretty non-trivial
> amount of work :)
>
> Given these, we''re looking at going with #2 and thus only having
Xen
> work on PAE-capable hardware in the development tree.  And we''re
> planning to try to execute this switchover the beginning of next week.
> Note that this will not affect bare metal installs at all.
Personally, I like 4 better.  In addition, if someone wants to turn on PAE for 
their 32bit Xen boxes, it''s not that hard to rebuild your own kernel
RPMs.
It only takes 6.5 hours on this old-school mobile P4 notebook, and on my Dual 
1.6GHz Athlon, it''s about 1 hour to build a complete set of FC5 kernel 
packages.

However, I have another question about PAE; let''s say I have a box with
2-4GB
RAM and I run a non-PAE kernel (Xen or otherwise, but I am using Xen, of 
course :) ) ... what kind of performance hit should I expect to take from 
turning on PAE?  I''ve heard numbers around 3:1, before, but that
doesn''t
quite sound right to me.

Now, let''s say I scale up the RAM in that box to 8GB after turning on
PAE.
Will there be a difference in memory access performance compared to using PAE 
with 4GB or less?

What about if I two Xen guests that decide to start running some memory 
intensive operations and they just happen to be in different 4GB regions?  
i.e. guest A is in the "lower" 4GB and guest B is in the upper 4GB. 
Would
there be a difference if they were within 4GB of each other.

Lastly, are you talking about 16GB or 64GB support.  As I understand things, 
there are almost no 32bit processors (if any) that can actually use PAE to 
access RAM past 16GB.

Thanks.
-- 
Lamont R. Peterson <lamont@gurulabs.com>
Senior Instructor
Guru Labs, L.C. [ http://www.GuruLabs.com/ ]
GPG Key fingerprint: F98C E31A 5C4C 834A BCAB  8CB3 F980 6C97 DC0D D409

On Wednesday 17 May 2006 09:31pm, Lamont R. Peterson wrote:
[snip]
> Personally, I like 4 better.  In addition, if someone wants to turn on PAE
> for their 32bit Xen boxes, it''s not that hard to rebuild your own
kernel
> RPMs. It only takes 6.5 hours on this old-school mobile P4 notebook, and on
> my Dual 1.6GHz Athlon, it''s about 1 hour to build a complete set
of FC5
> kernel packages.
s/Athlon/Opteron/

Oops.
[snip]
-- 
Lamont R. Peterson <lamont@gurulabs.com>
Senior Instructor
Guru Labs, L.C. [ http://www.GuruLabs.com/ ]
GPG Key fingerprint: F98C E31A 5C4C 834A BCAB  8CB3 F980 6C97 DC0D D409

On Wed, 2006-05-17 at 21:31 -0600, Lamont R. Peterson
wrote:
> Personally, I like 4 better.
Well, yes.  But that requires someone to actually do the work.  And
it''s
*hard*.  And the pragmatic side of things means we need to have
something sooner rather than later
>   In addition, if someone wants to turn on PAE for 
> their 32bit Xen boxes, it''s not that hard to rebuild your own
kernel RPMs.
> It only takes 6.5 hours on this old-school mobile P4 notebook, and on my
Dual
> 1.6GHz Athlon, it''s about 1 hour to build a complete set of FC5
kernel
> packages.
Yes, but then you have to rebuild install trees too.  Your hv + dom0
kernel + domU kernels all have to match.  And that''s not really a
position that can be sanely supported.  If someone reports a bug, how do
you know if it''s due to the fact that they recompiled with PAE[1] or
not
> However, I have another question about PAE; let''s say I have a box
with 2-4GB
> RAM and I run a non-PAE kernel (Xen or otherwise, but I am using Xen, of 
> course :) ) ... what kind of performance hit should I expect to take from 
> turning on PAE?  I''ve heard numbers around 3:1, before, but that
doesn''t
> quite sound right to me.
The numbers shouldn''t be anywhere near that bad.  Also, keep in mind
that until just recently, the smp kernel has required PAE for as long as
we''ve been doing 2.6 kernels.
> Now, let''s say I scale up the RAM in that box to 8GB after turning
on PAE.
> Will there be a difference in memory access performance compared to using
PAE
> with 4GB or less?
This is where you start seeing impact.  What the impact is depends a lot
on your workload, though.[2]
> Lastly, are you talking about 16GB or 64GB support.  As I understand
things,
> there are almost no 32bit processors (if any) that can actually use PAE to 
> access RAM past 16GB.
Xen only currently supports up to 16 GB

Jeremy

[1] Which will tickle different bugs
[2] Based on my understanding from people who have run benchmarks.

On Wed, May 17, 2006 at 02:07:59PM -0400, Jeremy Katz
wrote:
> As we move forward with Xen enablement, there''s a desire for
> being able to access more than 4 gigs of RAM on 32-bit Xen hosts.  The
> options for handling this are
> 1) Another kernel.  This is bad due to 
>    a) we''re running out of CD space already
>    b) keeping things matched up between the HV and the guest kernels
>    c) migration is worlds of pain with two types of kernels
> 2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
> non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
> work a lot of earlier PentiumM laptops
> 3) Do nothing, tell people to use 64bit if they want more than 4 gigs of
> RAM
> 4) Make the PAE code handled at runtime.  This is a pretty non-trivial
> amount of work :)
> 
> Given these, we''re looking at going with #2 and thus only having
Xen
> work on PAE-capable hardware in the development tree.  And we''re
> planning to try to execute this switchover the beginning of next week.
> Note that this will not affect bare metal installs at all.
> 
> Jeremy
Judging from the feedback I would derive that

o in later production environments usually hardware with PAE support
  will be used.

o during development, though, people would like to test xen on their
  non-PAE hardware like their laptops.

So maybe rawhide should continue with both PAE and non-PAE kernels and
decide on dropping the non-PAE when a release is about to be cut?
Otherwise you will keep out a large amount of (admittedly casual)
testers.

And maybe until then the runtime handling emerges out of the blue and
solves all issues. It''s that improbable that it has to happen ;)
-- 
Axel.Thimm at ATrpms.net

On Sat, 2006-05-20 at 16:14 +0200, Axel Thimm wrote:
> On Wed, May 17, 2006 at 02:07:59PM -0400, Jeremy Katz wrote:
> > As we move forward with Xen enablement, there''s a desire for
> > being able to access more than 4 gigs of RAM on 32-bit Xen hosts.  The
> > options for handling this are
> > 1) Another kernel.  This is bad due to 
> >    a) we''re running out of CD space already
> >    b) keeping things matched up between the HV and the guest kernels
> >    c) migration is worlds of pain with two types of kernels
> > 2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
> > non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
> > work a lot of earlier PentiumM laptops
> > 3) Do nothing, tell people to use 64bit if they want more than 4 gigs
of
> > RAM
> > 4) Make the PAE code handled at runtime.  This is a pretty non-trivial
> > amount of work :)
> > 
> > Given these, we''re looking at going with #2 and thus only
having Xen
> > work on PAE-capable hardware in the development tree.  And
we''re
> > planning to try to execute this switchover the beginning of next week.
> > Note that this will not affect bare metal installs at all.
> > 
> > Jeremy
> 
> Judging from the feedback I would derive that
> 
> o in later production environments usually hardware with PAE support
>   will be used.
> 
> o during development, though, people would like to test xen on their
>   non-PAE hardware like their laptops.
> 
> So maybe rawhide should continue with both PAE and non-PAE kernels and
> decide on dropping the non-PAE when a release is about to be cut?
I don''t think so.  I think you missed the "worlds of pain"
part about
having two kernels.  It also becomes a resource issue.

I think option 1 is simply too much burden.  So options 2 and 3 are
left.  It seems to come down to which is the "greater good".  Which
group is larger?  The ones that don''t have PAE hardware, or the ones
that have machines with >= 4 gigs of RAM that are non-64bit.

Personally, I think option 2 is fine.  Of course, both my machines have
PAE :).

josh

On Sat, May 20, 2006 at 09:47:17AM -0500, Josh Boyer
wrote:
> On Sat, 2006-05-20 at 16:14 +0200, Axel Thimm wrote:
> > On Wed, May 17, 2006 at 02:07:59PM -0400, Jeremy Katz wrote:
> > > As we move forward with Xen enablement, there''s a desire
for
> > > being able to access more than 4 gigs of RAM on 32-bit Xen hosts.
The
> > > options for handling this are
> > > 1) Another kernel.  This is bad due to 
> > >    a) we''re running out of CD space already
> > >    b) keeping things matched up between the HV and the guest
kernels
> > >    c) migration is worlds of pain with two types of kernels
> > > 2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
> > > non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
> > > work a lot of earlier PentiumM laptops
> > > 3) Do nothing, tell people to use 64bit if they want more than 4
gigs of
> > > RAM
> > > 4) Make the PAE code handled at runtime.  This is a pretty
non-trivial
> > > amount of work :)
> > > 
> > > Given these, we''re looking at going with #2 and thus
only having Xen
> > > work on PAE-capable hardware in the development tree.  And
we''re
> > > planning to try to execute this switchover the beginning of next
week.
> > > Note that this will not affect bare metal installs at all.
> > > 
> > > Jeremy
> > 
> > Judging from the feedback I would derive that
> > 
> > o in later production environments usually hardware with PAE support
> >   will be used.
> > 
> > o during development, though, people would like to test xen on their
> >   non-PAE hardware like their laptops.
> > 
> > So maybe rawhide should continue with both PAE and non-PAE kernels and
> > decide on dropping the non-PAE when a release is about to be cut?
> 
> I don''t think so.  I think you missed the "worlds of
pain" part about
> having two kernels.  It also becomes a resource issue.
Not within rawhide, or?
> 
> I think option 1 is simply too much burden.  So options 2 and 3 are
> left.  It seems to come down to which is the "greater good". 
Which
> group is larger?  The ones that don''t have PAE hardware, or the
ones
> that have machines with >= 4 gigs of RAM that are non-64bit.
> 
> Personally, I think option 2 is fine.  Of course, both my machines have
> PAE :).
If personal bits matter, then I''d go for 3. I have no 32 bit machine
with >= 4GB, but quite a few 64 bits ones. And the toy machines I
would use to play with rawhide have no PAE. I guess whoever needs that
much memory also needs something like x86_64'' in-chip memory
controller.

(the only systems I''ve recently seen with large memories running on 32
bits were 64-bits platforms with Debian, due to Debian not supporting
multilib ...)
-- 
Axel.Thimm at ATrpms.net

Am Samstag, den 20.05.2006, 16:14 +0200 schrieb Axel Thimm:
> On Wed, May 17, 2006 at 02:07:59PM -0400, Jeremy Katz wrote:
> > As we move forward with Xen enablement, there''s a desire for
> > being able to access more than 4 gigs of RAM on 32-bit Xen hosts.  The
> > options for handling this are
> [...] 
> > 2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
> > non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
> > work a lot of earlier PentiumM laptops
> [...]
> > Given these, we''re looking at going with #2 and thus only
having Xen
> > work on PAE-capable hardware in the development tree.  And
we''re
> > planning to try to execute this switchover the beginning of next week.
> > Note that this will not affect bare metal installs at all.
> [...] 
> So maybe rawhide should continue with both PAE and non-PAE kernels and
> decide on dropping the non-PAE when a release is about to be cut?
> Otherwise you will keep out a large amount of (admittedly casual)
> testers.
Well, I was always against kernel''s in Fedora Extras (and I still am,
[mostly]). But having a Xen non-PAE kernel in Extras sounds like the
proper solution for the above problem. But having kernels in Extras
would only be okay for me if
- they are build with the same spec-file as the other kernels
- they are build on the same build system in the same step as the other
kernels
- they are moved to the proper Extras repo in the same moment as the
other kernels are pushed out

There are some technical problems that probably would need to be solved
before the above could be realized, but that should be possible if we
really want to.

CU
thl
-- 
Thorsten Leemhuis <fedora@leemhuis.info>

On Sat, May 20, 2006 at 05:40:20PM +0200, Thorsten Leemhuis
wrote:
> Am Samstag, den 20.05.2006, 16:14 +0200 schrieb Axel Thimm: 
> > On Wed, May 17, 2006 at 02:07:59PM -0400, Jeremy Katz wrote:
> > > As we move forward with Xen enablement, there''s a desire
for
> > > being able to access more than 4 gigs of RAM on 32-bit Xen hosts.
The
> > > options for handling this are
> > [...] 
> > > 2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
> > > non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
> > > work a lot of earlier PentiumM laptops
> > [...]
> > > Given these, we''re looking at going with #2 and thus
only having Xen
> > > work on PAE-capable hardware in the development tree.  And
we''re
> > > planning to try to execute this switchover the beginning of next
week.
> > > Note that this will not affect bare metal installs at all.
> > [...] 
> > So maybe rawhide should continue with both PAE and non-PAE kernels and
> > decide on dropping the non-PAE when a release is about to be cut?
> > Otherwise you will keep out a large amount of (admittedly casual)
> > testers.
> 
> Well, I was always against kernel''s in Fedora Extras (and I still
am,
> [mostly]). But having a Xen non-PAE kernel in Extras sounds like the
> proper solution for the above problem. But having kernels in Extras
> would only be okay for me if
> - they are build with the same spec-file as the other kernels
> - they are build on the same build system in the same step as the other
> kernels
> - they are moved to the proper Extras repo in the same moment as the
> other kernels are pushed out
> 
> There are some technical problems that probably would need to be solved
> before the above could be realized, but that should be possible if we
> really want to.
Basically you want src.rpms in Core, but a way to move subpackages
from a "Core build" to Extras, correct?

That only covers Jeremy''s "no place on CD" point, what about
migration and maintenance/security etc.?
-- 
Axel.Thimm at ATrpms.net

On Wed, May 17, 2006 at 02:07:59PM -0400, Jeremy Katz wrote:
> 2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
> non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
> work a lot of earlier PentiumM laptops
Will the kernel src.rpm stay supporting building non-PAE kernels (via
changing some macro)?

-- 
--    Jos Vos <jos@xos.nl>
--    X/OS Experts in Open Systems BV   |   Phone: +31 20 6938364
--    Amsterdam, The Netherlands        |     Fax: +31 20 6948204

On Sat, May 20, 2006 at 05:47:40PM +0200, Jos Vos wrote:
> On Wed, May 17, 2006 at 02:07:59PM -0400, Jeremy Katz wrote:
> 
> > 2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
> > non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
> > work a lot of earlier PentiumM laptops
> 
> Will the kernel src.rpm stay supporting building non-PAE kernels (via
> changing some macro)?
That''s doomed to break, reminds me of the switch to allow
kernel-source(code) to build. Once it''s off by default, noone will
look anymore into it, and one day it will be removed as cruft (which
it will have become ;).
-- 
Axel.Thimm at ATrpms.net

Am Samstag, den 20.05.2006, 17:46 +0200 schrieb Axel
Thimm:
> On Sat, May 20, 2006 at 05:40:20PM +0200, Thorsten Leemhuis wrote:
> > Am Samstag, den 20.05.2006, 16:14 +0200 schrieb Axel Thimm: 
> > > On Wed, May 17, 2006 at 02:07:59PM -0400, Jeremy Katz wrote:
> > > > As we move forward with Xen enablement, there''s a
desire for
> > > > being able to access more than 4 gigs of RAM on 32-bit Xen
hosts.  The
> > > > options for handling this are
> > > [...] 
> > > > 2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
> > > > non-laptop hardware, this is a non-issue.  It does mean that
xen won''t
> > > > work a lot of earlier PentiumM laptops
> > > [...]
> > > > Given these, we''re looking at going with #2 and
thus only having Xen
> > > > work on PAE-capable hardware in the development tree.  And
we''re
> > > > planning to try to execute this switchover the beginning of
next week.
> > > > Note that this will not affect bare metal installs at all.
> > > [...] 
> > > So maybe rawhide should continue with both PAE and non-PAE
kernels and
> > > decide on dropping the non-PAE when a release is about to be cut?
> > > Otherwise you will keep out a large amount of (admittedly casual)
> > > testers.
> > 
> > Well, I was always against kernel''s in Fedora Extras (and I
still am,
> > [mostly]). But having a Xen non-PAE kernel in Extras sounds like the
> > proper solution for the above problem. But having kernels in Extras
> > would only be okay for me if
> > - they are build with the same spec-file as the other kernels
> > - they are build on the same build system in the same step as the
other
> > kernels
> > - they are moved to the proper Extras repo in the same moment as the
> > other kernels are pushed out
> > 
> > There are some technical problems that probably would need to be
solved
> > before the above could be realized, but that should be possible if we
> > really want to.
> 
> Basically you want src.rpms in Core, but a way to move subpackages
> from a "Core build" to Extras, correct?
Yes.
> That only covers Jeremy''s "no place on CD" point, what
about
> migration and maintenance/security etc.?
Build from the same specfile in the same step as the other
(kernel-)packages, so maintenance/security is and should still be in the
hands of the core maintainer. 

CU
thl


-- 
Thorsten Leemhuis <fedora@leemhuis.info>

> > > > 1) Another kernel.  This is bad due to
> > > >    a) we''re running out of CD space already
> > > >    b) keeping things matched up between the HV and the guest
kernels
> > > >    c) migration is worlds of pain with two types of kernels
> > > Well, I was always against kernel''s in Fedora Extras
(and I
> > > still am, [mostly]). But having a Xen non-PAE kernel in Extras
> > > sounds like the proper solution for the above problem. But
> > > having kernels in Extras would only be okay for me if
> > Basically you want src.rpms in Core, but a way to move subpackages
> > from a "Core build" to Extras, correct?
> 
> Yes.
> 
> > That only covers Jeremy''s "no place on CD" point,
what about
> > migration and maintenance/security etc.?
> 
> Build from the same specfile in the same step as the other
> (kernel-)packages, so maintenance/security is and should still be in the
> hands of the core maintainer. 
But that''s what the xen fedorians want to avoid, maintaining HV and
migration for several kernels. The CD space argument is only valid for
the time when the next release is to be cut, so it isn''t high priority
right now.
-- 
Axel.Thimm at ATrpms.net

On Sat, 2006-05-20 at 16:14 +0200, Axel Thimm wrote:
> Judging from the feedback I would derive that
> 
> o in later production environments usually hardware with PAE support
>   will be used.
>
> o during development, though, people would like to test xen on their
>   non-PAE hardware like their laptops.
Basically.
> So maybe rawhide should continue with both PAE and non-PAE kernels and
> decide on dropping the non-PAE when a release is about to be cut?
> Otherwise you will keep out a large amount of (admittedly casual)
> testers.
But this then requires doing all of the work to handle the case of both
as well as breaking any sort of upgradeability.  Plus, the testing will
then largely end up on what''s not shipped.  Which isn''t really
what you
want.  Not to mention the last minute name switcheroo (which was quite
nearly a disaster before FC5 :-)
> And maybe until then the runtime handling emerges out of the blue and
> solves all issues. It''s that improbable that it has to happen ;)
If we could only be so lucky... :)

Jeremy

On Sat, 2006-05-20 at 17:26 +0200, Axel Thimm wrote:
> On Sat, May 20, 2006 at 09:47:17AM -0500, Josh Boyer wrote:
> > On Sat, 2006-05-20 at 16:14 +0200, Axel Thimm wrote:
> > > So maybe rawhide should continue with both PAE and non-PAE
kernels and
> > > decide on dropping the non-PAE when a release is about to be cut?
> > 
> > I don''t think so.  I think you missed the "worlds of
pain" part about
> > having two kernels.  It also becomes a resource issue.
> 
> Not within rawhide, or?
There''s still just as much pain needed to do it just in the devel tree.
The only thing that removes is the CD space burden/
> > I think option 1 is simply too much burden.  So options 2 and 3 are
> > left.  It seems to come down to which is the "greater good".
Which
> > group is larger?  The ones that don''t have PAE hardware, or
the ones
> > that have machines with >= 4 gigs of RAM that are non-64bit.
> > 
> > Personally, I think option 2 is fine.  Of course, both my machines
have
> > PAE :).
> 
> If personal bits matter, then I''d go for 3. I have no 32 bit
machine
> with >= 4GB, but quite a few 64 bits ones. And the toy machines I
> would use to play with rawhide have no PAE. I guess whoever needs that
> much memory also needs something like x86_64'' in-chip memory
> controller.
> 
> (the only systems I''ve recently seen with large memories running
on 32
> bits were 64-bits platforms with Debian, due to Debian not supporting
> multilib ...)
Sadly, many people continue to run 32-bit distros even on brand new
hardware due to dependencies on "other stuff"[1]

Jeremy

[1] Think flash and the endless going on about that in 64bit browsers,
or on the even more painful side there are things which require kernel
modules :-/

On Sat, 2006-05-20 at 17:40 +0200, Thorsten Leemhuis
wrote:
> Am Samstag, den 20.05.2006, 16:14 +0200 schrieb Axel Thimm: 
> > On Wed, May 17, 2006 at 02:07:59PM -0400, Jeremy Katz wrote:
> > > As we move forward with Xen enablement, there''s a desire
for
> > > being able to access more than 4 gigs of RAM on 32-bit Xen hosts.
The
> > > options for handling this are
> > [...] 
> > > 2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
> > > non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
> > > work a lot of earlier PentiumM laptops
> > [...]
> > > Given these, we''re looking at going with #2 and thus
only having Xen
> > > work on PAE-capable hardware in the development tree.  And
we''re
> > > planning to try to execute this switchover the beginning of next
week.
> > > Note that this will not affect bare metal installs at all.
> > [...] 
> > So maybe rawhide should continue with both PAE and non-PAE kernels and
> > decide on dropping the non-PAE when a release is about to be cut?
> > Otherwise you will keep out a large amount of (admittedly casual)
> > testers.
> 
> Well, I was always against kernel''s in Fedora Extras (and I still
am,
> [mostly]). But having a Xen non-PAE kernel in Extras sounds like the
> proper solution for the above problem. But having kernels in Extras
> would only be okay for me if
> - they are build with the same spec-file as the other kernels
> - they are build on the same build system in the same step as the other
> kernels
> - they are moved to the proper Extras repo in the same moment as the
> other kernels are pushed out
This involves huge fundamental changes to the build infrastructure that
I''m really not sure are worth doing
> There are some technical problems that probably would need to be solved
> before the above could be realized, but that should be possible if we
> really want to.
The other problem is the kernel binaries *have* to be with the
installer, too -- otherwise, you can''t install the guest as everything
(HV + dom0 kernel + domU kernel) has to either be PAE or not.  Mixing
and matching can''t happen :(

Jeremy

On Mon, May 22, 2006 at 10:51:24AM -0400, Jeremy Katz
wrote:
> > o during development, though, people would like to test xen on their
> >   non-PAE hardware like their laptops.
> 
> Basically.
Once you get into the habit of having your "desktop" as a Xen guest
you
migrate between your desktop and laptop when you travel you''ll stop
seeing it
as a devlopment tool, very fast..

Alan Cox wrote:
> Once you get into the habit of having your "desktop" as a Xen
guest you
> migrate between your desktop and laptop when you travel you''ll
stop seeing it
> as a devlopment tool, very fast.. 
>   
How do you migrate your storage?

Even with gigabit ethernet, copying a 50GB virtual disk can take a long 
time.

-- 
error compiling committee.c: too many arguments to function

Dimi Paun wrote:
> On Mon, 2006-05-22 at 18:24 +0300, Avi Kivity wrote:
>   
>> How do you migrate your storage?
>>     
>
> You don''t just copy it blindly, you can synch it
> differentially, via something like rsync for example.
> I''m sure you can do even better with a special 
> filesystem that can track what changed since last
> synch.
>   
Well, you could have the desktop and laptop in a raid-1 over nbd with a 
write intent bitmap, so that only changed blocks are copied. Seems like 
everything is in place for that.

-- 
error compiling committee.c: too many arguments to function

On Mon, May 22, 2006 at 06:24:07PM +0300, Avi Kivity
wrote:
> Even with gigabit ethernet, copying a 50GB virtual disk can take a long 
> time.
Rsync.

Alan Cox wrote:
> On Mon, May 22, 2006 at 06:24:07PM +0300, Avi Kivity wrote:
>   
>> Even with gigabit ethernet, copying a 50GB virtual disk can take a long
>> time.
>>     
>
> Rsync.
>   
Still have to read every byte on the disk. I doubt you''d get away with 
less than 20 minutes.


-- 
Do not meddle in the internals of kernels, for they are subtle and quick to
panic.

On Mon, May 22, 2006 at 07:48:53PM +0300, Avi Kivity
wrote:
> >Rsync.
> 
> Still have to read every byte on the disk. I doubt you''d get away
with
> less than 20 minutes.
That isn''t a problem I find. It takes me a good hour to pack 8)

On Mon, May 22, 2006 at 10:51:26AM -0400, Jeremy Katz
wrote:
> On Sat, 2006-05-20 at 17:26 +0200, Axel Thimm wrote:
> > On Sat, May 20, 2006 at 09:47:17AM -0500, Josh Boyer wrote:
> > If personal bits matter, then I''d go for 3. I have no 32 bit
machine
> > with >= 4GB, but quite a few 64 bits ones. And the toy machines I
> > would use to play with rawhide have no PAE. I guess whoever needs that
> > much memory also needs something like x86_64'' in-chip memory
> > controller.
> > 
> > (the only systems I''ve recently seen with large memories
running on 32
> > bits were 64-bits platforms with Debian, due to Debian not supporting
> > multilib ...)
> 
> Sadly, many people continue to run 32-bit distros even on brand new
> hardware due to dependencies on "other stuff"[1]
> 
> Jeremy
> 
> [1] Think flash and the endless going on about that in 64bit browsers,
> or on the even more painful side there are things which require kernel
> modules :-/
So it''s Desktop x86_64 users vs us poor ia32-no-PAE laptops. ;)

Anyway, I think you should pick what will make development for you
easier/faster, which is probably more worth than any additional few
random community testers. Maybe I''ll get the chance to upgrade my
laptop and then I''ll make sure for the new one to be PAE-capable
(unless this means too much $$$, but it sounded like any new laptop is
supposed to have PAE).
-- 
Axel.Thimm at ATrpms.net

Alan Cox wrote:
> On Mon, May 22, 2006 at 07:48:53PM +0300, Avi Kivity wrote:
>   
>>> Rsync.
>>>       
>> Still have to read every byte on the disk. I doubt you''d get
away with
>> less than 20 minutes.
>>     
>
> That isn''t a problem I find. It takes me a good hour to pack 8)
>   
Actually rsync is slower than copying. Rsync has to read and write the 
entire image on the target, while a copy only needs to write it.

Rsync is excellent when network bandwidth is low, but with GbE, network 
bandwidth exceeds disk bandwidth.

-- 
error compiling committee.c: too many arguments to function

Dave Jones wrote:
> On Wed, May 17, 2006 at 10:49:03PM +0200, Antonio Vargas wrote:
> 
>  > >2) Switch the 32-bit xen kernels to require PAE.  For most
"current"
>  > >non-laptop hardware, this is a non-issue.  It does mean that xen
won''t
>  > >work a lot of earlier PentiumM laptops
>  > 
>  > xen always needed p6 hardware (didnt run on k6-II for example),
>  > and AFAIK p6 supports PAE always, so not any lose here i think
> 
> Sadly, not the case..
> Celerons, and some Pentium-M''s (as Jeremy noted above) are
> P6 class, yet lack PAE.
Ouch, this just bit my Dell D800 Pentium M 1.3GHz :-(

Obviously there haven''t been Xen kernels available for a while, the 
previous one I had was 2.6.17-1.2307_FC6xen which worked for me, now 
I''ve updated to 2.6.17-1.2462.fc6xen and can''t boot due to
lack of PAE,
presumably this does at least mean fc6t2 is "real close now" ...

I was already considering upgrading to a D820 to get HVM capability once 
the merom CPUs hit the shelves, I guess this will provide additional 
impetus ...