Xen devel - Apr 2008 - Vanilla Linux 64-bit paravirt guest support

Hello,

Can someone tell me what the status of 64-bit paravirtualization support is in 
the vanilla Linux tree?  It seems that 2.6.24 only has 32-bit PV support.  And 
it seems that 2.6.25 will be getting 64-bit PV support.  Will it be complete? 
Some lkml posts seem to suggest that not all hypercalls are supported for
64-bit?

On a related note: is it possible to compile the PV network/block drivers for a 
HVM Linux guest with the newest kernels?  Compilation failed and a previous 
message on the list says that you can''t do that.  The reason
I''d want to do this
is to let me run a 64-bit recent Linux kernel with PV drivers.

Thanks,
Mike

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Michael Abd-El-Malek wrote:
> Hello,
>
> Can someone tell me what the status of 64-bit paravirtualization 
> support is in the vanilla Linux tree?  It seems that 2.6.24 only has 
> 32-bit PV support.  And it seems that 2.6.25 will be getting 64-bit PV 
> support.  Will it be complete?
No, 2.6.25 will not have 64-bit Xen support.  I''m hoping it will be 
ready for .26.
> On a related note: is it possible to compile the PV network/block 
> drivers for a HVM Linux guest with the newest kernels?  Compilation 
> failed and a previous message on the list says that you can''t do 
> that.  The reason I''d want to do this is to let me run a 64-bit
recent
> Linux kernel with PV drivers. 
No, we don''t support that for the moment, though it would be a useful 
thing to have.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge wrote:
> Michael Abd-El-Malek wrote:
>> Can someone tell me what the status of 64-bit paravirtualization 
>> support is in the vanilla Linux tree?  It seems that 2.6.24 only has 
>> 32-bit PV support.  And it seems that 2.6.25 will be getting 64-bit PV 
>> support.  Will it be complete?
> 
> No, 2.6.25 will not have 64-bit Xen support.  I''m hoping it will
be
> ready for .26.
Is 64-bit domU support available anywhere at the moment?  For example, what is 
the status of the git://git.et.redhat.com/xen-pvops-64.git tree?  I pulled that 
tree and tried building 64-bit Xen domU support (since this tree allows you to 
configure the kernel with that capability, unlike the vanilla Linux tree).  But 
compilation failed in enlighten.c because xen_smp_ops isn''t defined in
x86_64.
>> On a related note: is it possible to compile the PV network/block 
>> drivers for a HVM Linux guest with the newest kernels?  Compilation 
>> failed and a previous message on the list says that you can''t
do
>> that.  The reason I''d want to do this is to let me run a
64-bit recent
>> Linux kernel with PV drivers. 
> 
> No, we don''t support that for the moment, though it would be a
useful
> thing to have.
Is there a reason this wouldn''t work, or is it just "scripting
changes" that
need to be done?

Thanks,
Mike

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Michael Abd-El-Malek wrote:
> Is 64-bit domU support available anywhere at the moment?  For example, 
> what is the status of the git://git.et.redhat.com/xen-pvops-64.git 
> tree?  I pulled that tree and tried building 64-bit Xen domU support 
> (since this tree allows you to configure the kernel with that 
> capability, unlike the vanilla Linux tree).  But compilation failed in 
> enlighten.c because xen_smp_ops isn''t defined in x86_64.
Redhat have some patches which they''re shipping in Fedora 9.  Once F9
is
out the door, I''m hoping they''ll polish them into an
upstreamable form.
I don''t know whether that git tree represents what''s in F9, or
if that''s
somewhere else; at the very least I''d expect you''d be able to
pull the
patches out of the srpm.
>>> On a related note: is it possible to compile the PV network/block 
>>> drivers for a HVM Linux guest with the newest kernels?  Compilation
>>> failed and a previous message on the list says that you
can''t do
>>> that.  The reason I''d want to do this is to let me run a
64-bit
>>> recent Linux kernel with PV drivers. 
>>
>> No, we don''t support that for the moment, though it would be a
useful
>> thing to have.
>
> Is there a reason this wouldn''t work, or is it just
"scripting
> changes" that need to be done? 
In order to simplify the initial upstreaming work, I removed the support 
for pv-in-hvm, since it adds a moderate amount of extra complexity.  
We''d need to put that back in - it shouldn''t be too hard, but
it hasn''t
been a priority so far.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, 2008-04-10 at 09:01 -0500, Jeremy Fitzhardinge
wrote:
> Michael Abd-El-Malek wrote:
> > Is 64-bit domU support available anywhere at the moment?  For example,
> > what is the status of the git://git.et.redhat.com/xen-pvops-64.git 
> > tree?  I pulled that tree and tried building 64-bit Xen domU support 
> > (since this tree allows you to configure the kernel with that 
> > capability, unlike the vanilla Linux tree).  But compilation failed in
> > enlighten.c because xen_smp_ops isn''t defined in x86_64.
Try building without CONFIG_SMP, it doesn''t support that yet.
> Redhat have some patches which they''re shipping in Fedora 9.  Once
F9 is
> out the door, I''m hoping they''ll polish them into an
upstreamable form.
> I don''t know whether that git tree represents what''s in
F9, or if that''s
> somewhere else; at the very least I''d expect you''d be
able to pull the
> patches out of the srpm.
Yep, this tree:

  http://git.et.redhat.com/?p=xen-pvops-64.git

is the work-in-progress x86_64 tree.

This tree:

  http://git.et.redhat.com/?p=linux-2.6-fedora-pvops.git

is what we''re actually shipping for F-9. It includes the x86_64 work,
but some other paravirt_ops patches too, most of which are queued up
upstream.

Cheers,
Mark.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Mark McLoughlin wrote:
> s what we''re actually shipping for F-9. It includes the x86_64
work,
> but some other paravirt_ops patches too, most of which are queued up
> upstream.
What else do you have?  Hm.  I was really hoping to drop /proc/xen/...

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Apr 10, 2008, at 10:11 AM, Mark McLoughlin wrote:
> On Thu, 2008-04-10 at 09:01 -0500, Jeremy Fitzhardinge wrote:
>> Michael Abd-El-Malek wrote:
>>> Is 64-bit domU support available anywhere at the moment?  For  
>>> example,
>>> what is the status of the git://git.et.redhat.com/xen-pvops-64.git
>>> tree?  I pulled that tree and tried building 64-bit Xen domU
support
>>> (since this tree allows you to configure the kernel with that
>>> capability, unlike the vanilla Linux tree).  But compilation  
>>> failed in
>>> enlighten.c because xen_smp_ops isn''t defined in x86_64.
>
> Try building without CONFIG_SMP, it doesn''t support that yet.
Other than SMP support, does the tree represent a fully functional 64- 
bit PV domU support? Does it also allow all hypercalls?  Put another  
way: is a 64-bit PV domU from that tree less capable than a 64-bit PV  
domU from Xen''s linux-2.6.18.8 tree?
>> Redhat have some patches which they''re shipping in Fedora 9. 
Once
>> F9 is
>> out the door, I''m hoping they''ll polish them into an
upstreamable
>> form.
>> I don''t know whether that git tree represents what''s
in F9, or if
>> that''s
>> somewhere else; at the very least I''d expect you''d be
able to pull
>> the
>> patches out of the srpm.
>
> Yep, this tree:
>
>  http://git.et.redhat.com/?p=xen-pvops-64.git
>
> is the work-in-progress x86_64 tree.
>
> This tree:
>
>  http://git.et.redhat.com/?p=linux-2.6-fedora-pvops.git
>
> is what we''re actually shipping for F-9. It includes the x86_64
work,
> but some other paravirt_ops patches too, most of which are queued up
> upstream.
Which tree do you recommend I use?

Thanks,
Mike

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, 2008-04-10 at 09:25 -0500, Jeremy Fitzhardinge
wrote:
> Mark McLoughlin wrote:
> > s what we''re actually shipping for F-9. It includes the
x86_64 work,
> > but some other paravirt_ops patches too, most of which are queued up
> > upstream.
> 
> What else do you have?  Hm.  I was really hoping to drop /proc/xen/...
Okay, here''s the list:

    xen x86_64: Initial x86_64 support for Xen paravirt_ops

Eduardo''s x86_64 stuff

    xen x86_64: Add 64 bit version of privcmd_hypercall()
    xen: Add Xen''s /sys/hypervisor interface
    xen: Add /proc/xen/xenbus
    xen: Add /proc/xen/privcmd
    xen: Add /proc/xen/capabilities
    xen: Add empty xenctrl module

This is /proc/xen and /sys/hypervisor stuff pulled from the dom0 tree.
In that tree we''ve got mostly full compatibility with the
linux-2.6.18-xen userspace interfaces AFAIR.

I probably don''t need all of these for DomU, but I went for the safe
bet
of including anything that userspace might be relying on.

Happy for us to drop /proc/xen in the future - this is just a "get
things going" thing.

    xen debug: Add xprintk to log directly via hypercall

Just for debugging

    xen: Add a vmlinuz target

bzImage is the way forward, but for now it was easier to just do stick
with vmlinuz - e.g. until there''s support for bzImage Dom0 and we get
newer Xen HV and userspace in Fedora 

    xen blkfront: Delay wait for block devices until after the disk is added.
    xen: Add compatibility aliases for frontend drivers
    xen: Module autoprobing support for frontend drivers
    xen: Enable Xen console by default in domU
    xen pvfb: Para-virtual framebuffer, keyboard and pointer driver
    xen: Make xen-blkfront write its protocol ABI to xenstore
    xen: Do not pin/unpin PMD pages

All queued upstream

    xen x86_64: Only define load_user_cs_desc() on 32 bit
    xen execshield: fix endless GPF fault loop
    xen execshield: Add xen-specific load_user_cs_desc()
    squashfs: Fix build without CONFIG_SMP

Fedora specific (e.g. execshield)

Cheers,
Mark.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, 2008-04-10 at 10:32 -0400, Michael Abd-El-Malek
wrote:
> On Apr 10, 2008, at 10:11 AM, Mark McLoughlin wrote:
> > On Thu, 2008-04-10 at 09:01 -0500, Jeremy Fitzhardinge wrote:
> >> Michael Abd-El-Malek wrote:
> >>> Is 64-bit domU support available anywhere at the moment?  For
> >>> example,
> >>> what is the status of the
git://git.et.redhat.com/xen-pvops-64.git
> >>> tree?  I pulled that tree and tried building 64-bit Xen domU
support
> >>> (since this tree allows you to configure the kernel with that
> >>> capability, unlike the vanilla Linux tree).  But compilation  
> >>> failed in
> >>> enlighten.c because xen_smp_ops isn''t defined in
x86_64.
> >
> > Try building without CONFIG_SMP, it doesn''t support that yet.
> 
> Other than SMP support, does the tree represent a fully functional 64- 
> bit PV domU support? 
No, it''s a work-in-progress - ia32 emulation is also missing and
we''re
tracking down a nasty pagetable pinning bug atm.

Even then it would only be on par with 32-bit pv_ops DomU, which itself
doesn''t yet have all the Xen features of 2.6.18 tree.
> Does it also allow all hypercalls?  Put another  
> way: is a 64-bit PV domU from that tree less capable than a 64-bit PV  
> domU from Xen''s linux-2.6.18.8 tree?
It depends on how you define "less capable" - e.g. some might think a
tree based on a 1.5 year old kernel  is less capable even if it does
have more xen features ... :-)
> >> Redhat have some patches which they''re shipping in Fedora
9.  Once
> >> F9 is
> >> out the door, I''m hoping they''ll polish them
into an upstreamable
> >> form.
> >> I don''t know whether that git tree represents
what''s in F9, or if
> >> that''s
> >> somewhere else; at the very least I''d expect
you''d be able to pull
> >> the
> >> patches out of the srpm.
> >
> > Yep, this tree:
> >
> >  http://git.et.redhat.com/?p=xen-pvops-64.git
> >
> > is the work-in-progress x86_64 tree.
> >
> > This tree:
> >
> >  http://git.et.redhat.com/?p=linux-2.6-fedora-pvops.git
> >
> > is what we''re actually shipping for F-9. It includes the
x86_64 work,
> > but some other paravirt_ops patches too, most of which are queued up
> > upstream.
> 
> Which tree do you recommend I use?
I''d use the xen-pvops-64 tree unless you are specifically wanting to
help with Fedora''s kernel-xen packages.

Cheers,
Mark.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Michael Abd-El-Malek wrote:
> On Apr 10, 2008, at 10:11 AM, Mark McLoughlin wrote:
>> On Thu, 2008-04-10 at 09:01 -0500, Jeremy Fitzhardinge wrote:
>>> Michael Abd-El-Malek wrote:
>>>> Is 64-bit domU support available anywhere at the moment?  For
example,
>>>> what is the status of the
git://git.et.redhat.com/xen-pvops-64.git
>>>> tree?  I pulled that tree and tried building 64-bit Xen domU
support
>>>> (since this tree allows you to configure the kernel with that
>>>> capability, unlike the vanilla Linux tree).  But compilation
failed in
>>>> enlighten.c because xen_smp_ops isn''t defined in
x86_64.
>>
>> Try building without CONFIG_SMP, it doesn''t support that yet.
>
> Other than SMP support, does the tree represent a fully functional 
> 64-bit PV domU support? Does it also allow all hypercalls?  Put 
> another way: is a 64-bit PV domU from that tree less capable than a 
> 64-bit PV domU from Xen''s linux-2.6.18.8 tree?
The 32-bit domU implementation is less capable than 2.6.18-xen.  For 
example, it does not yet support suspend/resume/migrate or ballooning, 
and a number of interesting features are not yet in a released kernel 
(pvfb).  Further out I hope we''ll get things like dom0 support in as
well.

(I''m not sure what you mean by "allow all hypercalls"; do you
mean "all
features"?)

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Mark McLoughlin wrote:
> On Thu, 2008-04-10 at 09:25 -0500, Jeremy Fitzhardinge wrote:
>   
>> Mark McLoughlin wrote:
>>     
>>> s what we''re actually shipping for F-9. It includes the
x86_64 work,
>>> but some other paravirt_ops patches too, most of which are queued
up
>>> upstream.
>>>       
>> What else do you have?  Hm.  I was really hoping to drop /proc/xen/...
>>     
>
> Okay, here''s the list:
>
>     xen x86_64: Initial x86_64 support for Xen paravirt_ops
>
> Eduardo''s x86_64 stuff
>
>     xen x86_64: Add 64 bit version of privcmd_hypercall()
>     xen: Add Xen''s /sys/hypervisor interface
>     xen: Add /proc/xen/xenbus
>     xen: Add /proc/xen/privcmd
>     xen: Add /proc/xen/capabilities
>     xen: Add empty xenctrl module
>
> This is /proc/xen and /sys/hypervisor stuff pulled from the dom0 tree.
> In that tree we''ve got mostly full compatibility with the
> linux-2.6.18-xen userspace interfaces AFAIR.
>
> I probably don''t need all of these for DomU, but I went for the
safe bet
> of including anything that userspace might be relying on.
>
> Happy for us to drop /proc/xen in the future - this is just a "get
> things going" thing.
>   
I''d rather fix userspace now, rather than let anyone get the impression
that /proc/xen/ is going to be a supported interface.  We''ll probably 
have to end up making /proc/xen/ exist for backwards compat, but we 
should make it clear that it isn''t the canonical path.

Not that I''ve thought about where it should be.  /sys/hypervisor/xen?  
xendom0fs?

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

This is a fix to typo in p2m.c. Courtesy to Xiao Wang
(sirouni@yahoo.com.cn) for pointing out this bug.


==================diff -r 8d750b7acfa3 xen/arch/x86/mm/p2m.c
--- a/xen/arch/x86/mm/p2m.c     Thu Apr 10 11:11:25 2008 +0100
+++ b/xen/arch/x86/mm/p2m.c     Thu Apr 10 04:13:59 2008 -0500
@@ -941,7 +941,7 @@ void p2m_change_type_global(struct domai
                     mfn = l1e_get_pfn(l1e[i1]);
                     gfn = get_gpfn_from_mfn(mfn);
                     /* create a new 1le entry with the new type */
-                    flags = p2m_flags_to_type(nt);
+                    flags = p2m_type_to_flags(nt);
                     l1e_content = l1e_from_pfn(mfn, flags);
                     paging_write_p2m_entry(d, gfn, &l1e[i1],
                                            l1mfn, l1e_content, 1);



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Apr 10, 2008, at 10:57 AM, Jeremy Fitzhardinge wrote:
> Michael Abd-El-Malek wrote:
>> On Apr 10, 2008, at 10:11 AM, Mark McLoughlin wrote:
>>> On Thu, 2008-04-10 at 09:01 -0500, Jeremy Fitzhardinge wrote:
>>>> Michael Abd-El-Malek wrote:
>>>>> Is 64-bit domU support available anywhere at the moment? 
For
>>>>> example,
>>>>> what is the status of the
git://git.et.redhat.com/xen-pvops-64.git
>>>>> tree?  I pulled that tree and tried building 64-bit Xen
domU
>>>>> support
>>>>> (since this tree allows you to configure the kernel with
that
>>>>> capability, unlike the vanilla Linux tree).  But
compilation
>>>>> failed in
>>>>> enlighten.c because xen_smp_ops isn''t defined in
x86_64.
>>>
>>> Try building without CONFIG_SMP, it doesn''t support that
yet.
>>
>> Other than SMP support, does the tree represent a fully functional  
>> 64-bit PV domU support? Does it also allow all hypercalls?  Put  
>> another way: is a 64-bit PV domU from that tree less capable than a  
>> 64-bit PV domU from Xen''s linux-2.6.18.8 tree?
>
> The 32-bit domU implementation is less capable than 2.6.18-xen.  For  
> example, it does not yet support suspend/resume/migrate or  
> ballooning, and a number of interesting features are not yet in a  
> released kernel (pvfb).  Further out I hope we''ll get things like
> dom0 support in as well.
For my purposes, suspend/resume/migrate isn''t required.  But I might  
need ballooning.  Would I be able to take the balloon driver from  
2.6.18.8 and use it in those new trees?  The new tree seems to have  
all the hypercalls and defines the increase_reservation/ 
decrease_reservation op codes.  I can add the stub code for these calls.
> (I''m not sure what you mean by "allow all hypercalls";
do you mean
> "all features"?)
I was thinking along the lines of grant hypercalls for mapping another  
domU''s grants.  HVM can''t let me do that.  But I need to be
able to do
this for my work.

Thanks,
Mike

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Michael Abd-El-Malek wrote:
> For my purposes, suspend/resume/migrate isn''t required.  But I
might
> need ballooning.  Would I be able to take the balloon driver from 
> 2.6.18.8 and use it in those new trees?  The new tree seems to have 
> all the hypercalls and defines the 
> increase_reservation/decrease_reservation op codes.  I can add the 
> stub code for these calls.
I have pending patches for balloon support in x86.git 
(http://people.redhat.com/mingo/x86.git/README)
>> (I''m not sure what you mean by "allow all
hypercalls"; do you mean
>> "all features"?)
>
> I was thinking along the lines of grant hypercalls for mapping another 
> domU''s grants.  HVM can''t let me do that.  But I need to
be able to do
> this for my work. 
If you have a kernel driver, then you can use those hypercalls.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, Apr 10, 2008 at 10:04:31AM -0500, Jeremy Fitzhardinge
wrote:
> Not that I''ve thought about where it should be. 
/sys/hypervisor/xen?
Yes.
> xendom0fs?
xen_hypfs to follow the namingscheme used for the other implementation:

| # grep s390 /proc/filesystems
| nodev   s390_hypfs

It is not restricted to dom0.

Bastian

-- 
Conquest is easy. Control is not.
		-- Kirk, "Mirror, Mirror", stardate unknown

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, 2008-04-10 at 15:11 +0100, Mark McLoughlin
wrote:
> On Thu, 2008-04-10 at 09:01 -0500, Jeremy Fitzhardinge wrote:
> > Michael Abd-El-Malek wrote:
> > > Is 64-bit domU support available anywhere at the moment?  For
example,
> > > what is the status of the
git://git.et.redhat.com/xen-pvops-64.git
> > > tree?  I pulled that tree and tried building 64-bit Xen domU
support
> > > (since this tree allows you to configure the kernel with that 
> > > capability, unlike the vanilla Linux tree).  But compilation
failed in
> > > enlighten.c because xen_smp_ops isn''t defined in x86_64.
> 
> Try building without CONFIG_SMP, it doesn''t support that yet.
> 
> > Redhat have some patches which they''re shipping in Fedora 9. 
Once F9 is
> > out the door, I''m hoping they''ll polish them into an
upstreamable form.
> > I don''t know whether that git tree represents what''s
in F9, or if that''s
> > somewhere else; at the very least I''d expect you''d
be able to pull the
> > patches out of the srpm.
> 
> Yep, this tree:
> 
>   http://git.et.redhat.com/?p=xen-pvops-64.git
> 
> is the work-in-progress x86_64 tree.
FYI, here''s the config options that''s currently needed to get
this tree
working:

# CONFIG_SMP is not set
# CONFIG_NUMA is not set
# CONFIG_NEED_MULTIPLE_NODES is not set

# x86_64 breaks with a different CONFIG_PHYSICAL_START, currently
CONFIG_PHYSICAL_START=0x200000

# x86_64 breaks with CONFIG_SPARSEMEM_VMEMMAP, currently
# CONFIG_SPARSEMEM_VMEMMAP is not set

# 32-bit emulation isn''t ready yet
# CONFIG_IA32_EMULATION is not set

Cheers,
Mark.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel