Xen devel - Nov 2005 - struct shared_info extensibility (or lack thereof)

While looking at "steal" cpu time accounting, I am running
into problems with struct shared_info.  Right now it looks
like I need to modify struct shared_info, but this is not
possible in a backwards compatible way.

I think we should change struct shared_info to be extensible,
so we can expand it at a future date without breaking the
interface.  Right now (before 3.0) would be the best time to
change the data structure to something extensible.

I am thinking it should look something like this:

struct shared_info {
	single variables;
	padding[];
	{ vcpu_data, vcpu_time, padding[] }  [ MAX_VIRT_CPUS ];
}

The reason for putting the vcpu_* structures together on a per
virtual cpu basis is that this way we can extend the number of
virtual CPUs in the future, without breaking the interface with
older guests.

The current stared_info struct of two [MAX_VIRT_CPUS] arrays
followed by the single variables is just impossible to extend
in the future.

Any objections to changing the interface now ?

-- 
All Rights Reversed

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 28 Nov 2005, at 16:47, Rik van Riel wrote:
> The reason for putting the vcpu_* structures together on a per
> virtual cpu basis is that this way we can extend the number of
> virtual CPUs in the future, without breaking the interface with
> older guests.
>
> The current stared_info struct of two [MAX_VIRT_CPUS] arrays
> followed by the single variables is just impossible to extend
> in the future.
MAX_VIRT_CPUS is already quite big (32). Increasing it to the next 
power-of-two will overflow the shared_info page. So, if/when we want to 
do that we''ll place the extra vcpu info in a separate shared page.

Given how close 3.0.0 is now, I don''t think this is a good change to be
making.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 28 Nov 2005, at 16:47, Rik van Riel wrote:
> The reason for putting the vcpu_* structures together on a per
> virtual cpu basis is that this way we can extend the number of
> virtual CPUs in the future, without breaking the interface with
> older guests.
Actually, thinking about it, putting all vcpu info in a 64-byte block 
makes sense for cache locality....

I don''t think that relocating the vcpu info at the end of the page buys
us anything significant though.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, 28 Nov 2005, Keir Fraser wrote:
> On 28 Nov 2005, at 16:47, Rik van Riel wrote:
> 
> > The reason for putting the vcpu_* structures together on a per
> > virtual cpu basis is that this way we can extend the number of
> > virtual CPUs in the future, without breaking the interface with
> > older guests.
> 
> Actually, thinking about it, putting all vcpu info in a 64-byte block 
> makes sense for cache locality....
> 
> I don''t think that relocating the vcpu info at the end of the page
buys
> us anything significant though.
It would allow us to grow the maximum supported number of
VCPUs in the future, without breaking backwards compatibility.

Of course, it would be useful to then put a few longs worth
of padding in front of the VCPU array, so we also have expansion
space in the non-per-VCPU part of shared_info.

Two or three longs worth of padding per VCPU would be great, too. ;)

Having an interface that can be extended compatibly in the
future will make everybody''s life so much easier.  Improving
Xen without breaking the already deployed virtual machines.

-- 
All Rights Reversed

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 28 Nov 2005, at 18:01, Rik van Riel wrote:
> It would allow us to grow the maximum supported number of
> VCPUs in the future, without breaking backwards compatibility.
Any extension in the future can easily be made to backwards support old 
OSes, although they will be restricted to 32 VCPUs.
> Of course, it would be useful to then put a few longs worth
> of padding in front of the VCPU array, so we also have expansion
> space in the non-per-VCPU part of shared_info.
Well, I''ll leave vcpu_info where it is. We could maybe add some padding
directly after it, but not sure it''s worth it.
> Two or three longs worth of padding per VCPU would be great, too. ;)
Six bytes plus another eight bytes are easily available (padding 
vcpu_info up to 64 bytes).
> Having an interface that can be extended compatibly in the
> future will make everybody''s life so much easier.  Improving
> Xen without breaking the already deployed virtual machines.
Sure, but extending MAX_VIRT_CPUS to 64 will require a big change one 
way or another. Either shared_info becomes multi-page, or we explicitly 
put extra vcpu_info''s on a separate page. Probably we will do the 
latter if required.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel